Control processing device

What is AI technical title?
AI technical title is built by Patsnap AI team. It summarizes the technical point description of the patent document.
The control processing device addresses the limitations of QR code-based advertising metrics by using sound data analysis to generate more detailed indicators, improving the measurement of user interactions without user intervention.

JP2026007425APending Publication Date: 2026-01-16BROTHER KOGYO KK

View PDF 1 Cites 0 Cited by

Patent Information

Application Number: JP2024107236
Authority / Receiving Office: JP · JP
Patent Type: Applications
Current Assignee / Owner
Filing Date: 2024-07-03
Publication Date: 2026-01-16

AI Technical Summary

Technical Problem

Existing advertising effectiveness measurement systems fail to capture sufficient indicators due to the time-consuming nature of QR code reading, limiting the number of measurable interactions.

Method used

A control processing device connected to a printing device and an electro-acoustic transducer that detects and analyzes sound data after printing to generate index information related to advertisements, without requiring user interaction.

Benefits of technology

Enables the acquisition of more comprehensive advertising indicators by analyzing sound data post-printing, enhancing the measurement of user interactions beyond QR code-based methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

Figure 2026007425000001_ABST

Patent Text Reader

Abstract

To acquire more indexes related to advertisement printing.SOLUTION: A terminal 300 and a voice and emotion analysis server 700, which are connected to a printer 200 and a smart speaker 600 and include a control unit 310, execute a process of acquiring a voice datum from the smart speaker 600, a process of determining whether or not a datum corresponding to a predetermined keyword group is included in the voice datum after the printer 200 starts printing an advertisement, and a process of generating indicator information related to the advertisement according to the keyword when it is determined that the datum corresponding to the keyword is included.SELECTED DRAWING: Figure 4

Need to check novelty before this filing date? Find Prior Art

Description

[Technical Field]

[0001] The present invention relates to a control processing device connected to a printing device and an electro-acoustic transducer. [Background technology]

[0002] Conventionally, as described in Patent Document 1, for example, a technique is known in which a QR code (registered trademark) included in a printed advertisement is read and accesses to a web page on a network are counted to obtain metrics related to the advertisement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-198069 Summary of the Invention [Problem to be solved by the invention]

[0004] In the advertising effectiveness measurement system described in Patent Document 1, unless a user reads the QR code (registered trademark) on a printed advertisement, it is not counted as an index. Since the operation of reading a QR code (registered trademark) is time-consuming for users, it is thought that there will be a certain number of users who read an advertisement but do not go as far as to read the QR code (registered trademark). As a result, there is a limit to the number of advertising-related indexes that can be obtained, and there has been a problem in that a sufficient number cannot be obtained.

[0005] An object of the present disclosure is to provide a control processing device that can acquire more indicators related to advertising printing. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention provides a control processing device connected to a printing device and an electro-acoustic transducer and having a control unit, wherein the control unit executes an acquisition process for acquiring sound data from the electro-acoustic transducer, a judgment process for determining whether the sound data acquired in the acquisition process after the printing device starts printing an advertisement contains predetermined specific sound data, and a generation process for generating index information related to the advertisement in accordance with the specific sound data if the judgment process determines that the specific sound data is included.

[0007] The control processing device of the present invention is connected to a printing device and an electroacoustic transducer. When an advertisement is printed by the printing device, sound is detected by the electroacoustic transducer and corresponding sound data is output. The control unit acquires the sound data through an acquisition process, determines whether the sound data includes specific sound data through a determination process, and if the specific sound data is included, generates index information related to the advertisement through a generation process.

[0008] In the present invention, the indicator information is generated based on the sound generated after the start of printing the advertisement, so the indicator information can be acquired without any special operation by the user. According to the present invention, more indicators related to advertisement printing can be acquired than when counting is performed based on reading a QR code (registered trademark). [Effects of the Invention]

[0009] According to the present invention, it is possible to obtain more indicators related to advertisement printing. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram illustrating an example of the overall configuration of an information distribution system including a control processing device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a terminal device. [Figure 3] FIG. 1 is a block diagram illustrating an example of a smart speaker. [Figure 4]2 is a diagram showing the flow of information and processing contents between devices included in the information distribution system. FIG. [Figure 5] FIG. 10 is a diagram showing an example of the content of an advertising image printed on an advertising flyer. [Figure 6] FIG. 10 is a diagram illustrating classification of keywords. DETAILED DESCRIPTION OF THE INVENTION

[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described with reference to the accompanying drawings. In this embodiment, the present invention is applied to an information distribution system that distributes advertising information from a server to a printer and prints it.

[0012] <Overall configuration of information distribution system> FIG. 1 shows an example of the overall configuration of an information distribution system 1 including a processing control device according to an embodiment. As shown in FIG. 1, the information distribution system 1 includes an advertisement provision server (SSP: Supply Side Platform) 100, a printer 200, a terminal device 300, a user management server (DMP: Data Management Platform) 400, an advertisement management server (DSP: Demand Side Platform) 500, a smart speaker 600, and a voice and emotion analysis server 700. The servers 100, 400, 500, and 700, the printer 200, the terminal device 300, and the smart speaker 600 each include a communication unit and can communicate with each other via a network NT. The network NT includes at least one of the Internet, a local area network (LAN), and a wide area network (WAN). The communication unit may include multiple types of communication circuits for connecting to the network NT. In this case, devices connected to the network NT may communicate with multiple other devices using different communication circuits. In this embodiment, the multiple types of communication circuits are collectively referred to as a communication unit.

[0013] In this embodiment, the printer 200 is an example of a printing device, the terminal device 300 is an example of an external device, the smart speaker 600 is an example of an electro-acoustic transducer, and the terminal device 300, the smart speaker 600, and the voice / emotion analysis server 700 are examples of a control processing device.

[0014] <User management server (DMP)> The user management server 400 manages information about users via the terminal device 300. The user management server 400 includes a control unit, a storage unit, a communication unit, and the like, all of which are not shown.

[0015] <Advertising Server (SSP)> The advertisement providing server 100 is a server installed and managed by, for example, the manufacturer of the printer 200. The advertisement providing server 100 receives an advertisement image and a predetermined group of keywords (described later) from the advertisement management server 500 at a predetermined timing. The advertisement providing server 100 transmits the acquired advertisement image and group of keywords to the terminal device 300 as described later in this embodiment, and then causes the terminal device 300 to distribute the advertisement image to the printer 200 for printing.

[0016] In detail, the advertisement providing server 100 has a control unit 110, a storage unit 115, a communication unit 140, etc. The control unit 110, the storage unit 115, the communication unit 140, etc. are connected via a bus 105 so as to be able to transmit and receive information. The storage unit 115 is configured by, for example, a RAM, a ROM, an EEPROM, an HDD, etc. The storage unit 115 stores various programs 120 and various data 130. The various programs 120 include programs for executing advertising content control processing 121 and advertising distribution processing 122, which will be described later, and the various data 130 includes a terminal ID of the terminal device 300, a user ID, an advertising image, etc.

[0017] The control unit 110 is a device that performs data processing, and is, for example, a processor such as a CPU. The control unit 110 executes various programs stored in the storage unit 115. The control unit 110 performs various processes, including data communication with the printer 200, terminal device 300, user management server 400, and advertisement management server 500, which are connected to the network NT. The communication unit 140 includes hardware such as a NIC (Network Interface Card) for connecting to Ethernet, is connected to the network NT, and communicates with the printer 200, terminal device 300, user management server 400, and advertisement management server 500.

[0018] <Advertising management server (DSP)> The advertisement management server 500 transmits an advertisement image to be printed by the printer 200 to the advertisement providing server 100. The advertisement management server 500 has a control unit, a storage unit, a communication unit, etc. (not shown). In this example, the communication unit receives a bid request from the advertisement providing server 100. Terminal devices or servers (not shown) of multiple advertisers are connected to the network NT. The control unit of the advertisement management server 500 holds an auction in response to the bid request for advertisement images submitted by the advertisers, and acquires suitable advertisement images through bids in the auction. When a successful bid that meets the specified conditions of the auction is made, the control unit of the advertisement management server 500 acquires the corresponding advertisement image and keyword group from the terminal device, etc., of the successful advertiser. The acquired advertisement image and keyword group are transmitted from the advertisement management server 500 to the advertisement providing server 100 via the communication unit, and then further transmitted from the advertisement providing server 100 to the terminal device 300 and the printer 200 for printing. The advertisement providing server 100 may acquire the advertisement image from the advertisement management server 500, or the advertisement management server 500 may acquire the advertisement image from another external server and transmit it to the advertisement providing server 100.

[0019] <Printer> A printer ID is assigned to the printer 200. The printer 200 includes a control unit 210, a memory unit 215, a display unit 240, an operation unit 250, a communication unit 260, and a printing unit 270. The control unit 210, the memory unit 215, the display unit 240, the operation unit 250, the communication unit 260, and the printing unit 270 are connected via a bus 205 so as to be able to send and receive information.

[0020] The storage unit 215 is configured with, for example, RAM, ROM, EEPROM, HDD, etc. The storage unit 215 stores various programs 220 and various data 230. The various programs 220 include, for example, a program for executing control content of the printer 200. The various data 230 include a printer ID and an advertisement image received from the advertisement providing server 100, acquired from a terminal device of a successful bidder among the advertisers, etc., and transmitted from the advertisement providing server 100. At least a portion of the above various data may be stored in the storage unit 115 of the advertisement providing server 100. Alternatively, at least a portion of the original data may be stored in one of the storage unit 215 of the printer 200 and the storage unit 115 of the advertisement providing server 100, and copy data of the original data may be stored in the other one of them.

[0021] The control unit 210 is a device that processes data and is, for example, a processor such as a CPU. The control unit 210 executes various programs stored in the memory unit 215. The display unit 240 is, for example, a liquid crystal display and is capable of displaying various information. The operation unit 250 is, for example, a keyboard, buttons, etc., and accepts input operations by the user. The user can input various instructions to the printer 200 by operating the operation unit 250. The communication unit 260 is connected to the network NT and is capable of communication with the advertisement providing server 100, the advertisement management server 500, the user management server 400, and the terminal device 300. The printing unit 270 includes a print engine and forms and prints an image on the printing paper, for example, by an inkjet method, while removing and transporting printing paper from a paper tray using a transport mechanism (not shown).

[0022] The cartridge holder 280 is configured to be able to house cartridges 285. The cartridges 285 are configured with cartridges of multiple ink colors, such as cyan, magenta, yellow, and black. The cartridge holder 280 is provided with an ink amount sensor (not shown) that can detect the remaining amount or consumed amount of each of the cartridges of the multiple ink colors using a known method.

[0023] The printing unit 270 performs printing processing while consuming ink from the cartridge 285. The reader / writer 290 is provided in the cartridge holder 280. The reader / writer 290 reads and writes cartridge information from an IC chip provided in the cartridge 285 housed in the cartridge holder 280. For example, the reader / writer 290 detects the type of cartridge 285 housed in the cartridge holder 280 by reading the cartridge information from the IC chip. Note that the amount of ink used or remaining in the cartridge 285 may be detected by the above reading by the reader / writer 290.

[0024] <Terminal Device> The terminal device 300 is, for example, a portable terminal such as a notebook PC owned by a user, and in this example, is connected to the network NT via wireless communication. The terminal device 300 may be any appropriate terminal other than the printer 200, and may be, in addition to a notebook PC, for example, a smartphone, a tablet PC, or a desktop PC. Alternatively, the terminal device 300 may be a portable terminal or a fixed terminal provided in a karaoke booth or various stores for selecting songs, selecting menus, browsing products, etc.

[0025] 2, the terminal device 300 has a control unit 310, a storage unit 315, a display unit 340, an operation unit 350, a communication unit 360, etc. The storage unit 315 stores programs for executing a speaker control process 321 and a print control process 322, which will be described later, and various data 330. The control unit 310 executes the various programs 321 and 322 stored in the storage unit 315. A user can input various instructions to the terminal device 300 by operating the operation unit 350.

[0026] <Smart speaker> The smart speaker 600 is a multi-function speaker that responds to questions and instructions uttered by a user through voice recognition using so-called artificial intelligence (AI) technology. As shown in FIG. 3, the smart speaker 600 includes a control unit 610, a storage unit 615, a microphone 640, a speaker 650, a communication unit 660, and the like.

[0027] The storage unit 615 stores programs for executing voice signal processing 621, voice input / output processing 622, keyword extraction processing 623, and keyword determination processing 624 (described later), as well as various data 630 including a keyword database 631. The control unit 610 executes the various programs 621, 622, 623, and 624 stored in the storage unit 615. Questions and commands uttered by the user are electrically input by a microphone 640 and converted into digital data by the voice input / output processing 622, and then the voice signal processing 621 generates voice data in a predetermined data format based on the voice data. Note that this voice data is an example of sound data, and the voice signal processing 621 is an example of acquisition processing. The keyword extraction processing 623 performs voice recognition and morphological analysis on the content recorded in the voice data to extract morphemes (words), and the keyword determination processing 624 determines whether a specific keyword is included from the many extracted morphemes. The voice signal processing 621 and keyword determination processing 624 register and refer to keywords in the keyword database 631, and control various devices and output voices from the speaker 650 in response to questions and commands from the user.

[0028] <Speech and emotion analysis server> The voice and emotion analysis server 700 performs voice recognition and morphological analysis on the content of the received voice data to extract morphemes (words), and analyzes the user's emotions based on the relationship between the many extracted morphemes and a specific group of keywords received in advance. The voice and emotion analysis server 700 has a control unit, a storage unit, a communication unit, etc. (not shown), and in this example, receives the voice data and group of keywords from the terminal device 300 via the communication unit.

[0029] Each of the above servers 100, 400, 500, and 700 may be configured as a single "server," or may be configured as multiple servers. Alternatively, at least two of the functions of the servers 100, 400, 500, and 700 may be provided in a single "server." Here, the control processing device may be the control unit 110 or the control unit 120. It may also be a control unit (not shown) possessed by the user management server 400, or a control unit (not shown) possessed by the advertisement management server 500. Furthermore, the control processing device is not limited to a single control unit, and may be one in which processing is shared among multiple control units.

[0030] <Features of the embodiment> As described above, in this embodiment, an advertising image is delivered to the printer 200 at a predetermined timing, and the advertisement is printed. At this time, advertisers have a strong desire to obtain indicator information related to the advertisement, such as the impression that users had when viewing the advertisement. However, until now, such indicator information was obtained only when the user actively responded, and therefore it was not possible to obtain a sufficient number of indicator information. A feature of this embodiment is that it obtains more indicators related to advertisement printing. The details of this process are explained below with reference to the drawings.

[0031] <Information flow and processing content in this embodiment> 4 is a diagram showing the flow of information and processing contents between the devices included in the information distribution system 1 of this embodiment. In this example of this embodiment, it is assumed that the index information related to the distributed advertisement is obtained as a result of sentiment analysis performed on the content of comments made by users who have viewed the advertisement.

[0032] First, as shown in the upper right corner of the figure, an advertising image and a group of keywords are input in advance by the advertiser into the advertising management server 500 (DSP), and when a successful bid that meets the specified conditions of the auction is made, the advertising image and group of keywords are sent to the advertising provision server 100 (SSP).

[0033] In the advertisement providing server 100, the advertisement content control unit 101 generates print data to be printed by the printer 200 based on the received advertisement image, and the advertisement distribution unit 102 transmits the print data and a group of keywords corresponding to the image content to the terminal device 300. The advertisement content control unit 101 and the advertisement distribution unit 102 are functional units that are realized when the control unit 110 executes the content control process 121 and the advertisement distribution process 122 stored in the storage unit 115, respectively.

[0034] The terminal device 300 transmits the print data received by the print control unit 302 to the printer 200, which prints the data as an advertisement flyer. When the printer 200 completes printing the advertisement flyer, it returns a print completion notification to the advertisement providing server 100 via the print control unit 302 of the terminal device 300.

[0035] In addition, in the terminal device 300, the smart speaker control unit 301 receives the keyword group, and at that time the smart speaker control unit 301 sends a command requesting data acquisition to the smart speaker 600. The smart speaker control unit 301 and the print control unit 302 are functional units that are realized when the control unit 310 executes the speaker control process 321 and the print control process 322 stored in the storage unit 315, respectively.

[0036] When the smart speaker 600 receives a data acquisition request from the terminal device 300, the audio input / output unit 602 converts the audio waveform signal electrically collected via the microphone 640 into digital data by A / D conversion or the like. The audio processing control unit 601 then generates the audio data in a predetermined data format and transmits it to the terminal device 300. It is desirable that the timing at which the audio input / output unit 602 starts and stops acquiring the audio waveform signal be appropriately set according to the purpose of the audio collection. In this embodiment, since the purpose is to obtain emotion analysis results from the content of comments made by users who have viewed the advertisement, as described above, it is assumed that audio is collected from the time the printer 200 starts printing the flyer advertisement until a predetermined time after the printing is completed. The audio processing control unit 601 and the audio input / output unit 602 are functional units realized when the control unit 610 executes the audio signal processing 621 and the audio input / output processing 622 stored in the storage unit 615, respectively.

[0037] The terminal device 300 receives the voice data transmitted from the smart speaker 600 at the smart speaker control unit 301 of the terminal device 300, and then transmits this to the voice and emotion analysis server 700 together with a group of keywords that were previously received.

[0038] The voice and emotion analysis server 700 performs voice recognition on the received voice data to convert what the user has said into a text data string, and then performs morphological analysis on the text data string to convert it into a data string of morphemes, which are semantically divided into the smallest units.The server then compares this morpheme data string with a group of keywords received separately, and performs predetermined calculations on the relationships between them to analyze how the user felt about the flyer advertisement.

[0039] The above-mentioned speech recognition may be performed by waveform analysis using appropriate known techniques such as so-called artificial intelligence (AI techniques such as RNN), and detailed description thereof will be omitted here. Note that this speech recognition is an example of extraction processing, and the text data string is an example of sound data. The above-mentioned morphological analysis may be performed using appropriate known techniques such as so-called natural language processing. Regarding emotion analysis, appropriate known techniques such as rule-based natural language processing (described later) or so-called artificial intelligence (AI techniques using attention, transformer, etc.) may be used to determine the relationship with the received keyword group, and a score may be output as an index value in the form of a single scalar value, for example. The emotion analysis results thus output are then transmitted to the smart speaker control unit 301 of the terminal device 300.

[0040] The smart speaker control unit 301 of the terminal device 300 transmits the received emotion analysis results as they are to the user management server 400, and the emotion analysis results are received by the customer behavior acquisition unit 401 of the user management server 400. The received emotion analysis results are classified into predetermined categories such as by user or by advertisement content via the customer behavior history control unit 402, and are registered / updated in the behavior history database 403.

[0041] As described above, by accessing the behavior history database 403 of the user management server 400, the advertiser can easily know the analysis results of the feelings that users have toward the flyer advertisement in the form of quantitative index values.

[0042] The information processing control executed after the print data for the flyer advertisement and the keyword group are output from the advertisement providing server 100 is summarized below. The control unit performs a printing process in which the print data is printed by a printer, and a sound collection process in which audio is collected from a microphone in accordance with the printing timing and an audio waveform signal is generated. Next, the control unit performs a sound data generation process in which audio data is generated from the audio waveform signal, a text data string conversion process in which the audio data is converted into a text data string using speech recognition, a morpheme data string conversion process in which the text data string is converted into a morpheme data string using morpheme analysis, an emotion scoring process in which an index value is calculated for the morpheme data string using artificial intelligence technology or keyword matching, and an analysis result registration process in which an index value resulting from the emotion analysis is registered in a database. Here, the sound data generation process, text data string conversion process, morpheme data string conversion process, emotion scoring process, and analysis result registration process may be performed after the printing process and the audio collection process are completed, or may be performed in parallel.

[0043] Of these, in the example steps shown in FIG. 4, the text data string conversion step, the morpheme data string conversion step, and the emotion scoring step are executed by the external speech and emotion analysis server 700, but if the specifications of the terminal device 300 are high, all of these steps may be executed by the terminal device 300.

[0044] In addition, although only the sound collection process and the voice data generation process are performed in the smart speaker 600, as described above, the smart speaker 600 is originally capable of performing the text data string conversion process and the morpheme data string conversion process by keyword extraction process and keyword determination process. Therefore, the smart speaker 600 may also be responsible for outputting the morpheme data string corresponding to the content of the user's speech.

[0045] Alternatively, if the terminal device 300 itself is equipped with a microphone 640 and has high processing power specifications, all processes from the sound collection process to the emotion scoring process may be performed by the terminal device 300 alone. In this case, both the smart speaker 600 and the voice and emotion analysis server 700 are not required.

[0046] Alternatively, the user may not own the terminal device 300, and instead a separate external distribution server with functions equivalent to the smart speaker control unit 301 and print control unit 302 described above may access and control the user's printer 200 and smart speaker 600.

[0047] <About the content of flyer advertisements and keyword groups and emotional scoring> An example of the case where the emotion scoring process is performed using a rule-based natural language processing technique will be described below. Fig. 5 shows an example of the content of an advertising image printed on an advertising flyer. In the illustrated example, a rectangular piece of paper is oriented horizontally, and is divided into thirds horizontally. An advertisement for a training gym ("ABC Muscle" in the illustrated example) is placed in the left column, an advertisement for a beverage ("DEF Drink" in the illustrated example) is placed in the upper part of the center column, an advertisement for a rental shop ("JKL Music" in the illustrated example) is placed in the lower part of the center column, an advertisement for a cram school ("GHI Cram School" in the illustrated example) is placed in the upper part of the right column, and an advertisement for a pharmacy ("MNO Pharmacy" in the illustrated example) is placed in the lower part of the right column.

[0048] As shown in this example, multiple advertisements may be printed on a flyer at once, and the combination of these advertisements is determined by the status of successful bids in the auction in the advertisement management server 500. Note that one advertisement may be printed alone depending on the status of successful bids, etc. The group of keywords output from the advertisement management server 500 via the advertisement providing server 100 corresponds to all advertisements to be displayed in the advertisement image of the flyer advertisement.

[0049] Here, a keyword group is a collection of word data (word data) that includes any words or combinations of words specified by the advertiser, and each keyword can be classified into the categories shown in Figure 6 based on its content. Keywords can be broadly classified into general words and related words. General words are commonly used nouns, pronouns, verbs, auxiliary verbs, adjectives, and adverbs, and are words used to understand the meaning of user comments expressed in morpheme data strings. These general words further include the intermediate categories of action words and emotion words. These are particularly important words for verifying how users reacted to flyer advertisements.

[0050] Action words are words that express the user's intention to take a specific action after seeing a flyer advertisement. In this example, the action words include subcategories of purchasing behavior words and pre-purchase behavior words. Examples of purchasing behavior words include "I'll buy," "I want it," and "Let's take a look." Examples of pre-purchase behavior words include "preview," "window shopping," "search online," "go to the store," "order online," "delivery," and "dot com." If it is determined that a morphological data string contains many of these purchasing behavior words or pre-purchase behavior words, this can serve as indicator information indicating that the user was interested enough to take some kind of action that led to purchasing behavior. These action words are an example of action word data, and the resulting indicator information is an example of the sixth type of information.

[0051] Emotional words express the emotions felt by users who saw the advertisement flyer. The emotional words in this example include subcategories of positive and negative words. Positive words expressing positive emotions include, for example, "looks good," "usable," "cheap," "good job," "should buy," "cool," and "this is acceptable." Purchasing behavior words may also be used as positive words. Negative words expressing negative emotions include, for example, "no good," "useless," "too expensive," "not cool," "this isn't available," "impossible," and "impossible." Knowing the prevalence of these positive and negative words in a morpheme data string can provide indicator information indicating whether the user felt a favorable or unfavorable impression toward the advertisement. Words that negatively suggest purchasing behavior, such as "won't buy," "don't want," and "don't need," may also be included. These emotional words are an example of emotional word data. Indicative information indicating a favorable impression felt by the user based on the positive words is an example of seventh information, and indicator information indicating a negative impression felt by the user based on the negative words is an example of eighth information.

[0052] The other major category, related words, are words related to the names and proper nouns directly written in the flyer advertisement. These related words further include the medium categories of service name / product name words, distributor name words, and advertisement words. These are particularly important words for verifying which of the multiple advertisements in the flyer advertisement the user is paying attention to. If it is found that a morphological data string contains many related words, this can be used as indicator information that indicates that the user paid attention to the corresponding advertisement. This indicator information is an example of the fifth information.

[0053] Service name and product name words, which are one type of related words, are words that represent the names of services and products presented in the advertisement or proper nouns. For example, in the advertisement example shown in Figure 5, these words include "gym," "orange," "rental," "cram school," and "PQ cold medicine." If it turns out that a morphological data string contains many of these service name and product name words, this can be used as an indicator that the user in question paid attention to the corresponding advertisement. Note that these service name and product name words are an example of provided word data.

[0054] The source name word, which is one of the related words, is a word that represents the name or trade name of the source (advertiser) that distributed the advertisement. For example, in the advertisement example shown in Figure 5, these words include "ABC Muscle," "DEF Drink," "GHI Cram School," "JKL Music," and "MNO Pharmacy." If it is found that many of these source name words are included in the morpheme data string, this can be indicator information that indicates that the user paid attention to the corresponding advertisement. Here, for well-known products or services, the source name word is not necessarily included in the advertisement, but the source name word may be a word that is not included in the advertisement. Note that this source name word is an example of name word data.

[0055] Advertisement words, which are one type of related word, are general or abstract names written in the advertising title field of a flyer advertisement. For example, in the example advertisement shown in Figure 5, these words include "unlimited use," "personal trainer," "new release," "20% off," "elementary school student," "350 yen," and "DVD." If it is determined that a morpheme data string contains many of these advertisement words, this can serve as indicator information that indicates that the user paid attention to the corresponding advertisement. Here, advertisement words are words written in the advertisement, and it is preferable that they can identify which of multiple advertisements the words refer to. Therefore, they may also include service name / product name words and distributor name words. Note that these advertisement words are an example of described word data.

[0056] The advertiser pre-enters a group of keywords arbitrarily selected for each category along with the advertisement image into the advertisement management server 500. In this example, emotion scoring in the speech and emotion analysis server uses rules to verify whether a group of keywords is included in a morpheme data string representing the user's speech content, the number of times the keywords are included, and the relationships between the keywords via particles and other elements. This allows for quantitative scoring of the emotion analysis of the user's interest in each advertisement, based on a positive or negative level. Furthermore, the accuracy of the verification can be further improved by using a so-called polarity dictionary, which stores the degree of positivity and negativity of each morpheme and their respective weightings. The specific emotion scoring method can be implemented using any known technology, and detailed description thereof will be omitted here. The speech data corresponding to the group of keywords is an example of specific sound data. The process of determining whether a group of keywords is included in a morpheme data string is an example of a determination process. The process of scoring the index value is an example of a generation process. The process of storing the number of times a keyword is included in a morpheme data string is an example of a storage process. The number of times the keyword is included is an example of fourth information.

[0057] <Effects of the embodiment> As described above, the smart speaker 600 acquires the sound at least after the advertisement has been printed by the printer 200. Then, it is determined whether the acquired sound data contains a specific sound. More specifically, it is determined whether the acquired sound data contains specific sound data. Then, by determining whether the specific sound data is included, it is possible to acquire an index related to the printing of the advertisement.

[0058] According to the above description, it is possible to obtain index information based on the sound generated after the start of printing an advertisement, without the user having to perform any operation. According to this embodiment, it is possible to obtain more indexes related to advertisement printing than when index information is obtained in response to some user operation, such as when counting based on the reading of a QR code (registered trademark). Specifically, it is possible to obtain indexes related to advertisement printing for more users. Furthermore, it is possible to obtain more indexes related to advertisement printing for a single user.

[0059] Furthermore, according to the above explanation, the voice / emotion analysis server 700 acquires voice data from the smart speaker 600, determines whether the voice data contains specific sound data corresponding to a group of keywords, and generates index information related to the advertisement, but this is not limited to this.

[0060] For example, the functions of acquiring voice data, determining whether the voice data contains specific sound data corresponding to a keyword group, and generating index information related to advertisements may be installed in printer 200. This makes it possible to acquire more indexes related to advertising printing without providing a server or the like.

[0061] Furthermore, for example, the functions of acquiring voice data, determining whether the voice data contains specific sound data corresponding to a keyword group, and generating index information related to advertisements may be installed in the terminal device 300 or a single server. This makes it possible to acquire more indexes related to advertisement printing using an external device separate from the printer 200.

[0062] Furthermore, for example, the functions of acquiring voice data, determining whether the voice data contains specific sound data corresponding to a keyword group, and generating index information related to advertising may be installed in the smart speaker 600. This makes it possible to acquire more indexes related to advertising printing without providing a server or the like.

[0063] Furthermore, for example, the functions of acquiring voice data, determining whether the voice data contains specific sound data corresponding to a keyword group, and generating index information related to advertisements may be performed by multiple devices. For example, these functions may be divided between the smart speaker 600 and the voice and emotion analysis server 700. In this way, by dividing the functions appropriately and installing them in the smart speaker 600, the voice and emotion analysis server 700, etc., the degree of freedom in system configuration for achieving the above-mentioned effects is increased.

[0064] In this embodiment, the printer 200 acquires audio data for a predetermined period after the printer 200 starts printing an advertisement. By limiting the audio data to data acquired within the predetermined period, unnecessary data processing, such as when audio data is acquired indefinitely, can be prevented. This also reduces wasteful storage space. This also reduces the possibility of incorrectly recognizing sounds unrelated to the advertisement as specific sound data. This also reduces the possibility of incorrectly recognizing specific sound data related to the currently printed advertisement as specific sound data related to the next advertisement to be printed. Audio data may also be acquired during the period until the next advertisement is printed, thereby avoiding duplicate acquisition of audio data between different flyer advertisements. Audio data may also be acquired within a predetermined period after printing is completed, or within a predetermined period after detecting a user's voice. Sequentially discarding audio data stored in the terminal device 300 or the like after analysis can contribute to privacy and security measures and reduce storage space.

[0065] Also, the voice data may be acquired after the printer 200 starts printing the advertisement and recognizes the user's voice (a specific activation command to the smart speaker 600 or just a live voice). In this way, by limiting the voice data to the time period when the user is likely to view the advertisement, it is possible to prevent unnecessary processing of data.

[0066] In this embodiment, when the smart speaker 600 detects sound and outputs corresponding voice data, the voice and emotion analysis server 700 extracts a text data string and a morpheme data string corresponding to the user's speech from the acquired voice data, and determines whether the morpheme data string contains a keyword group. This embodiment can detect whether the user uttered a word containing a specific keyword group, thereby obtaining more detailed indicators related to advertising printing. While the smart speaker 600 is described here as an example of an electro-acoustic transducer, this is not limiting. The electro-acoustic transducer may be a microphone included in the printer 200 or a microphone included in the terminal device 300. More specifically, it may be a microphone included in a smartphone or a microphone included in a PC.

[0067] The keywords to be detected are not limited to clear words, but may be judged using intonation and the rhythmic intervals of non-verbal sounds such as groans, moans, and sighs.

[0068] In this embodiment, the keyword group is data registered by the advertisement distributor. By registering keywords themselves to check the effectiveness of their advertisements, the distributor can detect whether or not the user uttered those words and obtain more detailed indicators related to the advertisement printing. However, keywords that are common to all advertisements may be set regardless of whether they are registered words.

[0069] Furthermore, in this embodiment, when it is determined that a keyword is included in the morpheme data string, the fact that a corresponding specific keyword is included is stored. This makes it possible to determine not only whether a specific keyword was included, but also what keywords were included and how many of them were included. In addition, index information such as the number of times a keyword was extracted can be output to the advertisement distributor as feedback, making it possible to confirm what keywords were able to appeal to the user's feelings.

[0070] Furthermore, in this embodiment, the keyword group is related words that represent words related to the contents of the advertisement, and whether or not the related words are included is determined, and index information is generated as to whether or not the user has viewed the printed flyer advertisement. Index information as to whether or not the user has viewed the advertisement is very useful, and is information that the advertisement distributor wants to know. If related words that represent words related to the contents of the advertisement are detected from the sound data, it is highly likely that the user has viewed the advertisement, and therefore index information as to whether or not the user has viewed the advertisement is generated based on this. This makes it possible to provide more accurate feedback to the distributor that it is highly likely that the user viewed the advertisement.

[0071] In addition, in this embodiment, the advertisement has multiple ad space provided by multiple distribution sources, and it is determined which of the multiple ad space contains the related word, and index information is generated regarding which ad space the user viewed.

[0072] As described above, an advertisement may contain ad space from multiple publishers. While the publisher placing the advertisement considers it important to know whether at least a portion of the advertisement has been viewed by a user, the publisher considers it even more important to know whether the advertisement it placed among the multiple advertisements has been viewed. By determining which of the multiple ad space sections contained related words and generating index information regarding which of the ad space sections the user viewed, the publisher can more accurately provide feedback to the publisher that the user is likely to have viewed the publisher's advertisement.

[0073] In this embodiment, the related words include advertising words that represent the wording in the advertisement. This makes it possible to provide, as indicator information, for example, that the user is likely to be interested in the content of the wording in the advertisement based on the advertising words. It is also possible to provide, as indicator information, what advertising words are likely to catch the user's eye, what advertising words are likely to attract the user's interest, and so on.

[0074] In this embodiment, the related words include source name words that indicate the name of the advertisement source written in the advertisement. This makes it possible to provide, for example, indicator information on whether the user recognized the name of the advertisement source written in the advertisement based on the source name words.

[0075] In this embodiment, the related words include service name and product name words related to the service or product provided by the distributor of the advertisement, as described in the advertisement. This makes it possible to provide indicator information, for example, that the user may be interested in the content of the service or product described in the advertisement, based on the service name and product name words. Note that it may also include services that do not clearly offer any products.

[0076] Furthermore, in this embodiment, the keywords include action words that indicate the intention of a user who has viewed the advertisement to take a predetermined action. The system determines whether the action words are included in the morpheme data string and generates index information indicating whether the user is likely to take an action. Based on the action words, it is possible to provide index information such as whether the user is likely to take some kind of action based on the printed advertisement. For example, if an action word such as "search online" is extracted, there is a high possibility that the printed advertisement will trigger online access. Furthermore, if an action word such as "go to the store" is extracted, there is a high possibility that the user will visit the store where they will make a purchase. By providing feedback to the distributor that these action words have been detected, the distributor can learn about user behavior after viewing the advertisement and more specifically understand the effectiveness of the advertisement.

[0077] Furthermore, in this embodiment, the action is a purchasing behavior or a purchasing preparation behavior by the user, and the action word includes a purchasing behavior word or a purchasing preparation behavior word. This makes it possible to detect whether the user is likely to purchase the product or service related to the advertisement. For example, if a purchasing behavior word such as "let's buy" is extracted, the user is likely to take action to purchase. Furthermore, if a purchasing behavior word such as "order online" is extracted, the user is likely to take action to purchase through online shopping. By feeding back to the distributor that these purchasing behavior words have been detected, the distributor can know that the user is likely to take purchasing behavior after viewing the advertisement, and can more specifically know the effectiveness of the advertisement.

[0078] In this embodiment, the keywords include emotion words that express the emotions of users who viewed the advertisement. Emotion scoring determines whether emotion words are included in the morpheme data string and generates index information regarding whether the user felt a favorable impression of the advertisement. This makes it possible to provide index information, such as whether a user who viewed a printed advertisement felt a favorable impression of the advertisement, based on the emotion words. Note that the emotion words may be collated using words listed in a publicly known polarity dictionary that is commonly distributed and used, for example.

[0079] In this embodiment, the emotion words include positive words that express positive emotions expressed by the user, making it possible to detect positive words spoken by the user after viewing the advertisement.

[0080] Furthermore, in this embodiment, emotion scoring determines whether emotion words are included in the morpheme data string and generates index information regarding whether the user felt negative emotions toward the advertisement. This makes it possible to provide index information, such as whether a user who viewed a printed advertisement felt negative emotions toward the advertisement, based on the emotion words.

[0081] In this embodiment, the emotion words include negative words that indicate negative emotions expressed by the user, making it possible to detect negative words spoken by the user after viewing the advertisement.

[0082] The present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the spirit and technical concept of the present invention. Such modifications will be described below in order. The same reference numerals will be used to designate parts equivalent to those in the above-described embodiment, and descriptions thereof will be omitted or simplified as appropriate.

[0083] (1) When generating index information based on the printing sound of a printer In the above embodiment, the smart speaker 600 determines whether a specific sound is included in the collected sound, but the present invention is not limited to this. Another important indicator is whether the user is able to view the advertisement as planned.

[0084] Therefore, the specific sound data included in the sound collected by the smart speaker 600 is not limited to the user's speech, but may be printing sound data representing the sound of the printing operation of the printer 200. Then, the printing sound data is extracted from the sound data, and index information is generated from the printing sound data indicating at least one of whether the printer 200 has started printing an advertisement and whether it has finished printing the advertisement. This index information is an example of first information.

[0085] As a result, in this modified example, it is possible to provide, as indicator information, information such as whether printing of an advertisement has started or whether printing of the advertisement has finished, based on the printing sound data. Note that if the sound data does not contain printing sound data, indicator information is generated based on the determination that printing was not possible. Examples of sounds during the printing operation include the driving sound of the paper feed rollers, the driving sound of the transport rollers that transport paper, the driving sound of the carriage, error notification sounds such as out of paper, out of ink, and jam, and no sound.

[0086] (2) Generating index information based on abnormal sounds from a printer The specific sound data included in the sound collected by the smart speaker 600 may be abnormal sound data that represents the sound that occurs when an abnormality occurs in the printing operation of the printer 200. Then, the abnormal sound data is extracted from the sound data, and index information regarding whether the printer 200 failed to print the advertisement is generated based on whether the abnormal sound data is included. Note that this index information is an example of second information.

[0087] As a result, in this modified example, it is possible to provide, as indicator information, information such as the fact that some kind of abnormality has occurred in the printer 200 and that printing of the advertisement has failed, based on the abnormal sound data. One example of an abnormal sound is the sound of the paper feed rollers spinning when the paper feed tray runs out of paper, and if such abnormal sound data is included in the sound data, indicator information is generated based on the determination that printing was not possible. Other examples of abnormal sounds include information such as the absence of driving sounds from the transport rollers that transport paper, the absence of driving sounds from the carriage, the occurrence of error alarm sounds such as out of paper, out of ink, or a jam, or the absence of sound.

[0088] (3) Generating index information based on the sound of picking up a flyer advertisement The specific sound data included in the sound collected by the smart speaker 600 may be paper sound data representing the sound emitted when the user picks up the flyer advertisement. Then, the paper sound data is extracted from the sound data, and index information regarding whether the user picked up the printed advertisement is generated by determining whether the paper sound data is included. This index information is an example of third information.

[0089] As a result, in this modification, it is possible to provide information such as whether a user has picked up a printed advertisement as index information based on the paper sound data.

[0090] Furthermore, in the above, the control process shown in FIG. 4 does not limit the present invention to the procedures shown in the above processes, and steps may be added or deleted or the order may be changed within the scope that does not deviate from the spirit and technical idea of the invention.

[0091] In addition to the above, the methods according to the above embodiments and modifications may be used in appropriate combination.

[0092] Although not specifically illustrated, the present invention can be implemented with various modifications within the scope of the invention. [Explanation of symbols]

[0093] 1. Information distribution system 100 Advertisement providing server (example of distribution providing server) 200 Printer (an example of a printing device) 300 Terminal device (an example of an external device) 400 User Management Server 500 Advertising management server (example of distribution management server) 600 Smart speaker (an example of an electroacoustic transducer) 700 Voice and emotion analysis server (an example of an external device)

Claims

1. A control processing device connected to the printing device and the electro-acoustic transducer, the control processing device having a controller, The control unit an acquisition process for acquiring sound data from the electroacoustic transducer; a determination process for determining whether or not predetermined specific sound data is included in the sound data acquired in the acquisition process after the printing device starts printing the advertisement; a generating process for generating index information related to the advertisement in accordance with the specific sound data when it is determined in the determining process that the specific sound data is included; A control processing device that executes the above.

2. The control processing device The printer is equipped with 2. The control processing device according to claim 1.

3. The control processing device An external device connected to the printing device 2. The control processing device according to claim 1.

4. The control processing device The electroacoustic transducer is mounted 2. The control processing device according to claim 1.

5. The control processing device At least two of the printing device, an external device connected to the printing device, and the electro-acoustic transducer are mounted separately according to their functions.