Recognition determination device, recognition determination method, and recognition determination program

Abstract

To provide a recognition determination device, a recognition determination method, and recognition determination program capable of accurately determining whether or not information is recognized by a user.SOLUTION: A recognition determination device includes: a presentation information supply part configured to supply the presentation information which a presentation part present is made to present; a behavior information acquisition part configured to acquire behavior information related to user behavior being a presentation target of the presentation information; and a determination part configured to determine whether or not the presentation information is recognized by the user based on contents of the presentation information and contents of the behavior information while the presentation information is presented.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a recognition determination device, a recognition determination method, and a recognition determination program for determining whether target information has been recognized by a user. [Background technology]

[0002] Generally, in the advertising industry, advertising agencies create various advertisements at the request of advertisers and distribute the advertisements through various media, and receive payment from the advertisers for distributing the advertisements.

[0003] Furthermore, with the spread of internet advertising, contingency-based advertising is becoming more common. Contingency-based advertising is an advertising transaction mechanism in which, for example, if the advertisement distribution triggers a product purchase, the advertiser can receive payment for the distribution of the advertisement.

[0004] For example, Patent Document 1 discloses an information processing device that generates viewing information indicating the time an advertising image is viewed by a user, and a server device that uses the viewing information to calculate the amount to be charged to an advertiser. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 5245691 Summary of the Invention [Problem to be solved by the invention]

[0006] In recent years, internet advertising has begun to spread through media such as in-vehicle devices like car navigation systems and mobile devices like smartphones. Therefore, it is expected that performance-based advertising will continue to spread in the future in conjunction with various devices. For performance-based advertising to become widespread, it is essential to accurately determine whether an advertisement has been recognized by a user.

[0007] Furthermore, the presented advertisements are not limited to simple advertisements, and may contain information other than advertisements. For example, when a car navigation system is used as an advertising medium while moving, the advertisement may contain, in addition to the content of the advertisement, driving support information related to the location where the advertisement should be displayed (for example, information informing the driver when the displayed location is a location where caution is required, such as a location where accidents frequently occur).

[0008] It is important that all information, including a wide variety of advertisements, is recognized by users. Therefore, when target information is presented, it is preferable to accurately determine whether the presented information has been recognized by the user.

[0009] The present invention has been made in consideration of the above-mentioned points, and one of its objectives is to provide a recognition determination device, a recognition determination method, and a recognition determination program that can accurately determine whether information has been recognized by a user. [Means for solving the problem]

[0010] The invention described in claim 1 is characterized by having a presentation information supply unit that supplies presentation information to be presented by the presentation unit, a behavioral information acquisition unit that acquires behavioral information regarding the behavior of a user to whom the presentation information is to be presented, and a determination unit that determines whether the presentation information has been recognized by the user based on the content of the presentation information and the content of the behavioral information while the presentation information is being presented. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram of a mobile object including a recognition determination device according to a first embodiment and a server communicatively connected to the mobile object; [Figure 2] 1 is a block diagram of a recognition determination device according to a first embodiment. [Figure 3] FIG. 4 is a flowchart of an information presentation process in the recognition determination device according to the first embodiment. [Figure 4]FIG. 3 is a flowchart of a recognition determination process in the recognition determination device according to the first embodiment. [Figure 5] FIG. 3 is a diagram illustrating a recognition determination table in the recognition determination device according to the first embodiment. [Figure 6] FIG. 10 is a block diagram of a server including a recognition determination device according to a first modification of the first embodiment, and a mobile object communicatively connected to the server. [Figure 7] FIG. 10 is a block diagram of a mobile object including a recognition determination device according to a second modification of the first embodiment and a server connected to the mobile object for communication. DETAILED DESCRIPTION OF THE INVENTION

[0012] Examples of the present invention will be described in detail below. [Example]

[0013] 1 is a block diagram showing a mobile object MO equipped with a recognition determination device 10 according to a first embodiment, and a server 60 communicatively connected to the mobile object MO. The recognition determination device 10, the mobile object MO, and the server 60 will be outlined with reference to FIG. 1. The mobile object MO is, for example, an automobile, a motorcycle, a ship, or an aircraft. In this embodiment, a case will be described in which the mobile object MO is a vehicle such as an automobile.

[0014] First, the moving object MO will be described. The moving object MO has a presentation unit 20 capable of presenting various information. The presentation unit 20 includes a display 21 capable of displaying various information as images, and a speaker 22 capable of outputting various information as sounds.

[0015] Next, the moving object MO has a user monitoring device 30 that monitors the behavior and speech of the occupants (driver and passengers) of the moving object MO, i.e., the behavior of the user of the moving object MO. The user monitoring device 30 acquires user behavior information, which is information related to the behavior of the user, and voice information, which is information related to the voice uttered by the user.

[0016] Although not shown, the user monitoring device 30 includes, for example, an imaging device that captures an image of the user and an image recognition device that detects and recognizes the user's behavior (body movement, eye movement, etc.) from the captured image. The user monitoring device 30 also includes, for example, a microphone that detects sounds within the moving object MO and a voice recognition device that recognizes the voice uttered by the user and its content from the detected sounds.

[0017] The moving object MO also has a moving object monitoring device 40 that monitors the behavior of the moving object MO. The moving object monitoring device 40 acquires moving object behavior information, which is information related to the behavior of the moving object MO. The moving object monitoring device 40 has a detection device (not shown) that detects, for example, the position (current location), moving speed, moving direction, etc. of the moving object MO as the behavior of the moving object MO.

[0018] The moving object MO also has a navigation device 50 that provides travel guidance for the moving object MO. The navigation device 50 receives input of information about a destination from a user, and generates and outputs information that provides travel guidance for the moving object MO to the destination.

[0019] For example, the navigation device 50 acquires information about the current location of the moving object MO from the moving object monitoring device 40, and searches for at least one travel route from the current location to the destination. The navigation device 50 also generates and outputs information for providing travel guidance for the moving object MO along a route selected by the user from among the searched routes.

[0020] For example, the navigation device 50 generates and outputs map information highlighting the current location and travel route, audio information and image information informing the user of the approach of a point where the user should operate the mobile object MO (e.g., an intersection where the user should turn right or left), and audio information, image information, and audio information instructing the user to perform the operation (e.g., turn right or left).

[0021] The navigation device 50 also generates and outputs information to assist the user in operating the mobile object MO in order to control the movement of the mobile object MO. For example, the navigation device 50 generates audio and image information indicating that the user is approaching a point requiring attention, such as a merging point, and audio and image information prompting the user to change lanes. In this specification, information relating to guidance for and support for the movement of the mobile object MO, for example, is referred to as movement support information. In this embodiment, the movement support information is driving support information for a vehicle serving as the mobile object MO.

[0022] The mobile object MO also has a communication unit (not shown) for communicating with the outside world. In this embodiment, the mobile object MO communicates with the server 60 via wireless communication.

[0023] Next, the server 60 will be described. The server 60 has an advertisement information acquisition unit 61 that acquires various information (hereinafter referred to as advertisement information) related to advertisements provided by advertisers (not shown). For example, the advertisement information acquisition unit 61 acquires, from a plurality of advertisers, information related to the content of advertisements, information related to the location range on a map where the advertisement should be presented when a mobile object MO arrives (hereinafter referred to as advertisement presentation location), information related to the time range where the advertisement should be presented, and the like, and stores these.

[0024] The server 60 also includes a moving object information acquisition unit 62 that acquires information about the position and moving direction of the moving object MO from the moving object MO. For example, the moving object information acquisition unit 62 acquires moving object behavior information from the moving object monitoring device 40 of the moving object MO.

[0025] The server 60 also has an advertisement information selection unit 63 that selects advertisement information based on the position and moving direction of the mobile object MO from the advertisement information acquired (stored) by the advertisement information acquisition unit 61. The advertisement information selection unit 63 transmits the selected advertisement information to the mobile object MO. For example, the advertisement information selected by the advertisement information selection unit 63 and transmitted to the mobile object MO is information indicating advertisements for stores located around the mobile object MO in the moving direction of the mobile object MO.

[0026] Here, an overview of the recognition determination device 10 will be described. The recognition determination device 10 has a presentation information supply unit 11 that supplies information to be presented by the presentation unit 20 (hereinafter referred to as presentation information) to the presentation unit 20. In this embodiment, the presentation information supply unit 11 acquires, as the presentation information, advertising information corresponding to the current location of the mobile object MO from the server 60. Then, the presentation information supply unit 11 supplies the presentation information to the presentation unit 20.

[0027] Next, the recognition determination device 10 has a behavioral information acquisition unit 12 that acquires information related to the user's behavior (hereinafter referred to as behavioral information). The recognition determination device 10 also has a determination unit 13 that determines whether the presentation information has been recognized by the user based on the presentation information and the behavioral information. The determination unit 13 generates information related to the determination result of whether the presentation information has been recognized by the user (hereinafter referred to as recognition information), and transmits it to the server 60.

[0028] The server 60 has a recognition information acquisition unit 64 that acquires recognition information from the recognition determination device 10, and a billing information generation unit 65 that generates information on the amount of money to be charged to an advertiser for presenting an advertisement based on the recognition information. The billing information generation unit 65 sets different billing amounts depending on the recognition information.

[0029] For example, the billing information generation unit 65 generates billing information so that if it is determined that the advertising information has been recognized by the user, a normal advertising fee is charged, and if it is determined that the advertising information has not been recognized by the user, a lower advertising fee (for example, a fee discounted from the normal advertising fee) is charged. Then, every time the recognition information acquisition unit 64 acquires recognition information, the billing information generation unit 65 generates and stores billing information based on the recognition information. For example, the billing information generation unit 65 periodically outputs the billing information to the advertiser (for example, a terminal owned by the advertiser).

[0030] For example, the recognition determination device 10 and the server 60 constitute a performance-based advertising system that charges the advertiser a relatively high amount if it is determined that the advertisement has been recognized by the user, and charges the advertiser a relatively low amount if it is determined that the advertisement has not been recognized by the user.

[0031] In the above description, the navigation device 50 searches for a travel route for the moving object MO, generates map information, text information, and audio information related to route guidance, and generates travel assistance information. However, the route search, and generation of the guidance information and travel assistance information may be performed by the server 60. For example, the navigation device 50 may acquire conditions related to the route search, such as a destination and intermediate points, from the user and transmit them to the server 60. The server 60 may then search for a travel route to the destination and generate information related to the searched route and travel guidance for that route. In this case, for example, the navigation device 50 may acquire information related to the route guidance from the server 60 and present it to the presentation unit 20.

[0032] Fig. 2 is a block diagram showing an example configuration of the recognition determination device 10. A detailed configuration of the recognition determination device 10 will be described with reference to Fig. 2. First, the presentation information supply unit 11 of the recognition determination device 10 includes a presentation information acquisition unit 11A that acquires presentation information, and a data generation unit 11B that generates data for causing the presentation unit 20 to present the presentation information.

[0033] In this embodiment, the presentation information acquisition unit 11A acquires advertising information from the advertising information selection unit 63 of the server 60 as the presentation information.

[0034] The data generation unit 11B generates electronic data for outputting the presentation information acquired from the presentation information acquisition unit 11A from the presentation unit 20. In this embodiment, the data generation unit 11B includes an image data generation unit 11B1 that generates data (hereinafter referred to as image data) to be presented as an image on the display 21 of the presentation unit 20, and an audio data generation unit 11B2 that generates data (hereinafter referred to as audio data) to be presented as audio from the speaker of the presentation unit 20.

[0035] The data generation unit 11B generates the image data and audio data and supplies them to the presentation unit 20. The presentation unit 20 outputs the image data and audio data. The presentation unit 20 is provided in a position within the moving object MO where it can be presented to the user. For example, the display 21 is embedded in the dashboard of the vehicle. Furthermore, the speaker 22 is embedded in, for example, the door of the vehicle.

[0036] Next, the behavior information acquisition unit 12 of the recognition determination device 10 includes a voice information acquisition unit 12A that acquires voice information related to the user's voice within the moving object MO from the user monitoring device 30, a user behavior information acquisition unit 12B that acquires user behavior information related to the user's behavior from the user monitoring device 30, and a moving object behavior information acquisition unit 12C that acquires moving object behavior information related to the behavior of the moving object from the moving object monitoring device 40. The behavior information acquisition unit 12 acquires the voice information, user behavior information, and moving object behavior information as behavior information of the user.

[0037] Next, the determination unit 13 of the recognition determination device 10 determines the content (hereinafter referred to as the presentation content) of the presentation information (audio information and image information) supplied by the presentation information supply unit 11. The determination unit 13 also has an activity content determination unit 13B that determines the content (hereinafter referred to as the activity content) of the behavior information (audio information, user behavior information, and mobile object behavior information) acquired by the behavior information acquisition unit 12.

[0038] For example, the presentation content determination unit 13A and the behavior content determination unit 13B perform a matching process on the audio information and image information included in the presentation information and behavior information by dynamic programming (DP), and determine (recognize) the content.

[0039] The determination unit 13 also includes a relevance determination unit 13C that determines the relevance between the presented content and the activity content. The relevance determination unit 13C determines the relevance between the presented content and the activity content in three stages (e.g., low relevance, medium relevance, and high relevance), for example.

[0040] The determination unit 13 also includes a recognition determination unit 13D that determines whether the presented information has been recognized by the user based on the presented content and the activity content. For example, the recognition determination unit 13D determines that the presented information has been recognized by the user when there is a predetermined degree of association between the presented content and the activity content.

[0041] 3 is a diagram showing a flow of supplying presentation information by the presentation information supply unit 11. The flow of the presentation information supply process by the recognition determination device 10 will be described with reference to FIG. 3. First, the presentation information acquisition unit 11A acquires location information of the moving object MO from the moving object monitoring device 40 of the moving object MO (step S11). Then, the presentation information acquisition unit 11A transmits the location information of the moving object MO to the server 60 (step S12). These steps S11 and S12 are performed, for example, periodically.

[0042] This allows the mobile object information acquisition unit 62 of the server 60 to recognize the current location of the mobile object MO. Then, the advertising information selection unit 63 selects advertising information to be sent to the mobile object MO (advertising information to be presented to the user) based on the location information of the mobile object MO, and sends this to the mobile object MO.

[0043] In this embodiment, the presentation information acquisition unit 11A may transmit information regarding the moving direction in addition to the location information of the moving object MO to the server 60. In this case, the advertisement information selection unit 63 selects advertisement information based on the location and moving direction of the moving object MO and transmits it to the moving object MO.

[0044] Next, the presentation information acquiring unit 11A determines whether or not the presentation information has been acquired (step S13). First, in this embodiment, the presentation information acquiring unit 11A determines whether or not the advertisement information has been acquired from the server 60.

[0045] When it is determined that the presentation information has been acquired, the presentation information supply unit 11 supplies the presentation information to the presentation unit 20 (step S14). In this embodiment, in accordance with the presentation information, the image data generation unit 11B1 of the data generation unit 11B generates image data, and the audio data generation unit 11B2 generates audio data. The data generation unit 11B supplies the generated data to the presentation unit 20. As a result, the presentation unit 20 presents the acquired presentation information to the user.

[0046] If the presentation information acquisition unit 11A determines that the presentation information has not been acquired, the presentation information supply unit 11 terminates the presentation information supply flow with the determination result. The recognition determination device 10 performs the presentation information supply process by repeating, for example, such a flow.

[0047] 3 is merely an example. For example, the presentation information supply unit 11 may select presentation information to be supplied according to time from the advertising information acquired from the server 60, and supply the selected presentation information to the presentation unit 20.

[0048] Furthermore, the presentation information supply unit 11 may supply, as the presentation information, information displaying a list of all advertising information to be presented on the route when the navigation device 50 is providing guidance. In this case, the presentation information supply unit 11 transmits not only the location of the moving object MO but also information on the locations to which the moving object MO is scheduled to move, i.e., information on the moving route of the moving object MO, to the server 60, and obtains advertising information related to all of the location information from the server 60.

[0049] Next, Fig. 4 is a diagram showing a recognition determination flow by the recognition determination device 10. The flow of the recognition determination process by the determination unit 13 will be described with reference to Fig. 4. First, the presentation information supply unit 11 supplies the presentation information to the presentation unit 20 (step S21). Then, the behavioral information acquisition unit 12 acquires behavioral information when the presentation unit 20 presents the presentation information (step S22). The determination unit 13 starts a determination operation when the behavioral information acquisition unit 12 acquires the behavioral information.

[0050] The determination unit 13 acquires the presentation information and the behavioral information and determines the content of each of them. In this embodiment, the relevance determination unit 13C of the determination unit 13 first determines whether the content of the voice information (hereinafter referred to as voice content) among the behavioral content is related to the presentation content (step S23). In other words, it is determined whether the user's conversation is related to the advertisement.

[0051] Specifically, the relevance determination unit 13C determines the relevance between the presented content and the voice content, and supplies the determined relevance to the recognition determination unit 13D. If it is determined that the presented content and the voice content are related (if the relevance between them is equal to or greater than a predetermined relevance), the recognition determination unit 13D determines that the presented information has been recognized by the user (step S24).

[0052] Next, if it is determined that the presented content and the voice content are not related, the relevance determination unit 13C determines whether the content of the user behavior information (hereinafter referred to as user behavior content) among the action content is related to the presented content (step S25). If it is determined that the presented content and the user behavior content are related (if the relevance between them is equal to or greater than a predetermined relevance), the recognition determination unit 13D determines that the presented information has been recognized by the user (step S24).

[0053] Next, if it is determined that the presented content and the user behavior content are not related, the relevance determination unit 13C determines whether the content of the mobile object behavior information (hereinafter referred to as mobile object behavior content) among the behavior content is related to the presented content (step S26).If it is determined that the presented content and the mobile object behavior content are related (if the relevance between them is equal to or greater than a predetermined relevance), the recognition determination unit 13D determines that the presented information has been recognized by the user (step S24).

[0054] Then, when it is determined that the presentation content and the moving object behavior content are not related, the recognition determination unit 13D determines that the presentation information has not been recognized by the user (step S27). Note that, when the presentation information does not include content to be compared with the moving object behavior content, step S26 may be omitted. An example in which the presentation information includes content to be compared with the moving object behavior content will be described later. In this way, the determination unit 13 determines whether the presentation information has been recognized by the user.

[0055] After step S24 or step S27, the determination unit 13 (recognition determination unit 13D) transmits the determination result to the server 60 as recognition information (step S28), and ends the recognition determination flow. The recognition determination device 10 performs recognition determination of the presentation information by repeating, for example, such a flow.

[0056] 4 is merely an example. For example, any one of steps S23, S25, and S26 may be omitted depending on the behavioral information acquired by the behavioral information acquisition unit 12. Furthermore, the order of steps S23, S25, and S26 may be changed. For example, the relevance between the presented content and the user behavior content may be determined before determining the relevance between the presented content and the voice content (step S25 may be performed before step S23).

[0057] Alternatively, instead of this embodiment, when it is determined that the presented information has not been recognized by the user (after step S27), the presented information supply unit 11 may supply the presented information again to the presentation unit 20. In this case, the recognition determination device 10 may similarly determine the association between the presented content and the action content when the presented information is presented again, and transmit the determination result to the server 60 as recognition information.

[0058] The recognition information acquisition unit 64 of the server 60 acquires the recognition information from the recognition determination device 10. Furthermore, the billing information generation unit 65 determines the amount to be charged to the client (advertiser) of the presented information based on the recognition information, and generates billing information linked to this amount with the presented information.

[0059] For example, the billing information generation unit 65 generates billing information including, as an advertising fee for presenting the advertisement, a regular fee if it is determined that the advertisement has been recognized by the user, and a discounted fee that is a discount from the regular advertising fee if it is determined that the advertisement has not been recognized by the user. Furthermore, the billing information generation unit 65 supplies the billing information to the advertiser (charges the advertising fee) periodically or in response to a request from the advertiser.

[0060] FIG. 5 is a diagram showing an example of a table for recognition determination processing by the determination unit 13. A specific example of the criteria for recognition determination by the determination unit 13 will be described with reference to FIG. 5. Here, as an example, a case will be described in which it is determined that the presented information has been recognized by the user when the degree of association between the presented content and the activity content is medium or higher. First, consider a case in which the presented information supply unit 11 causes the presentation unit 20 to present, as audio information, advertising information that "dairy products are 30% off at store A" while the mobile object MO is moving around store A. In this case, the association degree determination unit 13C determines whether the audio information determined by the activity content determination unit 13B includes content that includes a keyword related to "dairy products."

[0061] For example, the advertisement information (presentation information) acquired from the server 60 may include metadata related to the content of the voice indicating the content of the advertisement. For example, in the case of an advertisement with the content "30% off dairy products at store A," the metadata may include metadata such as "store A," "dairy products," "milk," "milk," and "yogurt." The relevance determination unit 13C extracts keywords from the voice information (user conversation content) and compares them with the metadata. Then, if the keyword matches or is related to a part of the metadata, the relevance determination unit 13C may determine that there is a certain level of relevance between the presentation content and the behavioral content.

[0062] For example, if it is determined that the user has had a conversation containing the keyword "milk" (for example, a conversation such as "By the way, there's not much milk left, is there?"), the relevance determination unit 13C determines that there is a high relevance between the user's conversation and the content of the advertisement (presented content). In this case, the recognition determination unit 13D determines that the presented information has been recognized by the user.

[0063] The relevance determination unit 13C may determine whether the content of the voice information has changed between before and after the presentation information is presented, and whether the voice content after the presentation information is presented and the presentation content have a predetermined relevance or more. In this case, the behavior content determination unit 13B may acquire behavior information (e.g., voice information) from a predetermined time before the presentation information is presented.

[0064] In this case, for example, it is determined that the user was not engaged in a conversation about "dairy products" before the advertisement was presented, but started a conversation about "dairy products" after the advertisement was presented. Therefore, it is determined that the user engaged in an advertisement-related behavior both before and after the advertisement was presented by chance, regardless of the advertisement being presented, and the user engaged in an advertisement-related behavior due to the advertisement being recognized by the user. Furthermore, for example, if the content of the voice information does not change before and after the presentation of the presented information, the recognition determination unit 13D can determine that the presented information was not recognized by the user.

[0065] In other words, when the behavioral information includes the user's voice information, the judgment unit 13 judges the relevance between the presented content and the behavioral content based on the advertised product as the content of the presented information and the keywords contained in the voice as the content of the voice information, and determines whether the presented information has been recognized by the user based on the result of this relevance judgment.

[0066] Furthermore, the relevance determination unit 13C may determine that the advertising voice and the user's behavior are related when the timing of the start and stop of the presentation output of the voice included in the advertising information are linked to the timing of the change in the user's behavior. In this case, for example, the voice information acquisition unit 12A and the user behavior information acquisition unit 12B may continue to acquire the voice information and the user behavior information for a predetermined time (for example, 4 seconds) even after the presentation output of the advertising voice has ended. Furthermore, the behavior content determination unit 13B may determine the behavior content during the presentation of the advertising information and the behavior content for a predetermined time after the presentation.

[0067] Then, for example, the association degree determination unit 13C may determine whether or not the user's conversation stopped after the presentation of the presentation information started. Specifically, the association degree determination unit 13C may determine whether or not the content of the voice information determined by the action content determination unit 13B indicates that the user's conversation stopped for a predetermined time (for example, 5 seconds) after the presentation of the presentation information started and then resumed.

[0068] This corresponds to, for example, a case where the behavioral information acquisition unit 12 detects that the voice information before the presentation information is presented includes a voice uttered by the user, and the voice information is not acquired for a predetermined time after the presentation of the presentation information begins, and the user's voice is then detected again. In this case, the user may have stopped talking to listen to the advertising voice as advertising information. Therefore, for example, the association degree determination unit 13C may determine that there is a medium degree of association between the presentation content and the behavioral content. In this case, the recognition determination unit 13 also determines that the presentation content has been recognized by the user.

[0069] Also, consider a case where the presentation information supply unit 11 causes the presentation unit 20 to present, as image information, advertising information that states "dairy products are 30% off at store A." In this case, for example, the relevance determination unit 13C may determine whether or not the user behavior information determined by the behavior content determination unit 13B includes content indicating that the user viewed the display 21.

[0070] Then, for example, when it is determined that the user has been looking at the display 21 for a predetermined time or more, the relevance determination unit 13C may determine that there is a high relevance between the user's behavior (here, the movement of the user's eyes) and the content of the advertisement (presented content). In this case, the recognition determination unit 13D determines that the presented information has been recognized by the user.

[0071] Also, consider a case where the presentation information supply unit 11 causes the presentation unit 20 to present advertising information containing the content "30% off dairy products at store A" as image information, and simultaneously causes the presentation unit 20 to present music as audio information. In this case, for example, the relevance determination unit 13C may determine whether or not the user behavior information determined by the behavior content determination unit 13B includes content indicating that the user exhibited behavior in sync with the music.

[0072] For example, if it is determined that the user is swaying their body to the music, the relevance determination unit 13C may determine that there is a medium relevance between the user's behavior (here, the upper body movement) and the content of the advertisement (presented content). In this case, the recognition determination unit 13D determines that the presented information has been recognized by the user.

[0073] In other words, when the behavioral information includes user behavior information, the judgment unit 13 judges the degree of association between the presented content and the behavioral content based on the content of the presented information and the content of the user behavior information, and judges whether the presented information has been recognized by the user based on the judgment result of this degree of association.

[0074] Furthermore, when the presentation information includes music, the relevance determination unit 13C may determine the relevance between the content of the music and the content of the user's voice. For example, the relevance determination unit 13C may determine whether the voice information determined by the behavior content determination unit 13B includes content indicating that the user was making sounds in sync with the music. Then, for example, when it is determined that the user was humming or singing along with the music, the relevance determination unit 13C may determine that there is a medium relevance between the user's voice and the content of the advertisement (presentation content).

[0075] Next, consider a case where the presentation information supply unit 11 causes the presentation unit 20 to present advertising information including the content "30% off dairy products at store A" and travel support information for the mobile object MO (for example, information recommending lane changes at accident-prone areas) in succession as audio information.

[0076] In this case, the advertisement information including the advertisement content and the content related to the mobility support is generated in the server 60 and acquired by the recognition determination device 10, for example, as will be described below.

[0077] The advertisement information acquisition unit 61 acquires advertisement information including travel assistance information related to the advertisement presentation position from the advertiser. For example, if the advertiser's store is located near an intersection with poor visibility or a road junction, the advertiser may request that driving assistance information be presented simultaneously with the advertisement. In addition, the advertiser may intentionally request the presentation of an advertisement including driving assistance information.

[0078] For example, the advertisement information acquisition unit 61 provides the advertiser with map information highlighting points where driving assistance information should be presented, such as accident-prone areas and intersections where caution is required when driving (hereinafter referred to as "driving caution areas"). Based on the map information, the advertiser transmits advertisement information including information on the advertisement presentation position to the server 60.

[0079] Here, when the advertiser selects (requests) to present an advertisement at a driving caution point, the advertiser transmits advertisement information including the driving caution point as an advertisement presentation position. In this case, the advertisement information acquisition unit 61 combines the advertisement information acquired from the advertiser with the driving assistance information generated in the server 60 by linking them, and stores the combined advertisement information as advertisement information with driving assistance information.

[0080] It should be noted that the travel assistance information is not limited to being generated within the server 60. For example, the advertiser may create travel assistance information by itself to match the content of the advertisement. In this case, for example, the advertiser sets a driving caution point as the advertisement presentation position and transmits advertising information including driving assistance information related to the driving caution point. In this case, the advertising information acquisition unit 61 acquires the advertising information including the driving assistance information from the advertiser.

[0081] When the position of the moving object MO approaches a driving caution point (advertisement presentation position), the advertisement information selection unit 63 transmits advertisement information including travel assistance information associated with the driving caution point to the moving object MO. The presentation information supply unit 11 of the moving object MO causes the presentation unit 20 to present the advertisement information including the received travel assistance information. In other words, the server 60, together with the moving object MO, constitutes an advanced driver-assistance system (ADAS).

[0082] When advertising information including travel support information is presented to the presentation unit 20 in this manner for the moving body MO, the relevance determination unit 13C determines whether the moving body behavior information determined by the behavior content determination unit 13B includes content indicating that the moving body MO has exhibited the behavior recommended by the travel support information.

[0083] For example, if it is determined that the moving object MO has exhibited the behavior recommended by the travel assistance information (for example, has actually changed lanes), the relevance determination unit 13C may determine that there is a medium relevance between the behavior of the moving object MO as the user's action (i.e., the user's operation of the moving object MO) and the content of the advertisement (presented content). In this case, the recognition determination unit 13D may determine that the presented information has been recognized by the user.

[0084] It should be noted that the travel assistance information, i.e., in this embodiment, information contributing to safe driving at driving caution points, is information that the user of the moving object MO (in this embodiment, the vehicle occupants) needs to recognize with priority over other information. Therefore, when the moving object MO exhibits a behavior recommended by this travel assistance information, it is highly likely that the advertisement presented in association with this behavior has also been recognized by the user. Therefore, in this case, the recognition determination unit 13D may determine that the presented information has been recognized by the user.

[0085] For example, as described above, when the behavior information includes mobile object behavior information, the determination unit 13 may determine the degree of association between the presentation content and the behavior content based on the content of the presentation information and the content of the mobile object behavior information, and may determine whether the presentation information has been recognized by the user based on the determination result of this degree of association.

[0086] 5 is merely an example. For example, when the behavior information includes voice information and user behavior information, the recognition determination unit 13D may perform a recognition determination based on the content of each of the voice information and the user behavior information, or may perform a recognition determination using only one of the content. In other words, the recognition determination unit 13D may determine whether the user recognizes the presented information based on the content of the behavior information.

[0087] In this way, the recognition determination device 10 determines the degree of association between the content of information (e.g., advertising information) presented to a user and the content of the user's behavior after the presentation of the presented information, and determines whether the presented information has been recognized by the user based on this. Therefore, it is possible to accurately determine whether the information has been recognized by the user. Therefore, for example, a performance-based advertising system can be made to function accurately.

[0088] Furthermore, the behavioral information acquisition unit 12 acquires, for example, voice information related to a voice uttered by a user as behavioral information. In this case, the determination unit 13 determines whether the presented information has been recognized by the user based on the degree of association between the content of the presented information and the content of the voice information. Therefore, even if the presented information consists of only voice, for example, it is possible to accurately determine whether the presented information has been recognized by the user.

[0089] Furthermore, for example, if the content of the voice information changes between before and after the presentation information is presented and there is a predetermined degree of association between the content of the voice information after the presentation information is presented and the content of the presentation information, the determination unit 13 determines that the presentation information has been recognized by the user. Therefore, it is possible to accurately determine that the user's voice is due to the presentation information and accurately determine whether the presentation information has been recognized by the user.

[0090] Furthermore, for example, if the user's voice information is not acquired for a predetermined time after the presentation information is presented, the determination unit 13 determines that the presentation information has been recognized by the user, thereby making it possible to determine that the user was silent in order to recognize the presentation information, that is, that the presentation information may have been recognized by the user.

[0091] Furthermore, the behavior information acquisition unit 12 acquires, for example, user behavior information regarding the behavior of the user as the behavior information. In this case, the determination unit 13 determines whether or not the presentation information has been recognized by the user based on the degree of association between the content of the presentation information and the content of the user behavior information. Furthermore, the behavior information acquisition unit 12 acquires, for example, mobile object behavior information regarding the behavior of a mobile object MO as the behavior information. In this case, the determination unit 13 determines whether or not the presentation information has been recognized by the user based on the degree of association between the content of the presentation information and the content of the mobile object behavior information. This makes it possible to take into consideration a wide range of user behavior and accurately determine whether or not the presentation information has been recognized by the user.

[0092] Furthermore, when the presentation information includes travel assistance information, the determination unit 13 determines that the presentation information has been recognized by the user when the moving object MO exhibits behavior recommended by the travel assistance information. This makes it possible to determine that the behavior of the moving object MO as the user's action is due to the presentation information, i.e., that the presentation information may have been recognized by the user.

[0093] Furthermore, for example, when the determination unit 13 determines that the presentation information has not been recognized by the user, the presentation information supply unit 11 supplies the presentation information again. This increases the chances that the presentation information will be recognized by the user, and ensures that the presentation information will be recognized by the user.

[0094] In this embodiment, the recognition determination device 10 is mounted on a vehicle serving as a moving object MO, the presentation unit 20 is provided within the vehicle, and the user to whom the presentation information is to be sent is an occupant of the vehicle. However, the recognition determination device 10 does not need to be mounted within the moving object MO.

[0095] 6 is a block diagram showing a recognition determination device 10A according to a first modification of the first embodiment, a moving object MO, and a server 60A. In this modification, the recognition determination device 10A has the same configuration as the recognition determination device 10A, except that it is provided in the server 60A. In this modification, the moving object MO includes a presentation unit 20, a user monitoring device 30, a moving object monitoring device 40, and a navigation device 50.

[0096] In this modification, the recognition determination device 10A performs wireless communication with the moving object MO to acquire information about the moving object MO and the user of the moving object MO, and also performs recognition determination using various information in the server 60A.

[0097] This makes it possible to determine whether a user recognizes an advertisement when the advertisement is presented to a moving object MO that does not include the recognition determination device 10A. Therefore, for example, if the moving object MO has at least the presentation unit 20 and the user monitoring device 30, the advertisement presentation and user recognition determination can be performed by the recognition determination device 10A. Therefore, the recognition determination device 10A can be applied to a wide range of moving objects MO.

[0098] Furthermore, in this embodiment, the moving object MO is a vehicle, but the moving object MO may be a moving object other than a vehicle.

[0099] FIG. 7 is a block diagram showing a recognition determination device 10B according to a second modification of the first embodiment, a mobile object MO1, and a server 60. In this modification, the recognition determination device 10B is mounted on a mobile terminal serving as the mobile object MO1. For example, the mobile object MO1 is a mobile communication terminal such as a smartphone or a tablet. Furthermore, for example, the user of the mobile object MO1 is a person who uses the mobile terminal. For example, the user includes a person who operates the mobile terminal, a person who carries the mobile terminal, and a person who views the display of the mobile terminal.

[0100] In this modification, the recognition determination device 10B can be implemented as an application program installed in a mobile terminal, for example. The presented information is presented on the display 21 and speaker 22 of the mobile terminal.

[0101] Furthermore, for example, the user's voice information and behavior information as behavior information can be acquired from a user monitoring program 30A that monitors the user using a microphone and an imaging device mounted on the mobile terminal. Furthermore, the mobile object behavior information as behavior information can be acquired from a mobile object monitoring program 40A that monitors the behavior of the mobile object MO using a positioning device mounted on the mobile terminal. Note that the travel assistance information as presentation information can be generated in the server 60, for example, as in the recognition determination device 10 of the first embodiment, or can be acquired by the server 60 from an advertiser.

[0102] In this modification, when the moving object MO1 is a mobile terminal, the recognition determination device 10B may acquire, for example, information indicating the behavior of the mobile terminal as moving object behavior information. Then, the recognition determination device 10B may determine whether the presented information has been recognized by the user based on the degree of association between the content of the moving object behavior information and the content of the presented information. This allows the recognition determination device 10B to accurately determine whether the information has been recognized by the user.

[0103] In this embodiment, the recognition determination device 10 is configured to present the presentation information to the user of the moving object MO, and the presentation information is presented to the presentation unit 20 provided in the moving object MO according to the location of the moving object MO. However, the user to be determined by the recognition determination device 10 is not limited to the user of the moving object MO.

[0104] For example, the presentation unit 20 that presents the presentation information may be a display and a speaker fixed on the street or in a store. In this case, the presentation information supply unit 11 may periodically supply the presentation information to the display and the speaker. The user for which the recognition determination is performed is a pedestrian, a customer, or the like who is near the fixed display and the speaker. Even in this case, the recognition determination device 10 can accurately determine whether the information that it has instructed to present has been recognized by the pedestrian by, for example, acquiring information on the behavior and voice of the pedestrian from outside.

[0105] As described above, the recognition determination device 10 includes a presentation information supply unit 11 that supplies presentation information to be presented by the presentation unit 20, a behavioral information acquisition unit 12 that acquires behavioral information related to the behavior of the user to whom the presentation information is presented, and a determination unit 13 that determines whether the presentation information has been recognized by the user based on the content of the presentation information and the content of the behavioral information while the presentation information is being presented. Therefore, it is possible to provide a recognition determination device 10 that can accurately determine whether information has been recognized by the user.

[0106] Furthermore, the server 60, together with the recognition determination device 10, constitutes a contingency-based advertising system. In this case, the server 60 charges the advertiser using the determination result by the recognition determination device 10. Specifically, as described above, the server 60 charges the normal advertising fee if the advertisement is recognized by the user, and charges a discounted advertising fee if the advertisement is not recognized by the user. This makes it possible to provide a contingency-based advertising system that can accurately distinguish whether or not the user recognizes the advertisement and accurately charge the advertising fee based on this.

[0107] Furthermore, for example, by performing information processing according to the flow shown in Fig. 4, the present invention can be embodied as a recognition determination method performed using the recognition determination device 10. That is, for example, the present invention can be embodied as a recognition determination method including the steps of: supplying presentation information to be presented by the presentation unit 20 (step S21); acquiring behavioral information related to the behavior of a user to whom the presentation information is presented (step S22); and determining whether the presentation information has been recognized by the user based on the content of the presentation information and the content of the behavioral information while the presentation information is being presented (steps S23 to S27). This makes it possible to provide a recognition determination method that can accurately determine whether information has been recognized by the user.

[0108] Furthermore, for example, in-vehicle equipment such as a car navigation system, a communication information terminal such as a smartphone, or various other devices can be used as a computer (calculator), and this computer can be operated as the recognition determination device 10, or can be implemented as a program that performs the processing shown in Figure 4.

[0109] That is, for example, the present invention can be implemented as a program that causes a computer to function as a presentation information supply unit 11 that supplies presentation information to be presented by the presentation unit 20, a behavioral information acquisition unit 12 that acquires behavioral information related to the behavior of a user to whom the presentation information is presented, and a determination unit 13 that determines whether the presentation information has been recognized by the user based on the content of the presentation information and the content of the behavioral information while the presentation information is being presented. This makes it possible to provide a program that can accurately determine whether information has been recognized by the user. [Explanation of symbols]

[0110] 10, 10A, 10B recognition judgment device 11 Presentation information supply section 12 Behavioral information acquisition unit 13 Judgment section

Claims

[Claim 1] a presentation information supply unit that supplies presentation information to be presented by the presentation unit; a behavior information acquisition unit that acquires behavior information regarding the behavior of a user to whom the presentation information is to be presented; A recognition determination device characterized by having a determination unit that determines whether the presented information has been recognized by the user based on the content of the presented information and the content of the behavioral information while the presented information is being presented.

Images