Psychological state information estimation system, controlling method for psychological state information estimation system, and controlling program for psychological state information estimation system

The psychological state information estimation system addresses the challenge of assessing public reactions to displayed content by using imaging and analysis of head movements and facial expressions to determine the interest and reactions of individuals, providing accurate psychological state information for evaluation and analysis.

JP2024165924APending Publication Date: 2024-11-28TRANSACTION MEDIA NETWORKS INC
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2023082520
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-28

AI Technical Summary

Technical Problem

Existing systems struggle to accurately assess the psychological states of unspecified individuals who pass by information display units in public places, such as signage displaying advertisements, as they are unable to effectively evaluate their interest or reactions to the displayed content.

Method used

A psychological state information estimation system that includes an imaging unit to capture images of individuals, a management unit to store and manage subject information, a face information storage unit to capture and store facial information, and a psychological state information acquisition unit to analyze head movement and facial expressions to determine the psychological state of individuals based on their interest in the displayed content.

Benefits of technology

Enables the accurate and automatic acquisition of psychological state information from large numbers of unspecified individuals, allowing for effective evaluation and analysis of the displayed information based on their head movements and facial expressions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165924000001_ABST
    Figure 2024165924000001_ABST
Patent Text Reader

Abstract

To provide a psychological state information estimation system or the like that easily and accurately acquires information on the psychological state of a passerby or the like who visually recognizes information for an indefinite number of people displayed on an information display unit.SOLUTION: A psychological state information estimation system 1 has: an information display unit 201 that includes a display 200 displaying information for an indefinite number of people; an imaging unit 300 that picks up an image of a subject P passing through the front of the information display unit; and a management unit 10 that manages the information display unit and imaging unit. The psychological state information estimation system stores information on a subject who stops on the basis of imaging information of the subject image picked up by the imaging unit, stores information on the face of the subject who stops, stores the information on the subject who stops and faces the information display unit as information on an interest subject who is interested in the information for an indefinite number of people, and acquires information on the psychological state of the interest subject on the basis of information on the movement of the head of the interest subject.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a psychological state information estimation system that can acquire psychological state information of subjects, such as an unspecified number of people, who view information intended for an unspecified number of people displayed on an information display unit such as a display, a control method for the psychological state information estimation system, and a control program for the psychological state information estimation system. [Background technology]

[0002] There have been proposals for evaluating participants in a meeting or the like by acquiring information on human movements such as "nodding" of the participants (for example, Patent Document 1, etc.). In this way, it is easy to obtain motion information such as nodding from participants who are seated in the same place for a long period of time during a meeting, etc., and it is also easy to obtain motion information from the same person, etc. on a continuous basis and evaluate them based on that motion information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 2022-120787 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in public places, it has been difficult to grasp the psychological state of a person passing in front of an information display unit such as a signage that displays advertisements aimed at an unspecified number of people in relation to the content of the signage.

[0005] Therefore, the present invention aims to provide a psychological state information estimation system that can easily and accurately acquire psychological state information of an unspecified number of subjects, such as passersby who view information intended for an unspecified number of people displayed on an information display unit, a control method for the psychological state information estimation system, and a control program for the psychological state information estimation system.

[0006] According to the present invention, the above-mentioned object is achieved by a psychological state information estimation system having an information display unit arranged on a display unit that displays information for an unspecified number of people, which is information intended for an unspecified number of people, an imaging unit that images a subject passing in front of the information display unit, and a management unit that manages the information display unit and the imaging unit, characterized in that the psychological state information estimation system further comprises: a stopping information storage unit that stores information about a subject who has stopped based on imaging information of the subject imaged by the imaging unit; a face information storage unit that images and stores facial information of the subject who has stopped; an interested subject information storage unit that stores the information about the stopped subject having the facial information facing the information display unit as information about a subject who is interested in the information for an unspecified number of people displayed on the information display unit; and a psychological state information acquisition unit that acquires psychological state information of the subject of interest based on head movement information that is head movement information of the subject of interest.

[0007] According to the above configuration, information about a subject who has stopped in front of the information display unit is stored in the stopping information storage unit, facial information about the stopped subject is captured and stored, and when the face of the stopped subject is facing the information display unit, the subject is stored as information about a subject of interest. Then, the psychological state information acquisition unit acquires psychological state information of the person of interest based on the head movement information of the person of interest. In this way, it is possible to determine from facial information whether or not an unspecified number of people passing in front of the information display unit are "interested" in the content displayed on the information display unit, and based on the head movements of the person who is interested, it is possible to automatically and accurately obtain information on the psychological state of the person (for example, whether or not they affirmed the information aimed at an unspecified number of people displayed on the information display unit). Based on such information, the information for the general public displayed on the information display unit can be easily evaluated and analyzed by the general public.

[0008] Preferably, the head movement information of the psychological state information estimation system includes head shaking movement information, and the psychological state information of whether the person of interest affirms the information intended for an unspecified number of people is obtained based on the head shaking movement information.

[0009] According to the above configuration, the head movement information includes head shaking movement information, and psychological state information of the person of interest as to whether or not he / she affirms the information intended for an unspecified number of people is obtained based on this head shaking movement information. Therefore, psychological state information of the person of interest as to whether or not he / she affirms the information intended for an unspecified number of people can be automatically obtained based on the head shaking movement information.

[0010] Preferably, the head movement information of the psychological state information estimation system includes head tilting movement information, head rotating movement information, and head pulling movement information, and is configured to obtain the psychological state information other than whether or not the information intended for an unspecified number of people is affirmative based on this movement information.

[0011] According to the above configuration, the head movement information includes head tilting movement information, head turning movement information, and head pulling movement information, and based on this movement information, psychological state information other than whether the person of interest affirms or not about the information intended for an unspecified number of people is obtained. Therefore, psychological state information of the person of interest other than whether or not they affirm it can be automatically obtained based on the head tilting movement information, etc.

[0012] Preferably, the psychological state information estimation system according to claim 1 or 2, characterized in that the psychological state information acquisition unit of the psychological state information estimation system acquires the psychological state information of the person of interest based on facial expression information, which is facial expression information of the person of interest.

[0013] According to the above configuration, the psychological state information acquisition unit acquires psychological state information of the person of interest based on facial expression information, which is information on the facial expression of the person of interest, so that not only psychological state information based on head movement information but also psychological state information based on facial expression information can be acquired, thereby improving the accuracy of the acquired psychological state information.

[0014] Preferably, the content of the information intended for an unspecified number of people displayed on the information display unit of the psychological state information estimation system becomes time-unit change information that changes over time, image information of the subject captured by the imaging unit is stored in association with the time-unit change information, and information on the subject who has stopped, the face information of the subject who has stopped, and information on the subject of interest are generated for each piece of time-unit change information.

[0015] According to the above configuration, information on stopped subjects, facial information on stopped subjects, and information on subjects of interest are generated for each piece of time-unit change information, making it possible to grasp psychological state information of an unspecified number of subjects, etc. for each piece of time-unit change information.

[0016] According to the present invention, the above-mentioned object is achieved by a psychological state information estimation system comprising an information display unit arranged on a display unit that displays information for an unspecified number of people, which is information intended for an unspecified number of people, an imaging unit that images a subject passing in front of the information display unit, and a management unit that manages the information display unit and the imaging unit, wherein the information display unit has a partial information display unit that is each part divided into a plurality of parts, and is characterized in that the psychological state information estimation system further comprises a partial interest subject information storage unit that stores information of a stopped subject having face information facing the partial information display unit as partial interest subject information who is interested in the information for the unspecified number of people displayed on the partial information display unit, and a partial psychological state information acquisition unit that acquires psychological state information of the partial interest subject based on head movement information that is head movement information of the partial interest subject.

[0017] According to the above configuration, different information can be displayed simultaneously in each partial information display section, which is each part of the information display section, and when the face of a stopped subject is directed toward each partial information display section, the subject can be stored as partial interested subject information for the information displayed in that partial information display section. Based on the head movement information, which is head movement information of the partially interested subject, psychological state information of the partially interested subject is acquired. Therefore, for an unspecified number of people passing in front of the partial information display unit, it is possible to judge from their facial information whether they are "interested" in the content displayed on the partial information display unit, and to automatically obtain psychological state information of the partially interested subject who is interested in that part based on the head movements, etc. Based on such information, the information for the general public that is displayed in the partial information display section can be easily evaluated and analyzed by the general public.

[0018] Preferably, the psychological state information estimation system is configured such that the information display unit displays a single image information, the partial information display unit displays divided image information obtained by dividing the image information displayed on the information display unit, and the divided image information is combined to form the single image information.

[0019] According to the above configuration, an image of a portion of a single image displayed on the information display unit can be displayed on each partial information display unit, which is a part of the information display unit, and when the face of a stopped subject is directed toward each partial information display unit, the subject can be stored as partial interested subject information for that portion of the single image displayed on the partial information display unit (for example, the product part of the single image related to the advertisement, or the actress part appearing in the advertisement, etc.). Based on the head movement information, which is head movement information of the partially interested subject, psychological state information of the partially interested subject is acquired. In this way, for an unspecified number of people passing in front of a partial information display unit, if their behavior etc. indicates that they are "interested" in the content displayed in the partial information display unit out of a single overall image, their psychological state information can be automatically and accurately grasped. Based on such information, the information for the general public that is displayed in the partial information display section can be easily evaluated and analyzed by the general public.

[0020] According to the present invention, the above-mentioned object is achieved by a control method for a psychological state information estimation system having an information display unit arranged on a display unit that displays information for an unspecified number of people, which is information intended for an unspecified number of people, an imaging unit that images a subject passing in front of the information display unit, and a management unit that manages the information display unit and the imaging unit, characterized in that the control method for a psychological state information estimation system comprises the steps of: storing stopped subject information in a stopping information storage unit based on imaging information of the subject imaged by the imaging unit; imaging facial information of the stopped subject and storing it in a facial information storage unit; storing the stopped subject information having the facial information facing the information display unit in an interested subject information storage unit as information of a subject of interest who is interested in the information for an unspecified number of people displayed on the information display unit; and acquiring psychological state information of the subject of interest based on head movement information which is head movement information of the subject of interest.

[0021] According to the present invention, the above-mentioned object is achieved by a control program for a psychological state information estimation system having an information display unit arranged in a display unit that displays information for an unspecified number of people, which is information intended for an unspecified number of people, an imaging unit that images a subject passing in front of the information display unit, and a management unit that manages the information display unit and the imaging unit, which is characterized by realizing a function of storing information of a subject who has stopped in a stopping information storage unit based on imaging information of the subject imaged by the imaging unit, a function of imaging facial information of the subject who has stopped and storing it in a facial information storage unit, a function of storing information of the stopped subject having the facial information facing the information display unit in an interesting subject information storage unit as information of a subject who is interested in the information for an unspecified number of people displayed on the information display unit, and a function of acquiring psychological state information of the subject of interest based on head movement information which is head movement information of the subject of interest. Effect of the Invention

[0022] As described above, the present invention has the advantage of providing a psychological state information estimation system that can easily and accurately acquire psychological state information of an unspecified number of subjects, such as passersby who view information intended for an unspecified number of people displayed on an information display unit, a control method for the psychological state information estimation system, and a control program for the psychological state information estimation system. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 is a schematic diagram showing, for example, a "signage effect analysis system 1" which is a mental state information estimation system including, for example, a management server 10 which is a management unit according to a first embodiment and a second embodiment of the present invention. [Diagram 2] 2 is a schematic block diagram showing the main configuration of a management server 10 of FIG. 1. [Diagram 3] FIG. 3 is a schematic block diagram showing the contents of the "first server side various information storage unit 100" in FIG. [Figure 4] FIG. 3 is a schematic block diagram showing the contents of the "second server side various information storage unit 600" in FIG. [Diagram 5] FIG. 4 is a schematic diagram showing the contents of the “advertising information storage unit 111” in FIG. [Figure 6] 4 is a schematic flow chart illustrating a method for determining psychological state information of a passerby; [Figure 7] 11 is another schematic flow chart illustrating a method for determining psychological state information of passersby; [Figure 8] 11 is another schematic flow chart illustrating a method for determining psychological state information of passersby; [Figure 9] FIG. 11 is a schematic explanatory diagram showing a determination method. [Figure 10] FIG. 4 is a schematic explanatory diagram showing the contents of the "head movement category standard information" in the "head movement category standard information storage unit 123" in FIG. 3. [Figure 11] FIG. 4 is a schematic explanatory diagram showing the contents of the "head movement category determining criterion information" in the "head movement category determining criterion information storage unit 126" in FIG. 3. [Figure 12] FIG. 13 is a schematic explanatory diagram showing the contents of the “facial expression category standard information” in the “facial expression category standard information storage unit 613.” [Figure 13] 4 is a main flowchart showing a specific example of the operation of signage effectiveness analysis system 1 according to the present embodiment. [Figure 14] 10 is another main flowchart showing a specific example of the operation of signage effectiveness analysis system 1 in accordance with the present embodiment. [Figure 15] FIG. 1 is a schematic diagram showing a specific example. [Figure 16] FIG. 11 is a schematic explanatory diagram showing a method for determining whether or not a passerby is paying attention, etc., according to the second embodiment. [Figure 17] FIG. 3 is a schematic block diagram showing the contents of the third server-side various information storage unit 1010 in FIG. [Figure 18] FIG. 11 is a schematic block diagram showing the contents of a fourth server-side various information storage unit 1400 in FIG. [Figure 19] 1 is a schematic flow chart showing a method for determining a psychological state of a passerby. [Figure 20]11 is another schematic flow chart showing a method for determining a psychological state of a passerby; [Figure 21] 11 is another schematic flow chart showing a method for determining a psychological state of a passerby; [Figure 22] FIG. 18 is a schematic diagram showing the contents of the “partial advertisement information storage unit 1111” in FIG. [Figure 23] FIG. 11 is a schematic diagram showing the main configuration of a digital signage 500 of a signage effectiveness analysis system according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are preferred specific examples of the present invention, and therefore various technically preferable limitations are applied. However, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited to these embodiments.

[0025] (First embodiment) (Main components of the Signage Effect Analysis System 1) FIG. 1 is a schematic diagram showing a mental state information estimation system, for example, a "signage effect analysis system 1", including a management server 10, which is a management unit according to a first embodiment and a second embodiment of the present invention. As shown in FIG. 1, the system 1 includes an information display unit, for example, a display unit "digital signage 200" equipped with a "display 201" that displays information for an unspecified number of people, an imaging unit, for example, a camera 300, and a management server 10, which are connected to each other via an Internet network 2 so that they can communicate with each other.

[0026] Here, digital signage 200 is a general term for media that transmits information using electronic display devices such as displays 201 in various locations, such as outdoors, in storefronts, in public spaces, and on public transport. In this embodiment, the display 201 is configured to display advertisements for, for example, clothes, cakes, donuts, sushi, etc. (examples of "information aimed at an unspecified number of people"), which are intended for the general public who pass in front of the display 201.

[0027] Furthermore, the camera 300 is configured to capture an image of a subject passing in front of the display 201, and is configured to be automatically adjusted by an adjustment device (not shown) so that the display 201 and the camera 300 are coaxial.

[0028] The management server 10 is configured to not only manage advertisements displayed on the display 201, but also to determine psychological state information, such as positive or negative feelings of the subject about the advertisement displayed on the digital signage 200, based on image information, such as the subject's movements, acquired by the camera 300 (an example of ``psychological state information estimation'').

[0029] In addition, advertisements for clothes and the like that are displayed on the display 201 of the digital signage 200 described above and are aimed at an unspecified number of people can be displayed with different content by specifying a time period. In this embodiment, as described below, an "advertisement for clothes," a "advertisement for cakes," a "advertisement for donuts," and an "advertisement for sushi" are displayed at different times. Therefore, it is possible to judge the psychological state of a passerby regarding a specific content of the advertisement at a specific time period.

[0030] Furthermore, in this embodiment, as described later, it is possible to ascertain the age and the like of an unspecified number of passersby (subjects) from images of the subjects.

[0031] The digital signage 200, camera 300, and management server 10 in FIG. 1 each have a computer, including a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), a hard disk, etc., and are connected via a bus.

[0032] FIG. 2 is a schematic block diagram showing the main configuration of the management server 10 of FIG. As shown in FIG. 2, the management server 10 has a "server-side control unit 11," which controls a "server-side communication device 12" for communicating with the digital signage 200 and camera 300 in FIG. 1, a "server-side display 13" which is a display unit for displaying various information, a "server-side various information input device 14" for inputting various information, a "clock 15," and the "first server-side various information storage unit 100," "second server-side various information storage unit 600," "third server-side various information storage unit 1010," and "fourth server-side various information storage unit 2010."

[0033] FIG. 3 is a schematic block diagram showing the contents of the "first server side various information storage unit 100" in FIG. 2, and FIG. 4 is a schematic block diagram showing the contents of the "second server side various information storage unit 600" in FIG. 2. The contents of the "first server side various information storage unit 100" in FIG. 3 and the "second server side various information storage unit 600" in FIG. 4 will be described later.

[0034] (Example of main operation of Signage Effect Analysis System 1) (Preliminary process) First, before operating the system 1, the following preparations are performed. The management server 10 stores, in the "advertising information storage unit 111" in FIG. 3, the video of the advertisement to be displayed on the display 201 of the digital signage 200 and its display time zone information (an example of "time unit change information").

[0035] FIG. 5 is a schematic diagram showing the contents of the "advertisement information storage unit 111" in FIG. The "advertising information" stored in the memory unit 111 includes, for example, as shown in FIG. 5, a "clothing advertisement" from 11:50 to 12:00, a "cake advertisement" from 12:00 to 12:10, a "shoes advertisement" from 12:10 to 12:20, a "donut advertisement" from 12:20 to 12:25, and a "cake advertisement" from 12:25 to 12:30.

[0036] (Analysis and judgment method in signage effectiveness analysis system 1) Hereinafter, an overview of the analysis (estimation) of psychological state information of a passerby (subject) passing in front of the digital signage 200 in the present system 1 will be described. 6 to 8 are schematic flow charts showing a method for determining mental state information of a passerby, and FIG. 9 is a schematic explanatory diagram showing the determination method. The following description will be given with reference to the flowchart and FIG.

[0037] First, in step (hereinafter referred to as "ST") 1 of Figure 6, the "advertising information selection unit (program) 112" of the "management server 10" in Figure 3 operates, and the management server 10 refers to the time information of the clock 15 of the management server and the advertising information storage unit 111, and displays an advertisement aimed at an unspecified number of people on the display 201 of the digital signage 200.

[0038] Next, proceed to ST2. In ST2, the camera 300 captures an image of a person passing in front of the display 201, and stores the image together with time information in the "image capture information storage unit 113" in FIG. Next, proceed to ST3. In ST3, the "stopping information acquisition unit (program) 114" in Figure 3 of the management server 10 operates, refers to the "image capture information storage unit 113", and determines whether or not the person has stopped in front of the display 201. The person who has stopped (a passerby) is stored as a "stopping target" in the "stopping information storage unit 115" together with the time information. Specifically, the movement of the passerby is measured in time series from images of the passerby's movements captured by the camera 300, and it is determined whether the passerby has stopped and this is stored.

[0039] Next, proceed to ST4. In ST4, the “face information acquisition unit (program) 116” of the management server 10 in FIG. 3 operates to refer to the “stopping information storage unit 115” and store the face information of the face of the subject who stopped in the “face information storage unit 117” together with the time information.

[0040] Next, proceed to ST5. In ST5, the "face direction determination unit (program) 118" of the management server 10 operates to refer to the "face information storage unit 117" and determine whether the face of the subject is facing the "display 201" or not. Then, if it is determined in ST6 that the person is "facing the display 201", the process proceeds to ST7, where the person (passerby) is determined to be "interested" in the advertisement being displayed on the display 201, and this is stored in the "interested person information storage unit 119" in Figure 3 together with the time information. A passerby (subject) P1 in FIG. 9 is an example of a person who is judged to be "interested."

[0041] Specifically, from images of the movements of passersby captured by camera 300, changes in the vertical direction, horizontal direction, tilt, etc. of the passersby's neck are measured in time series and stored. Next, the frequency distribution and inflection points over a given time range are determined using Fourier transform and second-order differentiation of the data measured over time, and facial movements, such as the direction of the subject's face, are determined (when the left and right tilt of the face is "0°").

[0042] On the other hand, if it is determined in ST6 that the person is "not facing the display 201", the person is determined to be "not interested" and this is stored in the "uninterested person information storage unit 120" in FIG. 3 together with the time information. A passerby P2 in FIG. 9 is an example of a person who is judged to be "not interested."

[0043] Next, proceed to ST9. In ST9, the “head movement category candidate information acquisition unit (program) 122” in FIG. 3 (an example of a psychological state information acquisition unit) operates to acquire the subject's head movement information stored in the “interested subject information storage unit 119” and refer to the “head movement category standard information storage unit 123” to determine whether the head movement corresponds to each “head movement category.”

[0044] FIG. 10 is a schematic explanatory diagram showing the contents of the "head movement category standard information" in the "head movement category standard information storage unit 123" in FIG. As shown in Figure 10, the information for each "head movement category relevant information" stored in the "head movement category standard information storage unit 123" is as follows, and will be explained in the order of "type of head movement category," "relevant movement," and "corresponding psychological state."

[0045] 1) The relevant action of the "head shaking action category (an example of "head shaking action information")" is the "head shaking action", and the corresponding psychological state is the "positive psychological state", which includes, for example, "agreement: agree with the content of the advertisement", "approval: approve the content of the advertisement", "understanding: understand the content of the advertisement", etc.

[0046] 2) The relevant action in the "head shaking action category (an example of "head shaking action information")" is "head shaking action", and the corresponding psychological state is "negative psychological state", which includes, for example, "rejection: disagreement with the content of the advertisement, etc.," "disagreement: having a different opinion about the content of the advertisement," "disapproval: the content of the advertisement is inappropriate," "difficulty or uncertainty: there is no clear answer about the content of the advertisement, or it is not possible to understand," etc.

[0047] 3) A relevant action in the "tilting head forward and backward action category (an example of "tilting head action information") is "tilting head forward and backward action," and a corresponding psychological state is "psychological state of curiosity, interest, or doubt," which includes, for example, "curiosity: curious about the content of the advertisement," "interest: interested in the content of the advertisement," "doubt: unable to understand the content of the advertisement or seeking additional information," "evaluation: analyzing the content of the advertisement," etc.

[0048] 4) The relevant action in the "action category of tilting head left or right (an example of "head tilting action information")" is "action of tilting head left or right," and the corresponding psychological states include "doubt or confusion: a state in which the content of the advertisement cannot be understood or is uncertain," "interest or curiosity: a state in which the content of the advertisement is interesting and one wants to know more details," "friendly attitude: a state in which one tries to learn about the content of the advertisement in a relaxed and friendly state," and "making it easy to hear: a state in which one tries to understand the content of the advertisement," etc.

[0049] 5) The relevant action in the "head-spinning action category (an example of "head-spinning action information")" is "head-spinning action," and the corresponding psychological state is "psychological state of confusion, fatigue, stress, pressure, and information overload," which includes, for example, "a state in which one's concentration on the advertisement is distracted due to increased tension or stress."

[0050] 6) The relevant action in the "head pulling action category (an example of "head pulling action information") is the "head pulling action," and the corresponding psychological state is the "psychological state of hesitation, anxiety, fear, tension, and confusion," which includes, for example, "a state in which stress increases due to anxiety or fear caused by the content of the advertisement, and the self-defense instinct is activated."

[0051] 7) The relevant action in the "head-bowing action category" is "head-bowing action," and the corresponding psychological state is "psychological states of respect, humility, embarrassment, and agreement."

[0052] 8) The corresponding behavior in the "head tilting back behavior category" is "head tilting back behavior" and the corresponding psychological state is "psychological state of confidence, arrogance, and provocation."

[0053] 9) The corresponding behavior of the "head thrusting behavior category" is "head thrusting behavior," and the corresponding psychological state is "psychological states of interest, attention, challenge, and doubt."

[0054] Next, in ST10, if it is determined that the head movement falls into one of the head movement categories, the process proceeds to ST11. In ST11, the head movement data corresponding to each head movement category is stored as "head movement category candidate information" in the "head movement category candidate information storage unit 124" in FIG. 3 in association with each head movement category. On the other hand, if there is no corresponding head movement category in ST10, the process ends.

[0055] Next, the process proceeds to ST12. In ST12, the "head movement category determination unit (program) 125" in Fig. 3 operates to refer to the "head movement category determination criteria information storage unit 126" and determine whether or not the "head movement category candidate information" in the "head movement category candidate information storage unit 124" satisfies all of the criteria A to D of the "head movement category determination criteria information."

[0056] In this embodiment, an example has been described in which all of the criteria A to D in the "Head Movement Category Determination Criteria Information" are satisfied; however, the present invention is not limited to this, and may also include cases in which any of the criteria is satisfied, or cases in which the criteria are prioritized and judged.

[0057] FIG. 11 is a schematic explanatory diagram showing the contents of the "head movement category determining criterion information" in the "head movement category determining criterion information storage unit 126" in FIG. As shown in FIG. 11, the head movement category determining criterion information specifically has four types, A to D. Below, the criteria will be described for each type of head movement category determining criterion information.

[0058] A) The standard for "head shaking count" is the number of times per second, specifically, the standards vary depending on age. B) The standard for the "range of head movement" is the angle, specifically, the threshold is dynamically set using the "moving average + 3σ method." For example, for subjects whose heads hardly move, even a slight movement of about 5° is detected. C) The "head movement duration" criterion is the time between when the movement begins and ends. D) The standard for "number of times head movement is repeated (number of times the movement is repeated)" is, for example, 2 to 4 times. It is to be noted that the above-mentioned criteria are merely examples, and the present invention is not limited to these.

[0059] Next, if it is determined in ST13 that the requirements are met, the process proceeds to ST 14. In ST14, the information is stored as "head movement category determination information" in the "head movement category determination information storage unit 611" in FIG. 4 in association with each head movement category. On the other hand, if it is determined in ST13 that the requirement is not met, the process ends.

[0060] Next, the process proceeds to ST15. In ST15, the "facial expression category information acquisition unit (program) 612" in Fig. 4 (an example of a "psychological state information acquisition unit") operates to acquire information on the facial expression of the subject's head stored in the "interested subject information storage unit 119" in Fig. 3, and refers to the "facial expression category standard information storage unit 613" in Fig. 4 to determine whether or not the facial expression corresponds to each "facial expression category." Specifically, the determination is made by measuring and comparing changes in the eyes, mouth, etc. from a neutral expression.

[0061] FIG. 12 is a schematic explanatory diagram showing the contents of the "facial expression category standard information" in the "facial expression category standard information storage unit 613." As shown in Figure 12, the "Facial Expression Category" is divided into 10 categories. Therefore, below, the facial expressions corresponding to each type of facial expression category will be described.

[0062] a) Facial expressions that fall into the "happy facial expression category" are those that fall into all of the following categories: "sparkling eyes (wide open eyes)," "turned corners of the mouth," and "puffed cheeks." b) Facial expressions that fall into the "sadness category" are those that fall into all of the following categories: "lowered eyebrows," "lowered corners of the mouth," and "moist eyes." c) Facial expressions that fall into the "anger expression category" are those that fall into all of the following categories: "frowning brow," "narrowing eyes," and "tightly closed mouth."

[0063] d) Facial expressions that fall into the "surprised expression category" are all of the following: "raised eyebrows," "wide open eyes," and "open mouth." e) Facial expressions that fall into the "contemptuous facial expression category" are those that fall into both the "one corner of the mouth turning up" and "narrowing of the eyes (one-eyed looking expression)." f) Facial expressions that fall into the "fearful facial expression category" are those that fall into all of the following categories: "raised eyebrows," "wide open eyes," and "open or narrowed mouth."

[0064] g) Facial expressions that fall into the "questioning facial expression category" are those that fall into all of the following categories: "eyebrows raised," "slight tilt of face," and "eyes directed toward the advertisement." h) Facial expressions that fall into the "interested facial expression category" are those that meet all of the following criteria: "turning the face toward the advertisement" and "opening the eyes wide and staring intently at the advertisement." i) Facial expressions that fall into the "sympathetic facial expression category" are those that fall into all of the following categories: "tilting the head," "lowering eyebrows," and "softening facial expression." j) Facial expressions that fall into the "arrogant facial expression category" are those that fall into all of the following categories: "tilting the head upward," "lowering the gaze," and "raising the nose high."

[0065] If it is determined to be applicable in ST16, proceed to ST17. In ST17, facial expression data corresponding to each facial expression category (such as the joyful expression category) is associated with each facial expression category as "facial expression category corresponding information" and stored in the "facial expression category information storage unit 614" in FIG.

[0066] Next, the process proceeds to ST18. In ST18, the "digital signage analysis unit (program) 615" in Fig. 4 operates to refer to the "head movement category determination information storage unit 611" and the "head movement category reference information storage unit 123" to acquire psychological state information for the corresponding movement category, and also acquires "facial expression detailed category information" from the "facial expression category information storage unit 614", and performs an analysis of the displayed advertisement based on the "psychological state information" and "facial information category information", and stores the analysis in the "analysis result information storage unit 616" in Fig. 4.

[0067] From the above, according to the "analysis and judgment method in signage effect analysis system 1" of this embodiment, psychological state information for each passerby (subject) passing in front of the digital signage 200 in Figure 1 can be obtained not only from head movements but also from facial expressions, and based on this psychological state information, appropriate analysis can be made of advertisements, etc. displayed on signage, etc.

[0068] (Specific operation example of the signage effectiveness analysis system 1) 13 and 14 are main flowcharts showing a specific example of the operation of signage effectiveness analysis system 1 according to the present embodiment. FIG. 15 is a schematic diagram showing a specific example. That is, the digital signage 200 in FIG. 1 changes its advertisements depending on the time of day, and at the same time automatically determines whether passersby passing in front of the digital signage 200 are disinterested, interested, paying attention, convinced, etc., and analyzes the advertisements based on the results. This is an example that will be explained in detail below using the flowcharts in FIG. 13 and FIG. 14 and the schematic diagram in FIG. 15.

[0069] First, in ST21, the “advertising information selection unit 112” of the management server 10 in Figure 3 operates to obtain time information from the “advertising information storage unit 111” and selects the “advertisement” to be displayed on the display 201 of the digital signage 200 at that time; specifically, in the case of 11:50, a “clothing advertisement” is selected and displayed on the display 201.

[0070] Next, the process proceeds to ST22. In ST2, the camera 300 in Fig. 1 captures an image of a person passing in front of the display 201, and stores the image together with time information in the "image capture information storage unit 113" in Fig. 3.

[0071] Next, the process proceeds to ST23. In ST23, the "stopping information acquisition unit (program) 114" in Fig. 3 of the management server 10 operates to refer to the "imaging information storage unit 113" to determine whether or not a person has stopped in front of the display 201, and stores the person who has stopped as a "stopping target person" together with time information in the "stopping information storage unit 115" in Fig. 3.

[0072] Next, the process proceeds to ST24. In ST24, the "face information acquisition unit (program) 116" in FIG. 3 of the management server 10 operates to refer to the "stopping information storage unit 115" and store the face information of the face of the subject who stopped in the "face information storage unit 117" together with the time information.

[0073] Next, the process proceeds to ST25. In ST25, the "face direction determination unit (program) 118" of the management server 10 in FIG. 3 operates to refer to the "face information storage unit 117" and determine whether the face of the target person is facing the "display 201".

[0074] In this embodiment, as shown in FIG. 15(1), the face of the target person (passerby) P2 is not facing the display 201, so it is determined that the target person is "not interested." In addition, the gender, age, etc. of the subject P2 are estimated from the movements, face, etc. Then, the "disinterested information" and "gender and age information" of the subject P2 are associated with the "time information" and stored in the "uninterested subject information storage unit 120" of FIG. This completes the determination (estimation) of the appeal level information (no interest) regarding the "clothing advertisement" for passerby P2.

[0075] Next, the process proceeds to ST26. In ST26, the "advertising information selection unit 112" operates again to refer to the "advertising information storage unit 111" and obtain time information from the clock 15, and select an advertisement to be displayed on the display 201 of the digital signage 200 at that time. Specifically, in the case of 12:00, an "advertisement for cake" is selected and displayed on the display 201.

[0076] Next, the process proceeds to ST27. ST27 executes the same steps as ST22 to ST24 described above, and stores the face information of the face of the subject who has stopped in front of the display 201 in the "face information storage unit 117" together with the time information.

[0077] Next, the process proceeds to ST28. In ST28, the "face direction determination unit (program) 118" in Fig. 3 operates and refers to the "face information storage unit 117". In the case of the "cake advertisement," as shown in FIG. 15(2), the face of the target person (passerby) P1 is directed toward the "display 201," so it is determined that the target person is "interested."

[0078] Next, the process proceeds to ST29. In ST29, the gender, age, etc. of the subject P2 are estimated based on the movements, face, etc. of the subject P2. Then, the "interested person information" and "gender and age information" of the target person P1 are associated with the "time information" and stored in the "interested person information storage unit 119" of FIG.

[0079] This completes the determination (estimation) of the appeal level information (interested) regarding the "cake advertisement" for the passerby (target person) P1.

[0080] Next, the process proceeds to ST30. In ST30, the head movement information of the subject stored in the "interested subject information storage unit 119" in Fig. 3 is obtained, and the "head movement category standard information storage unit 123" is referenced. It is then determined that the head movements of subject P1 fall into the "head shaking up and down movement category," "head tilting back and forth movement category," and "head rotating movement category," and the corresponding head movement information is associated with each category and stored as "head movement detailed category candidate information" in the "head movement category candidate information storage unit 124" in Figure 3.

[0081] Next, the process proceeds to ST31. In ST31, the "head movement category determination criteria information storage unit 126" in Fig. 3 is referenced, and the "head movement category candidate information" in the "head movement category candidate information storage unit 124" satisfies all of the criteria A to D in the "head movement category determination criteria information" in Fig. 11 because, in this embodiment, it is the "movement category of shaking head up and down", and therefore information on this movement category is stored in the "head movement category determination information storage unit 611" in Fig. 4 together with information on subject P1.

[0082] Next, the process proceeds to ST32. In ST32, the head expression information of the subject P1 stored in the "interested subject information storage unit 119" in Fig. 3 is obtained, and the "facial expression category standard information storage unit 613" in Fig. 4 is referenced. Then, since the facial expression of subject P1 corresponds to a "surprised expression," the corresponding facial expression information is associated with that category and stored as "facial expression category information" in the "facial expression category information storage unit 614" in Figure 4.

[0083] Next, the process proceeds to ST33. In ST33, since the head movement of the subject P1 indicates a "positive" psychological state and the facial expression indicates a "surprised" psychological state, an analysis result can be generated that the subject P1 is in a "positive" yet "surprised" psychological state with respect to the cake advertisement, and the result is stored in the "analysis result information storage unit 616".

[0084] This completes the judgment (estimation) of the psychological state information of the passerby (subject) P1 regarding the "cake advertisement." Therefore, the advertiser of the advertisement displayed on the digital signage 200 can accurately determine the effectiveness of the advertisement. In other words, psychological state information of an unspecified number of passersby who have viewed information such as an advertisement displayed on digital signage 200 can be provided to advertisers with high accuracy.

[0085] (Second embodiment) The main configuration of signage effectiveness analysis system 1000 according to this embodiment shown in FIG. 1 is common to the first embodiment described above, so the following description will focus mainly on the differences. FIG. 16 is a schematic explanatory diagram showing a method for determining whether or not a passerby is paying attention, etc., according to the embodiment. As shown in FIG. 16, in this embodiment, the screen of the information display unit, for example, display 410, is divided into a plurality of, for example, five, partial information display units, specifically, "split displays 411 to 415," are arranged. These split displays 411 etc. are configured to be capable of simultaneously displaying different pieces of information.

[0086] In this embodiment, when information is displayed on the split displays 411 to 415, the psychological state information of passersby can be analyzed for each split display 411, etc., which is different from the above-mentioned first embodiment. Therefore, the following description will be given taking as an example a case where black, white, red, green and yellow sneakers are displayed, for example.

[0087] (Preliminary process) First, before operating the system 1000 shown in FIG. 1, the following preparations are performed. 17 and 18 are schematic block diagrams showing the contents of the third server-side various information storage unit 1010 and the fourth server-side various information storage unit 1400 in FIG. Management server 10 in FIG. 1 stores, in "partial advertisement information storage unit 1111" in FIG. 17, advertisement images and the like to be displayed on divided displays 411 to 415 of display 410 of digital signage 400 in FIG.

[0088] FIG. 22 is a schematic diagram showing the contents of the "partial advertisement information storage unit 1111" in FIG. The "partial advertising information" stored in memory unit 1111 stores information indicating that, for example, as shown in FIG. 22, first split display 411 will display "black sneakers", second split display 412 will display "white sneakers", third split display 413 will display "red sneakers", fourth split display 414 will display "green sneakers", and fifth split display 415 will display "yellow sneakers" simultaneously.

[0089] (Method of Analysis and Judgment in Signage Effectiveness Analysis System 1000) An overview of the analysis (estimation) of psychological state information of a passerby (subject) passing in front of digital signage 400 in this system 1000 will be described below. 19 to 21 are schematic flow charts showing a method for determining the psychological state of a passerby. The following description will be given based on the flowchart.

[0090] First, in ST101 of FIG. 19, the "partial advertising information selection unit (program) 1112" of FIG. 17 of the "management server 10" operates, and the management server 10 refers to the partial advertising information storage unit 1111, and simultaneously displays black sneakers, etc. of FIG. 22, which are advertisements aimed at an unspecified number of people, on each of the five split displays (first split display 411 to fifth split display 415) of the display 410 of the digital signage 400 shown in FIG. 16.

[0091] Next, proceed to ST102. In ST102, camera 300 captures an image of a person passing in front of display 410, and stores the image together with time information in "image capture information storage unit 113" in FIG.

[0092] Next, proceed to ST103. In ST103, the "management server stopping information acquisition unit (program) 114" in Figure 17 of the management server 10 operates and refers to the "imaging information storage unit 113" to determine whether or not the person has stopped in front of the display 410, and stores the person who has stopped (a passerby) as a "stopped target" in the "stopping information storage unit 115" in Figure 17 together with the time information. Specifically, the movement of the passerby is measured in time series from images of the passerby's movements captured by the camera 300, and it is determined whether the passerby has stopped and this is stored.

[0093] Next, proceed to ST104. In ST104, the “face information acquisition unit (program) 116” of the management server 10 in FIG. 17 operates to refer to the “stopping information storage unit 115” and store the face information of the face of the subject who stopped in the “face information storage unit 117” together with the time information.

[0094] Next, proceed to ST105. In ST105, the “face direction part determination unit (program) 1118” in FIG. 17 of the management server 10 operates to refer to the “face information memory unit 117” and determine which “split display 411, etc.” of the “display 410” the subject's face is facing. Then, if it is determined in ST106 that the target person (passerby) is "looking toward one of the split displays 411 etc.", the process proceeds to ST107, where the target person (passerby) is determined to be "interested" in the advertisement displayed on the split display 411 etc. that the target person is looking toward, and is associated with the advertisement or split display 411 etc. and stored together with the time information in the "partially interested target information storage unit 1119" in Figure 17. FIG. 16 shows an example in which a passerby (subject) P1 is determined to be "interested" in "fourth split display 414 (green sneakers)".

[0095] Specifically, from images of the movements of passersby captured by camera 300, changes in the vertical direction, horizontal direction, tilt, etc. of the passersby's neck are measured in time series and stored. Next, the frequency distribution and inflection points over a given time range are determined using Fourier transform and second-order differentiation of the data measured over time, and facial movements, such as the direction of the subject's face, are determined (when the left and right tilt of the face is "0°").

[0096] On the other hand, if it is determined in ST106 that the person is "not facing the display 410", it is determined that the person is "not interested" and this is stored in the "partially uninterested person information storage unit 1120" in FIG. 17 together with the time information. A passerby P2 in FIG. 16 is an example of a person who is judged to be "not interested."

[0097] Next, proceed to ST109. In ST109, the "partial head movement category candidate information acquisition unit (program) 1122" in Figure 17 (an example of a partial psychological state information acquisition unit) operates to acquire the subject's head movement information stored in the "partial interesting subject information storage unit 1119" and refer to the "head movement category standard information storage unit 123" (see Figure 10 for details) to determine whether the head movement corresponds to each "head movement category."

[0098] Next, in ST110, if it is determined that the head movement falls into one of the head movement categories, the process proceeds to ST111. In ST111, the head movement data corresponding to each head movement category is stored as "partial head movement category candidate information" in the "partial head movement category candidate information storage section 1124" of FIG. 17 in association with each head movement category. On the other hand, if there is no corresponding head movement category in ST110, the process ends.

[0099] Next, the process proceeds to ST112. In ST112, the "partial head movement category determination unit (program) 1125" in Fig. 17 operates to refer to the "head movement category determination criteria information storage unit 126" (details in Fig. 11) and determine whether the "head movement category candidate information" in the "partial head movement category candidate information storage unit 1124" satisfies all of the criteria A to D of the "head movement category determination criteria information".

[0100] In this embodiment, an example has been described in which all of the criteria A to D in the "Head Movement Category Determination Criteria Information" are satisfied; however, the present invention is not limited to this, and may also include cases in which any of the criteria is satisfied, or cases in which the criteria are prioritized and judged.

[0101] Next, if it is determined in ST113 that the requirements are met, the process proceeds to ST114. In ST114, the information is stored as "partial head movement category determination information" in the "partial head movement category determination information storage unit 1411" in FIG. On the other hand, if it is determined in ST113 that the requirement is not met, the process ends.

[0102] Next, the process proceeds to ST115. In ST115, the "facial expression category information acquisition unit (program) 612" in Fig. 18 (an example of a "partial psychological state information acquisition unit") operates to acquire information on the facial expression of the subject's head stored in the "partially interested subject information storage unit 1119" in Fig. 17, and refers to the "facial expression category standard information storage unit 613" in Fig. 18 (details in Fig. 12) to determine whether or not the facial expression corresponds to each "facial expression category." Specifically, the determination is made by measuring and comparing changes in the eyes, mouth, etc. from a neutral expression.

[0103] If it is determined to be applicable in ST116, proceed to ST17. In ST117, facial expression data corresponding to each facial expression category (such as a joyful expression category) is associated with each facial expression category as "partial facial expression category information" and stored in "partial facial expression category information storage unit 1414" in FIG.

[0104] Next, the process proceeds to ST118. In ST118, the "digital signage analysis unit (program) 615" in Fig. 18 operates to refer to the "partial head movement category determination information storage unit 1411" and the "head movement category reference information storage unit 123" to acquire psychological state information for the corresponding movement category, and also acquires "facial expression detailed category information" from the "facial expression category information storage unit 614", and performs an analysis of the displayed advertisement based on the "psychological state information" and "facial information category information", and stores the analysis in the "analysis result information storage unit 616" in Fig. 18.

[0105] From the above, according to the "analysis and judgment method in signage effect analysis system 1" of this embodiment, psychological state information for each passerby (subject) passing in front of the digital signage 200 in Figure 1 can be obtained not only from head movements but also from facial expressions, and based on this psychological state information, appropriate analysis can be made of advertisements, etc. displayed on signage, etc.

[0106] Furthermore, according to the “analysis and judgment method in the signage effect analysis system 1000” of this embodiment, each passerby (subject) passing in front of the digital signage 400 in FIG. 1 can automatically and appropriately classify and acquire psychological state information about each advertisement, etc. displayed on each divided display 411 to 415 of the display 410. Therefore, even if advertisements are displayed divided on the divided displays 411 of the display 410 of the same digital signage 400, if a passerby evaluates each advertisement as "interested," the psychological state information can be analyzed individually and accurately and easily.

[0107] (Third embodiment) FIG. 23 is a schematic diagram showing the main configuration of digital signage 500 of a signage effectiveness analysis system according to the third embodiment of the present invention. As shown in FIG. 23, the digital signage 500 has a display 510, and the display 510 displays a single piece of image information, for example, "advertising image of a cosmetic product (CT) by an actress (FM)."

[0108] As shown in FIG. 23, the display 510 is divided into, for example, nine divided displays 511 to 519 which are partial information display sections. Unlike the second embodiment described above, each of the split displays 511 to 519 in this embodiment does not display a separate image, but rather displays an image of a portion of the "promotional image of cosmetics (CT) by actress (FM)" displayed on display 510.

[0109] In this way, the "promotional image of the cosmetics (CT) by the actress (FM)" displayed on the display 510 will be visually recognized by passersby (target persons) P11 and P12 in FIG. Then, the management server judges to which of the split displays 511 in FIG. 18 the faces of the passersby P11 and the like are directed, in the same manner as in the above-mentioned second embodiment.

[0110] Furthermore, similarly to the second embodiment described above, for each divided display 511 etc., if passersby P11 and P12 are "interested", their psychological state information is also acquired.

[0111] However, in this embodiment, unlike the second embodiment described above, each divided display 511 etc. does not display a different image, but displays a portion of a single image information displayed on display 510, namely, "promotional image of cosmetics (CT) by actress (FM)", for example, the portion displaying the actress (FM) and the portion displaying the cosmetics (CT). Therefore, by determining whether or not a passerby P11, etc. is "interested" for each divided display 511, etc., it is possible to automatically and accurately determine which part of the overall image displayed on the display 510 the passerby P11, etc. is "interested" in, and to determine their psychological state information.

[0112] Based on such information, it is possible to easily perform evaluation and analysis by passersby P11 and the like of the portions displayed on the split display 511 and the like (the portion displaying the actress (FM) and the portion displaying the cosmetics (CT)).

[0113] In the above-described embodiment, an example has been given in which the invention is realized as an apparatus, but the invention is not limited to this, and the invention may be distributed as a program that can be executed by a computer, stored on a storage medium such as a magnetic disk (e.g., a floppy (registered trademark) disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD), a magneto-optical disk (MO), or a semiconductor memory.

[0114] The storage medium may be any medium that can store a program and is computer readable. The storage format of the storage medium is not particularly limited.

[0115] In addition, an OS (operating system), database management software, network software, or other MW (middleware) running on a computer may execute some of the processes for realizing this embodiment based on instructions from a program installed on the computer from a storage medium.

[0116] Furthermore, the storage medium in the present invention is not limited to a medium independent of a computer, but also includes a storage medium that stores or temporarily stores a program downloaded via a LAN, the Internet, or the like.

[0117] Furthermore, the computer in the present invention only needs to execute each process in this embodiment based on a program stored in a storage medium, and may be a device consisting of a single personal computer, etc., or a system in which multiple devices are connected to a network.

[0118] In addition, the computer in this invention is not limited to a personal computer, but also includes an arithmetic processing unit, a microcomputer, etc. included in information processing equipment, and is a general term for equipment or devices that can realize the functions of the present invention by a program.

[0119] The above describes the embodiments of the present invention. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the claims. The configurations of the above embodiments can be partially omitted or arbitrarily combined in a manner different from the above. In this embodiment, camera 300 captures an image of a passerby in front of display 201 and stores the image in "image information storage unit 113" of management server 10 together with time information, and based on this stored information, if the passerby is "interested," information on the passerby's psychological state is obtained and advertisements, etc. displayed on display 411 are analyzed. However, the present invention is not limited to this, and the time of display 411, camera 300, and management server 10 may be synchronized, and at the same time that camera 300 captures an image of a passerby, if the passerby is "interested," information on the passerby's psychological state is obtained and advertisements, etc. displayed on display 411 are analyzed. [Explanation of symbols]

[0120] 1, 1000.... Signage effect analysis system, 2.... Internet network, 10.... Management server, 11.... Server side control unit, 12.... Server side communication device, 13.... Server side display, 14.... Server side various information input device, 15.... Clock, 100.... First server side various information storage unit, 111.... Advertising information storage unit, 112.... Advertising information selection unit, 113.... Imaging information storage unit, 114.... Standing information acquisition unit, 115.... Standing information storage unit, 116.... Face information acquisition unit, 1 17...Facial information storage unit, 118...Facial direction determination unit, 119...Interested person information storage unit, 120...Non-interested person information storage unit, 122...Head movement category candidate information acquisition unit, 123...Head movement category standard information storage unit, 124...Head movement category candidate information storage unit, 125...Head movement category determination unit, 126...Head movement category determination standard information storage unit, 125...Digital signage analysis unit, 126...Analysis result information storage unit, 200, 400, 500...Digital signage, 201, 510...display, 300...camera, 411, 511, etc....split display, 600...second server-side various information storage unit, 611...head movement category determination information storage unit, 612...facial expression category information storage unit, 613...facial expression category standard information storage unit, 614...facial expression category information storage unit, 615...digital signage analysis unit, 616...analysis result information storage unit, 1010...third server-side various information storage unit, 1111...partial advertisement information storage unit, 11 12... partial advertisement information selection unit, 1118... partial face direction determination unit, 1119... partial interested person information storage unit, 1120... partial non-interested person information storage unit, 1122... partial head action category candidate information acquisition unit, 1124... partial head action category candidate information storage unit, 1125... partial head action category determination unit, 1400... fourth server-side various information storage unit, 1411... partial head action category determination information storage unit, 1414... partial facial expression category information storage unit, P1 etc... passersby (target person)

Claims

1. An information display unit disposed in a display unit that displays information intended for an unspecified number of people; An imaging unit that images a target person passing in front of the information display unit; A psychological state information estimation system having the information display unit and a management unit that manages the imaging unit, A stopping information storage unit that stores information about a stopped subject based on image information of the subject captured by the imaging unit; A face information storage unit that captures and stores face information of the subject who has stopped; an interest target information storage unit that stores the stopped target information having the face information facing the information display unit as interest target information having an interest in the information for the general public displayed on the information display unit; A psychological state information estimation system comprising: a psychological state information acquisition unit that acquires psychological state information of a person of interest based on head movement information, which is head movement information of the person of interest.

2. The psychological state information estimation system of claim 1, characterized in that the head movement information includes head shaking movement information, and the psychological state information of whether the person of interest affirms the information intended for an unspecified number of people is obtained based on the head shaking movement information.

3. The psychological state information estimation system of claim 1 or 2, characterized in that the head movement information includes head tilting movement information, head turning movement information, and head pulling movement information, and is configured to obtain the psychological state information other than whether or not the information intended for an unspecified number of people is positive based on this movement information.

4. The psychological state information estimation system according to claim 1 or claim 2, characterized in that the psychological state information acquisition unit acquires the psychological state information of the person of interest based on facial expression information, which is facial expression information of the person of interest.

5. The psychological state information estimating system according to claim 1 or 2, characterized in that the content of the information intended for an unspecified number of people displayed on the information display unit is time-unit change information that changes over time, image information of the subject captured by the imaging unit is stored in association with the time-unit change information, and information on the subject who has stopped, the facial information of the subject who has stopped, and information on the subject of interest are generated for each of the time-unit change information.

6. An information display unit disposed in a display unit that displays information intended for an unspecified number of people; An imaging unit that images a target person passing in front of the information display unit; a management unit that manages the information display unit and the imaging unit, The information display unit includes a partial information display unit which is each of a plurality of divided parts, a partial interest target information storage unit that stores information of a stopped target person having face information facing the partial information display unit as partial interest target person information having an interest in the information for the general public displayed on the partial information display unit; A psychological state information estimation system comprising: a partial psychological state information acquisition unit that acquires psychological state information of the partial subject of interest based on head movement information, which is head movement information of the partial subject of interest.

7. The psychological state information estimation system of claim 6, characterized in that the information display unit displays a single image information, the partial information display unit displays divided image information obtained by dividing the image information displayed on the information display unit, and the divided image information is combined to form the single image information.

8. An information display unit disposed in a display unit that displays information intended for an unspecified number of people; An imaging unit that images a target person passing in front of the information display unit; A control method for a psychological state information estimation system having the information display unit and a management unit that manages the imaging unit, comprising: storing information about the subject who has stopped in a stopping information storage unit based on imaging information about the subject captured by the imaging unit; A step of capturing an image of face information of the subject who has stopped and storing the image in a face information storage unit; storing the stopped target person information having the face information facing the information display unit in an interested target person information storage unit as interested target person information having an interest in the information for the general public displayed on the information display unit; A control method for a psychological state information estimation system, comprising: a step of acquiring psychological state information of a person of interest based on head movement information which is head movement information of the person of interest.

9. An information display unit disposed in a display unit that displays information intended for an unspecified number of people; An imaging unit that images a target person passing in front of the information display unit; A psychological state information estimation system having the information display unit and a management unit that manages the imaging unit, a function of storing information about a subject who has stopped in a stopping information storage unit based on image information of the subject captured by the imaging unit; A function of capturing an image of the face of the subject who has stopped and storing the image in a face information storage unit; a function of storing the stopped target information having the face information facing the information display unit in an interest target information storage unit as interest target information of a target person who is interested in the information for an unspecified large number of people displayed on the information display unit; A control program for a psychological state information estimation system, characterized by realizing a function of acquiring psychological state information of a subject of interest based on head movement information, which is head movement information of the subject of interest.

Citation Information

Patent Citations

  • Information processing system, information processing method, and information processing program

    JP2015060580A

  • Psychological state estimation device and program

    JP2022064726A

  • Display control device, speed control device, display control method, and program

    JP2022189378A

  • Content displaying method

    WO2021130852A1

  • Learning device and evaluation information output device

    JP2022120787A