Emotion inference device, emotion inference method, and program

Abstract

An emotion inference device according to one aspect of the present invention includes: a subjectivity information acquisition unit for acquiring subjectivity information related to subjectivity of a target person; a psychological characteristic database for containing information on a plurality of psychological characteristics related to a sense of unity; a psychological characteristic calculation unit for calculating, on the basis of the subjectivity information and the information in the psychological characteristic database, a plurality of psychological characteristic scores respectively corresponding to the plurality of psychological characteristics; a behavior information acquisition unit for acquiring behavior information related to behavior of the target person; a behavior information processing unit for calculating, on the basis of the behavior information, target person behavior data by analyzing the behavior of the target person; an event information acquisition unit for acquiring event information related to the content of an event; an audience information acquisition unit for acquiring audience information related to an audience around the target person; an audience information processing unit for calculating color phase information on the entire audience at an event site on the basis of the audience information; and a sense-of-unity calculation unit for calculating sense-of-unity inference data in which the psychological characteristics related to the sense of unity are quantified by using the plurality of psychological characteristic scores, the target person behavior data, and the color phase information.

Description

Emotion estimation device, emotion estimation method, and program

[0001] One aspect of the present invention relates to an emotion estimation device, an emotion estimation method, and a program.

[0002] One of the indicators for evaluating the experience quality of events attended by a large audience, such as a live music concert, is the sense of unity felt by the audience. It is said that the sense of unity among the audience significantly influences the quality of the experience, and methods for improving the sense of unity among the audience have been studied (e.g., Non-Patent Document 1). Remote viewing via online video distribution has become particularly popular in recent years, but it is said that it is harder to feel a sense of unity among the audience when viewing remotely than when participating in person, and techniques for improving the sense of unity when viewing remotely have also been studied (e.g., Non-Patent Document 2).

[0003] Oshimi and Harada, "A Scale for Emotional Scenes in Sports Spectators," Sports Management Research, vol. 2, no. 2, pp. 163-178, 2010. Oshiro et al., "Evaluating the Effect of Active Swinging Movements on the Sense of Unity with the Audience During Remote Live Music Viewing," IEICE Technical Report, HCS2022-55(2023-01), pp. 1-6.

[0004] To measure or estimate the sense of unity felt by audiences at events, information such as the sounds and behaviors of the entire venue or all participants is often used. However, the sense of unity felt by each individual participant depends on their psychological characteristics, such as their personality and way of thinking, and even within the same event, the sense of unity felt by each individual varies. One method for investigating the sense of unity felt by individuals is through questionnaire surveys, but conducting surveys during an event is difficult, making it difficult to estimate a sense of unity synchronized with the event. Furthermore, no technology has been established to estimate an individual's sense of unity synchronized with an event while taking into account the relationship between individual psychological characteristics and events occurring within the event.

[0005] The present invention has been made in light of the above-mentioned circumstances, and an object of the present invention is to provide an emotion estimation device, an emotion estimation method, and a program that are capable of quantifying psychological characteristics related to a sense of unity.

[0006] an audience information acquisition unit that acquires event information related to content of an event; an audience information acquisition unit that acquires audience information related to audiences around the subject; an audience information processing unit that calculates hue information of the entire audience at a venue based on the audience information; and a sense of unity calculation unit that calculates sense of unity estimation data that quantifies psychological characteristics related to sense of unity, using the plurality of psychological characteristic scores, the subjective information, and the hue information.

[0007] According to one aspect of the present invention, it is possible to provide an emotion estimation device, an emotion estimation method, and a program capable of quantifying psychological characteristics related to a sense of unity.

[0008] FIG. 1 is a block diagram of an emotion estimation device according to a first embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of the hardware configuration of the emotion estimation device. FIG. 3 is a flowchart illustrating the operation of the emotion estimation device. FIG. 4 is a schematic diagram illustrating the operation of the emotion estimation device. FIG. 5 is a diagram illustrating an example of the contents of a questionnaire. FIG. 6 is a diagram illustrating an example of the contents of a psychological characteristic database. FIG. 7 is a diagram illustrating an example of subject behavior data. FIG. 8 is a diagram illustrating an example of subject volume data. FIG. 9 is a diagram illustrating an example of event story data. FIG. 10 is a diagram illustrating an example of audience color data. FIG. 11 is a diagram illustrating an example of audience volume data. FIG. 12 is a diagram illustrating an example of excitement timing data and melody timing data. FIG. 13 is a block diagram of an emotion estimation device according to a second embodiment of the present invention. FIG. 14 is a block diagram illustrating an example of the hardware configuration of the emotion estimation device. FIG. 15 is a flowchart illustrating the operation of the emotion estimation device.

[0009] Hereinafter, embodiments will be described with reference to the drawings. In the following description, elements having the same functions and configurations are designated by the same reference numerals, and redundant description will be omitted.

[0010] [1] First Embodiment [1-1] Configuration of the Emotion Estimation Device 1 FIG. 1 is a block diagram of the emotion estimation device 1 according to a first embodiment of the present invention. The emotion estimation device 1 is a device that estimates the emotions of subjects who have participated in an event such as a live music concert. The emotion estimation device 1 includes a subjective information acquisition unit 10, a behavioral information acquisition unit 20, an audience information acquisition unit 30, an event information acquisition unit 40, a control unit 50, a program storage unit 60, a data storage unit 61, and an output unit 62.

[0011] The subjective information acquiring unit 10 acquires information relating to the subject's subjectivity. The subjective information acquiring unit 10 acquires the information relating to the subject's subjectivity, for example, by using a questionnaire administered to the subject. The questionnaire is administered, for example, before an event.

[0012] The behavioral information acquisition unit 20 acquires information about the behavior and vocalizations of the subject. The behavioral information acquisition unit 20 includes, for example, a plurality of acceleration sensors attached to the subject's arms, chest, and thighs, and a microphone positioned near the subject's mouth or hand.

[0013] The audience information acquisition unit 30 acquires information about the audience around the subject. The audience information acquisition unit 30 includes, for example, a camera that captures still images or video of the entire venue, and a microphone that acquires sound information of the entire venue.

[0014] The event information acquisition unit 40 acquires information about the content of an event. The information about the content of an event is acquired, for example, before the event.

[0015] The program storage unit 60 stores a program for the control unit 50 to execute a predetermined process.

[0016] The data storage unit 61 stores data necessary for processing by the control unit 50. The data storage unit 61 includes a psychological trait database (also referred to as a psychological trait DB) 61A. The psychological trait database 61A stores information on psychological traits related to a sense of unity. The information on psychological traits stored in the psychological trait database 61A includes weighted scores corresponding to each of a plurality of psychological traits.

[0017] The control unit 50 includes a psychological characteristic calculation unit 51, a behavior information processing unit 52, an audience information processing unit 53, and a sense of unity calculation unit 54.

[0018] The psychological characteristic calculation section 51 calculates a plurality of scores corresponding to a plurality of psychological characteristics, respectively, based on the information acquired by the subjective information acquisition section 10 and the information read out from the psychological characteristic database 61A.

[0019] The behavioral information processing unit 52 analyzes the information acquired by the behavioral information acquiring unit 20. The behavioral information processing unit 52 determines the behavior of the subject and the volume of the voice emitted by the subject.

[0020] The audience information processing unit 53 performs color information processing, volume information processing, and melody information processing based on the information acquired by the audience information acquisition unit 30. In color information processing, the audience information processing unit 53 analyzes still images or video captured of the entire venue. Based on the analysis results, the audience information processing unit 53 calculates hue information of the entire audience in the venue. In volume information processing, the audience information processing unit 53 calculates information on sounds emitted by the audience. In melody information processing, the audience information processing unit 53 calculates information on the timing of cheers and applause, and information on the timing of singing and clapping.

[0021] The sense of unity calculation unit 54 calculates sense of unity estimation data that quantitatively estimates the sense of unity felt by the subject during the event at an actual live venue based on the data sent from the psychological characteristic calculation unit 51, the behavioral information processing unit 52, the audience information processing unit 53, and the event information acquisition unit 40.

[0022] The output unit 62 outputs the sense of unity estimation data calculated by the sense of unity calculation unit 54. The output unit 62 is configured by, for example, a display device. The display device displays the sense of unity estimation data in text or graph form.

[0023] (Hardware Configuration of Feeling Estimation Device 1) FIG. 2 is a block diagram showing an example of the hardware configuration of the feeling estimation device 1.

[0024] The feeling estimation device 1 can be configured by a computer. The feeling estimation device 1 includes a processor 63, a data storage unit 61, a program storage unit 60, a communication interface unit (also referred to as a communication I / F unit) 64, and an input / output interface unit (also referred to as an input / output I / F unit) 65. The data storage unit 61, the program storage unit 60, the communication interface unit 64, and the input / output interface unit 65 are connected to the processor 63 via a bus 66.

[0025] The processor 63 is configured by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), etc. The processor 63 executes the operations of the control unit 50 described above.

[0026] The program storage unit 60 includes, for example, a non-volatile memory that can be written to and read from as needed, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read-only memory (ROM). The program storage unit 60 stores programs necessary for the processor 63 to execute various processes according to this embodiment. That is, the psychological characteristic calculation unit 51, the behavior information processing unit 52, the audience information processing unit 53, and the sense of unity calculation unit 54 described above are realized by having the processor 63 execute the programs stored in the program storage unit 60.

[0027] The data storage unit 61 includes, for example, a non-volatile memory such as an HDD or SSD, and a volatile memory such as a RAM (Random Access Memory). The data storage unit 61 is also used as a working area for storing various data acquired and created in the course of the processor 63 executing various processes.

[0028] The communication interface unit 64 includes a wired communication module and / or a wireless communication module. The wireless communication module includes a wireless LAN (Local Area Network). The communication interface unit 64 executes interface processing in accordance with a predetermined communication standard with an external device.

[0029] An input device and an output device are connected to the input / output interface unit 65. The input device includes, for example, a keyboard and a mouse. The output device includes, for example, a liquid crystal display device or an organic EL (Electro Luminescence) display device. The input / output interface unit 65 performs interface processing for the input device and the output device in accordance with a predetermined standard. The input / output interface unit 65 (and the multiple devices connected thereto) is configured to be able to execute the functions of the subjective information acquisition unit 10, the behavior information acquisition unit 20, the audience information acquisition unit 30, the event information acquisition unit 40, and the output unit 62.

[0030] [1-2] Operation Next, the operation of the emotion estimation device 1 configured as described above will be described. FIG. 3 is a flowchart illustrating the operation of the emotion estimation device 1. FIG. 4 is a schematic diagram illustrating the operation of the emotion estimation device 1. The emotion estimation device 1 estimates the sense of unity felt by subjects during an event at an actual live performance venue. Note that in the following description, the order of multiple steps for acquiring various types of information may be changed as appropriate.

[0031] The subjective information acquisition unit 10 acquires information about the subject's subjectivity (subject subjective information) (step S100). In this embodiment, a questionnaire is conducted in advance on the subject, and response data for a plurality of questions regarding how the subject interacts with people around them is acquired in advance. Figure 5 is a diagram illustrating an example of the contents of the questionnaire.

[0032] The questionnaire includes, for example, 10 questions (Q1 to Q10). The 10 questions are composed of questions related to the relationship between an individual and a group. The number of questions is not limited to 10, but it is desirable to have about 10 to 30 questions. The subject answers each of the 10 questions using a five-point scale: "1 = not applicable," "2 = slightly not applicable," "3 = neither applicable nor unapplicable," "4 = slightly applicable," and "5 = applicable." The answers are judged by a score of 1 to 5 (answer score). The subjective information acquisition unit 10 calculates the answer score for each n-th question (in this embodiment, n = 1, 2, ... 10) as subjective information q ｎ and subjective information q ｎ to the psychological characteristic calculation unit 51. The method of conducting the questionnaire may be to obtain the information from responses to a paper questionnaire conducted in advance, or to prepare an input screen and have the subjects input the information.

[0033] Next, the psychological characteristic calculation unit 51 reads out the psychological characteristic database 61A from the data storage unit 61 (step S101). The psychological characteristic database 61A stores information on psychological characteristics related to a sense of unity (psychological characteristic information). Fig. 6 is a diagram illustrating an example of the contents of the psychological characteristic database 61A.

[0034] The psychological characteristic database 61A defines a plurality of psychological characteristics relating to the relationship between an individual and a group. In this embodiment, as an example, five types of psychological characteristics are defined: "susceptibility to influence," "normative attribution," "emotional attribution," "membership," and "collective self-esteem." In the psychological characteristic database 61A, weighted scores for each of the five psychological characteristics are defined for each question item used in the subjective information acquisition unit 10. The weighted scores for "susceptibility to influence," "normative attribution," "emotional attribution," "membership," and "collective self-esteem" for the n-th question item are respectively defined as i ｎ , n ｎ , a ｎ , m ｎ , g ｎ It is defined as:

[0035] The names of the multiple psychological traits are not limited to those described above and may be expressed differently. The number of psychological traits stored in the psychological trait database 61A is, for example, at least five and preferably not more than ten. The weighting points are merely examples, and different values ​​can be set as long as the relationship between the psychological traits and the question items is not impaired.

[0036] Next, the psychological characteristic calculation unit 51 calculates the subjective information q of the subject acquired by the subjective information acquisition unit 10. ｎ and the weighted score i defined in the psychological characteristics database 61A. ｎ , n ｎ , a ｎ , m ｎ , g ｎ Using the above, a plurality of scores (psychological trait scores) corresponding to the plurality of psychological traits are calculated by the following calculation (step S102): Ｉ is the susceptibility score, score Ｎ is the normative attribution score, Ａ is the score of affective attribution, Ｍ is the membership score, score Ｇ represents the collective self-esteem score.

[0037] score Ｉ =Σ n i ｎ q ｎ score Ｎ =Σ n n ｎ q ｎ score Ａ =Σ n a ｎ q ｎ score Ｍ =Σ n m ｎ q ｎ score Ｇ =Σ n g ｎ q ｎ "Σ n " means the sum of the calculation formulas for n=1, 2, . . . , 10. The psychological characteristic calculation unit 51 sends the calculated psychological characteristic scores to the sense of unity calculation unit 54.

[0038] Next, the behavioral information acquisition unit 20 acquires information about the subject's behavior and vocalizations (step S103). In this embodiment, the behavioral information acquisition unit 20 includes a plurality of acceleration sensors 21 attached to the subject's arms, chest, and thighs, and a microphone 22 positioned near the subject's mouth or hands. The plurality of acceleration sensors 21 acquire information about the subject's behavior (behavioral information). The microphone 22 acquires information about the subject's vocalizations (vocal information). The information acquired by the plurality of acceleration sensors 21 and the microphone 22 is sent to the behavioral information processing unit 52.

[0039] Next, the behavior information processing unit 52 analyzes the behavior information and vocalization information acquired by the behavior information acquisition unit 20. In this embodiment, the behavior information processing unit 52 uses the behavior information of the multiple acceleration sensors 21 to determine whether or not the subject stands up, swings his / her arms, and claps. The behavior information processing unit 52 chronologically analyzes the subject behavior data m Ｓ (Step S104). Ｓ Specifically, the behavior information processing unit 52 generates row vector data in which each of the cases where it is determined that there was standing up, swinging of arms, and clapping is scored as 1 in a time series, and the data consisting of the sum of the scores for standing up, swinging of arms, and clapping is defined as the subject behavior data m Ｓ The time series is, for example, at 5 minute intervals (determination is made every 5 minutes), and the measurement time is 1 minute.

[0040] The behavior information processing unit 52 also records the volume (dB) of the subject's voice at five-minute intervals using the vocal information from the microphone 22. This volume information is referred to as subject volume data v Ｓ The behavior information processing unit 52 calculates the target person volume data v Ｓ (Step S104). Ｓ 1 is a diagram illustrating an example of target person volume data v Ｓ is row vector data consisting of time-series information. The behavior information processing unit 52 then processes the subject behavior data m Ｓ and subject volume data v Ｓ is sent to the unity calculation unit 54.

[0041] Next, the event information acquisition unit 40 acquires information (event information) related to the content of the event (step S105). Specifically, the event information acquisition unit 40 acquires event information related to the time of excitement along the timeline of the event. The definition of excitement is determined by the event provider, and is set so that the higher the excitement, the higher the positive value. The event information acquisition unit 40 generates event story data s using the event information. FIG. 9 is a diagram illustrating an example of the event story data s. The event story data s is row vector data consisting of time-series information and is generated so as to have the same time intervals as the time-series data generated by the behavior information processing unit 52 and the audience information processing unit 53. The event story data s is set on a five-point scale from 1 point to 5 points. The event story data s may be defined in advance based on the content of the event, or may be generated by recording the excitement during the event. The event story data s acquired and generated by the event information acquisition unit 40 is sent to the unity calculation unit 54.

[0042] Next, the audience information acquisition unit 30 acquires information (audience information) about the audience around the subject (step S106). In this embodiment, the audience information acquisition unit 30 includes a camera 31 that captures still images or videos of the entire venue, and a microphone 32 that acquires sound information from the entire venue. The camera 31 and the microphone 32 are attached at predetermined positions in the venue. The information acquired by the camera 31 and the microphone 32 is sent to the audience information processing unit 53.

[0043] The audience information processing unit 53 performs color information processing, volume information processing, and melody information processing using the still image or video captured by the camera 31 and the sound information (volume information) captured by the microphone 32 (step S107).

[0044] In color information processing, the crowd information processing unit 53 analyzes still images or video captured of the entire venue and calculates hue information for the entire audience at the venue. Specifically, the crowd information processing unit 53 uses the RGB values ​​of the image to calculate HSV values ​​related to hue, saturation, and brightness. RGB is a method of expressing color using the three primary colors of red, green, and blue. HSV is a method of expressing color using the three elements of hue, saturation, and brightness. The crowd information processing unit 53 defines multiple color categories based on combinations of hue, saturation, and brightness, and classifies the image into the multiple color categories. The crowd information processing unit 53 calculates the distribution ratios of the multiple color categories in the entire image and calculates the percentage value of the color category with the highest distribution ratio. This percentage value is used as crowd color data c Ｏ FIG. 10 shows the crowd color data c Ｏ FIG. 10 is a diagram illustrating an example of crowd color data c Ｏ is row vector data consisting of time-series information. Ｏ is the aforementioned data m Ｓ , v Ｓ The crowd color data c is calculated by analyzing the color of the video at regular intervals. Ｏ The crowd information processing unit 53 may calculate the crowd color data c Ｏ is sent to the unity calculation unit 54.

[0045] In the volume information processing, the audience information processing unit 53 records the volume data of the sounds (including applause, cheers, laughter, singing, etc.) picked up by the microphone 32 and emitted by the audience at the same time intervals as described above. At this time, the volume of the sounds originating from the performers may be included or omitted. This volume data is referred to as the audience volume data v Ｏ The audience information processing unit 53 calculates the audience volume data v Ｏ 11 is a graph showing the crowd volume data v Ｏ 1 is a diagram illustrating an example of audience volume data v Ｏ is row vector data consisting of time-series information. The audience information processing unit 53 processes the audience volume data v Ｏ is sent to the unity calculation unit 54.

[0046] In the melody information processing, the audience information processing unit 53 determines the cheers and applause of the audience, and further determines singing and clapping, using volume information, frequency, etc. acquired by the microphone 32. The audience information processing unit 53 determines the timing at which cheers and applause occur as "excitement timing" and generates excitement timing data e at the same time intervals as described above. Ｏ The audience information processing unit 53 also calculates the melody timing data m at the same time intervals as described above, regarding the timing at which singing and clapping occurs as "melody timing." Ｏ 12 shows the timing data e Ｏ and melody timing data m Ｏ FIG. 10 is a diagram illustrating an example of excitement timing data e Ｏ and melody timing data m Ｏ are row vector data consisting of time-series information. Ｏ and melody timing data m Ｏ is sent to the unity calculation unit 54.

[0047] Next, the sense of unity calculation unit 54 calculates sense of unity estimation data u using the data sent from the behavior information processing unit 52, the audience information processing unit 53, and the event information acquisition unit 40, and the multiple psychological characteristic scores calculated by the psychological characteristic calculation unit 51 (step S108). The sense of unity estimation data u is data that quantitatively estimates the sense of unity felt by subjects during an event at an actual live venue. The sense of unity estimation data u is row vector data consisting of the time series described above.

[0048] First, the sense of unity calculation unit 54 performs the following calculations using the data sent from the behavior information processing unit 52 and the audience information processing unit 53 to determine the "behavior and vocalizations of the subject," "excitement of the surroundings," "homogeneity of the group," and "excitement phase."

[0049] Subject's behavior and vocalizations: Ｓ +v Ｓ Excitement around: e Ｏ +v Ｏ Group homogeneity: c Ｏ +m ＯExciting phase: s Next, the togetherness calculation unit 54 performs weighting using a plurality of psychological characteristic scores using equation (1) to calculate togetherness estimation data u.

[0050]

[0051] The above calculation is an example, and the combination of the psychological characteristic score and the row vector data does not necessarily have to be as described above. The sense of unity calculation unit 54 sends the sense of unity estimation data u to the output unit 62.

[0052] Next, the output unit 62 outputs the sense of togetherness estimation data u in a predetermined format including text, graphs, or the like (step S109). The output unit 62 may calculate and output an average value of the sense of togetherness estimation data u throughout the event or for each time period. For example, the output unit 62 displays the sense of togetherness estimation data u on a display device. The output unit 62 may transmit the sense of togetherness estimation data u to an external device using a communication function.

[0053] [1-3] Effects of the First Embodiment According to the first embodiment, the emotion estimation device 1 can quantify psychological characteristics related to a sense of unity that differ depending on the subject. Furthermore, the emotion estimation device 1 can quantitatively estimate the sense of unity that a subject felt when participating in an event along a time series, based on the relationship between the subject's own behavior during the event, the content of the event, and information about the entire audience. Furthermore, the emotion estimation device 1 can calculate data on the estimated sense of unity as sense of unity estimation data u. Furthermore, the emotion estimation device 1 can externally output the sense of unity estimation data u in a predetermined format.

[0054] [2] Second Embodiment The second embodiment assumes that a subject is remotely watching a live music concert using an internet live streaming service, etc. In the second embodiment, the psychological characteristics related to the sense of unity are quantified using the video of the live stream and information in the chat box.

[0055] 13 is a block diagram of the emotion estimation device 1 according to the second embodiment of the present invention. The emotion estimation device 1 includes a subjective information acquisition unit 10, a behavioral information acquisition unit 20, an audience information acquisition unit 30, an event information acquisition unit 40, a control unit 50, a program storage unit 60, a data storage unit 61, a display unit 70, and a communication unit 71.

[0056] The communication unit 71 is connected to a communication network 72. The communication network 72 includes the Internet. The communication unit 71 is capable of communicating with a distribution server (not shown) or the like via the communication network 72. The distribution server distributes live video to specific users using the communication network 72. The communication unit 71 receives video data, volume data, and the like from the communication network 72.

[0057] The display unit 70 is an interface that displays the live streaming video and chat box. The control unit 50 controls the display unit 70 to display the video and chat box.

[0058] The audience information acquisition unit 30 acquires the video and volume of the live broadcast displayed on the display unit 70. The audience information acquisition unit 30 also acquires information on the chat box included in the live broadcast displayed on the display unit 70.

[0059] As in the first embodiment, the audience information processing unit 53 performs color information processing, volume information processing, and melody information processing based on the information acquired by the audience information acquisition unit 30. Furthermore, the audience information processing unit 53 performs chat information processing based on the information acquired by the audience information acquisition unit 30. In the chat information processing, the audience information processing unit 53 calculates information related to the text volume of the chat.

[0060] The other configurations are the same as those of the first embodiment.

[0061] 14 is a block diagram showing an example of the hardware configuration of the feeling estimation device 1. The feeling estimation device 1 includes a processor 63, a data storage unit 61, a program storage unit 60, a communication interface unit 64, an input / output interface unit 65, a display device 73, and an input device 74.

[0062] The communication interface unit 64 performs interface processing with the communication network 72 .

[0063] A display device 73 and an input device 74 are connected to the input / output interface unit 65. The display device 73 displays images and videos. The display device 73 displays live streaming videos and a chat section. The sound of the live streaming is output, for example, by a speaker included in the display device 72. The input / output interface unit 65 and the display device 73 can execute the functions of the display unit 70 described above. The input device 74 includes a keyboard and a mouse. The input device 74 can input chat information.

[0064] The other configurations are the same as those of the first embodiment.

[0065] [2-2] Operation Next, we will explain the operation of the feeling estimation device 1 configured as above. Fig. 15 is a flowchart explaining the operation of the feeling estimation device 1. The operations of steps S200 to S205 in Fig. 15 are the same as the operations of steps S100 to S105 in the first embodiment.

[0066] Next, the audience information acquisition unit 30 acquires the video of the live stream, information about the sound of the live stream (volume information), and information in the chat box included in the live stream (chat information) (step S206). The audience information acquisition unit 30 may acquire the information about the live stream from the display unit 70 or from the communication unit 71.

[0067] Next, the audience information processing unit 53 performs color information processing, volume information processing, melody information processing, and chat information processing using the information acquired by the audience information acquisition unit 30 (step S207). The color information processing, volume information processing, and melody information processing are the same as those in the first embodiment.

[0068] In chat information processing, the audience information processing unit 53 determines the amount of text in the chat at the same time intervals as the time series data described above, and outputs the determined amount of chat data t Ｏ Calculate the chat data t Ｏis row vector data consisting of time-series information. Ｏ , crowd volume data v Ｏ , excitement timing data e Ｏ , melody timing data m Ｏ , and chat data t Ｏ is sent to the unity calculation unit 54.

[0069] Next, the sense of unity calculation unit 54 calculates the sense of unity estimation data u (step S208) using the data sent from the behavior information processing unit 52, the audience information processing unit 53, and the event information acquisition unit 40, and the multiple psychological characteristic scores calculated by the psychological characteristic calculation unit 51. Specifically, the sense of unity calculation unit 54 performs weighting using the multiple psychological characteristic scores using equation (2) to calculate the sense of unity estimation data u consisting of row vector data.

[0070]

[0071] The above calculation is an example, and the combination of the psychological characteristic score and the row vector data does not necessarily have to be as described above. For example, if the chat characteristic is highly emotional, the unity estimation data u may be calculated using equation (3).

[0072]

[0073] Furthermore, when chat has strong rules as its characteristics, the sense of unity estimation data u may be calculated using equation (4).

[0074]

[0075] Next, the display unit 70 displays the togetherness estimation data u in a predetermined format, such as text or a graph (step S209). An average value of the togetherness estimation data u throughout the event or for each time period may be calculated and displayed. Note that, as in the first embodiment, an output unit (not shown) may output the togetherness estimation data u in a predetermined format.

[0076] [2-3] Effects of the Second Embodiment According to the second embodiment, the feeling estimation device 1 can quantify psychological characteristics related to a sense of unity even in live streaming. Furthermore, the feeling estimation device 1 can calculate the sense of unity estimation data u by further using chat information.

[0077] [3] Modifications The processes according to the above-described embodiments can be stored as a program (software means) that can be executed by a computer on a storage medium such as a magnetic disk, optical disk, or semiconductor memory, or can be transmitted and distributed via a communication medium. The storage medium includes a storage medium provided in the computer or a storage medium provided in a device connected via a network. The computer can then load the program stored in the storage medium and execute the above-described processes by having its operation controlled by the loaded program.

[0078] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0079] DESCRIPTION OF SYMBOLS 1...Emotion estimation device 10...Subjective information acquisition unit 20...Behavioral information acquisition unit 21...Acceleration sensor 22...Microphone 30...Audience information acquisition unit 31...Camera 32...Microphone 40...Event information acquisition unit 50...Control unit 51...Psychological characteristic calculation unit 52...Behavioral information processing unit 53...Audience information processing unit 54...Sense of unity calculation unit 60...Program storage unit 61...Data storage unit 61A...Psychological characteristic database 62...Output unit 63...Processor 64...Communication interface unit 65...Input / output interface unit 66...Bus 70...Display unit 71...Communication unit 72...Communication network 73...Display device 74...Input device

Claims

an audience information acquisition unit that acquires crowd information about the audience around the subject; an audience information processing unit that calculates hue information of the entire audience in a venue based on the crowd information; and a unity calculation unit that calculates unity estimation data that quantifies psychological characteristics related to unity, using the plurality of psychological characteristic scores, the subjective information, and the information in the psychological characteristic database.

2. The emotion estimation device according to claim 1, wherein the sense of unity estimation data is row vector data consisting of time-series information.

3. The emotion estimation device according to claim 1, wherein the plurality of psychological characteristics include information on susceptibility to influence, normative attribution, emotional attribution, membership, and collective self-esteem.

4. The emotion estimation device described in claim 1, wherein the behavioral information acquisition unit further acquires vocalization information regarding the vocalization of the subject, the behavioral information processing unit calculates subject volume data regarding the volume of the vocalization of the subject based on the vocalization information, and the sense of unity calculation unit further uses the subject volume data to calculate the sense of unity estimation data.

5. The emotion estimation device described in claim 1, wherein the audience information processing unit calculates crowd volume data related to sounds emitted by the crowd based on the audience information, and the sense of unity calculation unit further uses the crowd volume data to calculate the sense of unity estimation data.

6. The emotion estimation device described in claim 1, wherein the audience information acquisition unit acquires chat information regarding chats included in the live streaming video, the audience information processing unit calculates chat data regarding the amount of text in the chat based on the chat information, and the sense of unity calculation unit further uses the chat data to calculate the sense of unity estimation data.

7. The emotion estimation device according to claim 1, further comprising an output unit that outputs the sense of unity estimation data.

8. An emotion estimation method comprising: acquiring subjective information regarding a subject's subjectivity; storing information on a plurality of psychological characteristics related to a sense of unity; calculating a plurality of psychological characteristic scores corresponding to the plurality of psychological characteristics based on the subjective information and the information on the plurality of psychological characteristics; acquiring behavioral information regarding the behavior of the subject; analyzing the behavior of the subject based on the behavioral information to calculate subject behavior data; acquiring event information regarding the content of an event; acquiring crowd information regarding the crowd around the subject; calculating hue information of the entire crowd in the venue based on the crowd information; and calculating unity estimation data that quantifies psychological characteristics related to a sense of unity using the plurality of psychological characteristic scores, the subject behavior data, and the hue information.

9. A program for causing a computer to function as each part of the emotion estimation device according to any one of claims 1 to 7.

Images