Emotion difference detection device, emotion difference detection method, and program

Abstract

To provide an apparatus that determines whether a subject's emotional expression is different from usual.SOLUTION: A difference detection apparatus of the present invention includes an emotional information acquisition unit that acquires emotional information of a subject, a reference emotional count unit that counts a reference emotion generation number based on the emotional information, an emotional count unit that counts an emotional generation number based on the emotional information, an abnormal value detection unit that detects whether the emotional generation number includes an abnormal value with respect to the reference emotion generation number, a determination unit that determines that when the abnormal value is included in the emotion generation number, a current emotional expression of the subject is different from usual, and an output unit that outputs a determination result. The emotional information includes an emotion of the subject and identification information of the subject. The reference emotion generation number is a generation number of the emotion of the subject per unit time in a first predetermined period, and the emotion generation number is a generation number of the emotion of the subject per unit time in a second predetermined period.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an emotion difference detection device, an emotion difference detection method, and a program. [Background technology]

[0002] When communicating face-to-face, it is possible to detect if the other person's behavior is different from usual from subtle differences in facial expressions and gestures, but when communicating remotely via a camera, etc., it is difficult to grasp the subtleties of the other person's facial expressions and emotions. Therefore, a technology is known that estimates the emotions of the person communicating from their facial expressions, etc. (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-133447 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are individual differences in how emotions are expressed, and there was a problem in that even if emotions were estimated using a uniform standard, it was not possible to know whether they were different from normal states.

[0005] Therefore, an object of the present invention is to provide a device that determines whether the way a subject's emotions are expressed is different from usual. [Means for solving the problem]

[0006] In order to achieve the above object, the emotion difference detection device of the present invention comprises: The system includes an emotion information acquisition unit, a reference emotion counting unit, an emotion counting unit, an abnormal value detection unit, a determination unit, and an output unit, the emotion information acquisition unit acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting unit counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting unit counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection unit detects whether the number of emotion occurrences includes an abnormal value with respect to the reference number of emotion occurrences; When the number of times of emotion occurrence includes an abnormal value, the determination unit determines that the subject's current emotion is being expressed differently than usual; The output unit outputs the determination result.

[0007] The emotion difference detection method of the present invention includes: The method includes an emotion information acquisition step, a reference emotion counting step, an emotion counting step, an outlier detection step, a determination step, and an output step, the emotion information acquiring step acquires emotion information of a subject, the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection step detects whether the number of occurrences of an emotion includes an abnormal value relative to the reference number of occurrences of an emotion, The determining step determines that the subject's current emotional expression is different from usual when the number of times the emotion is generated includes an abnormal value, The output step outputs the determination result.

[0008] The program of the present invention is a program for causing a computer to execute an emotion information acquisition procedure, a reference emotion counting procedure, an emotion counting procedure, an outlier detection procedure, a judgment procedure, and an output procedure, the emotion information acquisition step acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection step detects whether the number of occurrences of emotions includes an abnormal value relative to the reference number of occurrences of emotions; The determination step determines that the subject's current emotional expression is different from usual when the number of times the emotion occurs includes an abnormal value; The output procedure is a program characterized by outputting the determination result. [Effects of the Invention]

[0009] According to the present invention, it is possible to determine whether the subject's emotional expression is different from usual. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of the difference detection device according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of the difference detection device according to the first embodiment. [Figure 3] FIG. 3 is a flowchart showing an example of processing in the difference detection device according to the first embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the difference detection device according to the second embodiment. [Figure 5]FIG. 5 is a flowchart showing an example of processing in the difference detection device according to the second embodiment. [Figure 6] FIG. 6 is a block diagram illustrating an example of the configuration of the estimation unit in the difference detection device of the second embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of the hardware configuration of the estimation unit in the difference detection device according to the second embodiment. [Figure 8] FIG. 8 is a flowchart illustrating an example of processing by the estimation unit in the difference detection device according to the second embodiment. [Figure 9] FIG. 9 is an explanatory diagram regarding estimation of emotional changes by the estimation unit. DETAILED DESCRIPTION OF THE INVENTION

[0011] Embodiments of the present invention will be described with reference to the drawings. The present invention is not limited to the following embodiments. In the following drawings, the same parts are denoted by the same reference numerals. Furthermore, the descriptions of the embodiments can be mutually incorporated unless otherwise specified, and the configurations of the embodiments can be combined unless otherwise specified.

[0012] [Embodiment 1] Fig. 1 is a block diagram showing an example of the configuration of a difference detection device 10 according to this embodiment. As shown in Fig. 1, this device 10 includes an emotion information acquisition unit 11, a reference emotion counting unit 12, an emotion counting unit 13, an abnormal value detection unit 14, a determination unit 15, and an output unit 16. Although not shown, this device 10 may also include a storage unit.

[0013] The device 10 may be, for example, a single device including the above-described units, or a device in which the units can be connected via a communication network. The device 10 can also be connected to an external device (described later) via the communication network. The communication network is not particularly limited and any known network can be used, for example, a wired or wireless network. Examples of the communication network include the Internet, the World Wide Web (WWW), a telephone line, a Local Area Network (LAN), a Storage Area Network (SAN), a Delay Tolerant Networking (DTN), a Low Power Wide Area Network (LPWA), and a Local 5G (L5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), Local 5G, and LPWA. Examples of the wireless communication include direct communication between devices (Ad Hoc communication), infrastructure communication, and indirect communication via an access point. The device 10 may be incorporated into a server as a system. Furthermore, the device 10 may be, for example, a personal computer (PC, for example, a desktop or notebook type) on which the program of the present invention is installed, a smartphone, a tablet terminal, etc. The device 10 may be in the form of cloud computing or edge computing, for example, in which at least one of the above-mentioned units is located on a server and the other units are located on a terminal.

[0014] 2 shows a block diagram of the hardware configuration of the device 10. The device 10 includes, for example, a CPU 101, a memory 102, a bus 103, a storage device 104, an input device 106, a display device 107, and a communication device 108. The components of the device 10 are connected to each other via the bus 103 and their respective interfaces (I / F).

[0015] CPU 101 operates in cooperation with other components via a controller (system controller, I / O controller, etc.) and the like, and is responsible for overall control of device 10. In device 10, CPU 101 executes, for example, program 105 of the present invention and other programs, and also reads and writes various types of information. Specifically, CPU 101 functions as emotion information acquisition unit 11, reference emotion counter 12, emotion counter 13, abnormal value detection unit 14, judgment unit 15, and output unit 16. While device 10 includes a CPU as a computing device, it may also include other computing devices such as a GPU (Graphics Processing Unit) or APU (Accelerated Processing Unit), or may include a combination of a CPU and these.

[0016] The bus 103 can also be connected to, for example, external devices. Examples of the external devices include an external storage device (such as an external database), a printer, an external input device, an external display device, and an external imaging device. The device 10 can be connected to an external network (the communication line network) by, for example, a communication device 108 connected to the bus 103, and can also be connected to other devices via the external network.

[0017] An example of the memory 102 is a main memory (primary storage device). When the CPU 101 performs processing, the memory 102 reads various operating programs, such as the program of the present invention, stored in the storage device 104 (described later), and the CPU 101 receives data from the memory 102 and executes the programs. The main memory is, for example, a RAM (random access memory). The memory 102 may also be, for example, a ROM (read only memory).

[0018] The storage device 104 is also referred to as an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 104 stores an operating program including the program 105 of the present invention. The storage device 104 may be, for example, a combination of a recording medium and a drive for reading and writing data from and to the recording medium. The recording medium is not particularly limited and may be, for example, an internal or external type, such as a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, or memory card. The storage device 104 may be, for example, a hard disk drive (HDD) or a solid-state drive (SSD) in which the recording medium and drive are integrated. When the device 10 includes the storage unit, the storage device 104 functions as the storage unit, for example.

[0019] In the present device 10, the memory 102 and the storage device 104 can also store various information such as log information, information acquired from an external database (not shown) or an external device, information generated by the present device 10, and information used when the present device 10 executes processing. In this case, the memory 102 and the storage device 104 may store at least one piece of information selected from the group consisting of, for example, emotion information of the subject, identification information of the subject, attribute information of the subject, the number of times a reference emotion occurs, the number of times an emotion occurs, a score threshold, etc., which will be described later. Note that at least a portion of the information may be stored, for example, in an external server other than the memory 102 and the storage device 104, or may be stored in a distributed manner across multiple terminals using blockchain technology or the like.

[0020] The device 10 further includes, for example, an input device 106 and a display 107. Examples of the input device 106 include pointing devices such as a touch panel, track pad, and mouse; a keyboard; imaging means such as a camera and scanner; a card reader such as an IC card reader and a magnetic card reader; and audio input means such as a microphone. Examples of the display 107 include display devices such as an LED display and a liquid crystal display. In the first embodiment, the input device 106 and the display 107 are configured separately, but the input device 106 and the display 107 may be configured as an integrated device, such as a touch panel display. The device 10 may also include, for example, an imaging device such as a camera as the input device 106, and the CPU 101 may control the imaging device to acquire images.

[0021] Next, an example of the difference detection method of this embodiment will be described with reference to the flowchart of Fig. 3. The difference detection method of this embodiment is carried out as follows, for example, using the difference detection device 10 of Fig. 1 or Fig. 2. Note that the difference detection method of this embodiment is not limited to use with the difference detection device 10 of Fig. 1 or Fig. 2.

[0022] First, the emotion information acquisition unit 11 acquires emotion information of the subject (S1, emotion information acquisition step). The emotion information includes the emotion of the subject and identification information of the subject. The identification information of the subject is information that can identify the subject, such as the subject's name, an image of the subject, the ID of the subject's device, or the username or ID used by the subject in a remote conference. The emotion information acquisition unit 11 may acquire the emotion information from the emotion estimation engine via a communication network, or may acquire the emotion information from an external database in which emotion information estimated by the emotion estimation engine is recorded. In the latter case, the emotion information acquisition unit 11 may, for example, acquire emotion-related information (described below) and estimate the emotion of the subject based on the emotion-related information to acquire the emotion information of the subject. In this case, the emotion information acquisition unit 11 may also be referred to as an emotion estimation unit, for example.

[0023] The subject's emotion is, for example, information obtained by estimating the subject's emotion based on emotion-related information of the subject. The emotion-related information is, for example, information about the subject's state related to the subject's emotion, such as image information such as the subject's facial expression; vocal information such as voiced and unvoiced sounds; text information such as speech content; and vital data information such as blood pressure and heart rate. The emotion information is, for example, information estimated by a known emotion estimation engine based on the subject's emotion-related information. Specific examples of the emotion estimation engine are not particularly limited, and include, for example, those that estimate emotions based on image information such as facial expressions, such as Affdex, Microsoft Azure Face API, Amazon Rekognition, Realeyes, and User Local facial expression estimation AI; those that estimate emotions based on audio information, such as STEmotion, Empath, and BeyondVerbal's emotion estimation software, IBM Watson Tone Analyzer, and User Local voice emotion recognition AI; those that estimate emotions using natural language processing based on text information, such as User Local text emotion recognition AI; and those that estimate emotions based on vital data, such as NEC emotion analysis solutions. The emotion estimation engine may be, for example, one type, or two or more types may be used in combination. Note that the emotion estimation engine may be, for example, a configuration external to the device 10, or may be a configuration of the device 10.

[0024] The emotion information may include, for example, score information. The score information is information including, for example, a score indicating the degree of emotion for each type of emotion of the subject. The score may be, for example, a score calculated by the emotion estimation engine based on emotion-related information of the subject. The score may be, for example, qualitative information (e.g., whether or not the emotion occurred), quantitative information (e.g., how much the emotion occurred), or information on the probability of occurrence of each emotion. Furthermore, the score may be, for example, an absolute evaluation for each type of emotion, or a relative evaluation with respect to other emotions (whether the emotion is stronger or weaker than other emotions).

[0025] The emotion information may also include, for example, other information. Examples of the other information include priority emotion information, attribute information of the subject, information on the date and time when the emotion was estimated, information such as the location where the emotion was estimated, and information on the situation where the emotion was estimated. The priority emotion information is, for example, information linking a type of emotion with a priority. The priority is, for example, information indicating the weight of each type of emotion and is not particularly limited and can be set arbitrarily. Examples of the attribute information include information such as the gender, age, affiliation, and position in an organization of the subject. The attribute information of the subject may be linked to, for example, the above-mentioned identification information of the subject. Examples of the information on the date, time, and situation when the emotion was estimated include information on the date, time, and situation when emotion-related information serving as a basis for estimating the emotion was acquired. Specifically, when the emotion-related information is the estimation of the emotion of the subject based on video information of a remote conference held by the subject, the date and time information is, for example, the time when the remote conference was held, the location information is, for example, information identifying the remote conference, and the situation information is, for example, information on the participants of the remote conference, information on the agenda of the remote conference, etc. When the emotion information includes the other information, it is preferable that the emotion information and the other information are linked to each other, for example.

[0026] The emotion information acquisition unit 11 may acquire emotion information about one type of emotion of the subject, or emotion information about two or more types of emotions. In the latter case, the emotion information acquisition unit 11 may acquire emotion information about the subject for each type of emotion, for example. The type of emotion is not particularly limited, and may be, for example, an emotion that can be estimated by the emotion estimation engine. Specific examples of the emotion types include joy, anger, sadness, fear, disgust, contempt, and surprise. The emotion types may also be, for example, emotion types represented in Russell's circumplex model. Russell's circumplex model is a model that represents human emotions on a circle, with emotional valence (pleasant-unpleasant) on one axis and arousal (arousal-unarousal) on the other axis perpendicular to the one axis. In Russell's circumplex model of emotions, for example, the direction of the emotional valence and arousal levels from the center (the intersection of the axes) represents the type of emotion, and the distance from the center represents the intensity of the emotion. In this case, specific examples of the types of emotions include arousal, attention, excitement, vitality, happiness, pleasure, satisfaction, calmness, relaxation, tranquility, fatigue, lethargy, depression, sadness, discomfort, worry, stress, nervousness, tension, etc.

[0027] Next, the reference emotion counting unit 12 counts the number of occurrences of a reference emotion based on the emotion information (S2, reference emotion counting step). The number of occurrences of the reference emotion is the number of times the emotion occurs per unit time of the subject during a first predetermined period. The first predetermined period is, for example, a period considered to be the subject's usual state. The usual state is also referred to as, for example, a normal emotion. The unit time is not particularly limited and can be set to any unit, such as an hour or an event. If the unit time is in hours, examples include 10 minutes, 30 minutes, and 1 hour. If the unit time is in events, the unit may be, for example, the number of times an event occurs, or a unit for each event segment (for example, one class period (45 minutes) in an elementary school). The event is not particularly limited and examples include meetings such as web conferences, school classes, seminars, etc. The reference emotion counting unit 12 may, for example, count all emotions occurring per unit time during the first predetermined period among the emotion information of the subject acquired by the emotion information acquisition unit 11, or may count only emotions that satisfy a predetermined condition. The predetermined condition may be, for example, when the score for each emotion included in the emotion information exceeds an arbitrary threshold. In this case, the reference emotion counting unit 12 determines, for example, for each type of emotion, whether the score exceeds a threshold and counts the number of occurrences of emotions having a score exceeding the threshold as the reference emotion occurrence count. The threshold may, for example, be set to an arbitrary value for each type of emotion. By counting emotions having a score exceeding the threshold, for example, the influence of the accuracy of the emotion estimation engine used to estimate the emotion can be suppressed, allowing the reference emotion occurrence count to be more accurately calculated. For example, the reference emotion counting unit 12 may use data obtained by removing outliers from the number of occurrences of the subject's emotions per unit time during the first predetermined period as the reference emotion occurrence count. The first predetermined period is not particularly limited and may be, for example, a period longer than the second predetermined period described below, and may be measured in units of, for example, years, months, weeks, or days. Specific examples of the first predetermined period include 5 years, 3 years, 1 year, 6 months, 3 months, 1 month, 2 weeks, and 3 days.

[0028] When the device 10 includes the storage unit, the storage unit may store, for example, the number of occurrences of the reference emotion and the identification information of the subject in association with each other. This makes it possible to create, for example, a database representing the number of occurrences of the reference emotion for each subject, i.e., the usual emotions of each subject.

[0029] Next, the emotion counting unit 13 counts the number of emotion occurrences based on the emotion information (S3, emotion counting step). The number of emotion occurrences is the number of times an emotion occurs in the subject per unit time during a second predetermined period. The second predetermined period is, for example, a period during which differences in the subject's emotions are examined (e.g., examined, investigated, tested, or confirmed). The unit time is not particularly limited and can be set to any unit, such as an hour or an event. If the unit time is in hours, examples include 10 minutes, 30 minutes, and 1 hour. If the unit time is in events, examples include the number of times an event occurs, or a unit for each event segment (e.g., one class period (45 minutes) in an elementary school). The event is not particularly limited and examples include meetings such as web conferences; school classes; seminars; etc. Specifically, the emotion counting unit 13 may, for example, count all emotions occurring per unit time during the second predetermined period among the emotion information of the subject acquired by the emotion information acquiring unit 11, or may count only emotions that satisfy a predetermined condition. The predetermined condition may be, for example, when the score for each emotion included in the emotion information exceeds an arbitrary threshold. In this case, the emotion counting unit 13 determines, for example, for each type of emotion, whether the score exceeds a threshold, and counts the number of occurrences of emotions having a score exceeding the threshold as the number of occurrences of the emotion. The threshold may, for example, be set to an arbitrary value for each type of emotion. By counting emotions having a score exceeding the threshold, for example, the influence of the accuracy of the emotion estimation engine used to estimate the emotion can be suppressed, thereby enabling the number of occurrences of the emotion to be counted more accurately. The second predetermined period is not particularly limited and may, for example, be a period shorter than the first predetermined period, and the unit may be, for example, years, months, weeks, or days. Specific examples of the second predetermined period include 5 years, 3 years, 1 year, 6 months, 3 months, 1 month, 2 weeks, 3 days, 1 day, and half a day.

[0030] Next, the outlier detection unit 14 detects whether the emotion occurrence count includes an outlier compared to the reference emotion occurrence count (S4, outlier detection step). The outlier detection unit 14 can detect outliers using, for example, a known outlier detection method based on a statistical technique in which the reference emotion occurrence count is used as training data and the emotion occurrence count is used as validation data. The outlier detection method may be, for example, an outlier detection method based on a statistical model, or an outlier detection method based on the distance between data. Specific examples of outlier detection methods include normal distribution, mixed normal distribution, Hotelling's theory, k-nearest neighbor method, local outlier factor method (LOF method), principal component analysis (PCA), time series data, and one-class support vector machine (OCSVM), with detection using the LOF method or OCSVM being preferred.

[0031] Next, the determination unit 15 determines whether or not the number of times an emotion is generated includes an abnormal value (S5 (S5A, S5B, S5C), determination step). The determination unit 15 determines whether or not the number of times an emotion is generated includes an abnormal value (S5A). If the number of times an emotion is generated includes an abnormal value (S5A, YES), the determination unit 15 determines that the subject's current emotional expression is different from usual (S5B). For example, the determination unit 15 may determine that the subject's current emotional expression is different from usual if the number of abnormal values ​​included in the number of times an emotion is generated exceeds a threshold, or may determine that the subject's current emotional expression is different from usual if the proportion of abnormal values ​​included in the number of times an emotion is generated exceeds a threshold, or may determine that the subject's current emotional expression is different from usual if the score of at least one abnormal value among the abnormal values ​​included in the number of times an emotion is generated exceeds a threshold.

[0032] Then, when the determination unit 15 determines that the subject's current emotional expression is different from usual (S5B), the output unit 16 outputs that the subject's emotional expression is different from usual (S6, output step), and the process ends (END). The output unit 16 may also output information such as the type of emotion for which an abnormal value was detected and the score of the emotion.

[0033] If the number of emotion occurrences does not include an abnormal value (S5A, No), the determination unit 15 determines, for example, that the subject's emotional expression is not different from usual, that is, that there is nothing abnormal about the subject's current emotional expression (S5C). In this case, the output unit 16 may output, for example, that there is nothing abnormal about the subject's emotional expression (S6, output step), and end the process (END).

[0034] In this embodiment, S2 is performed before S3, but the difference detection method of the present invention is not limited to this. S2 and S3 may be performed in processes upstream of S4. For example, S2 may be performed after S3, or S2 and S3 may be performed simultaneously.

[0035] The difference detection device 10 of this embodiment can compare the subject's usual emotional expression with the subject's current emotional expression by, for example, comparing the number of times an emotion has occurred with the reference emotional expression number. Therefore, the difference detection device 10 of this embodiment can determine whether the subject's emotional expression is different from usual.

[0036] [Embodiment 2] This embodiment is similar to the detection device 10 of embodiment 1, except that an estimation unit is provided in addition to the configuration of the detection device 10 of embodiment 1, and the description thereof can be used. The detection device of this embodiment includes an estimation unit, and when the determination unit determines that the subject's current emotional expression is different from usual, the estimation unit estimates the subject's emotional behavior, and the output unit outputs the subject's emotional behavior.

[0037] Fig. 4 is a block diagram showing an example of the configuration of a detection device 10A of this embodiment. As shown in Fig. 4, the detection device 10A includes an estimation unit 20 in addition to the configuration of the detection device 10 of embodiment 1. The hardware configuration of the detection device 10A is the same as that of the detection device 10 of Fig. 2, except that the CPU 101 includes the configuration of the detection device 10A of Fig. 4 instead of the configuration of the detection device 10 of Fig. 1.

[0038] Next, the detection method of this embodiment will be described with reference to the flowchart of Fig. 5. The detection method of this embodiment can be implemented using, for example, the detection device 10A of this embodiment shown in Fig. 4. Note that the detection method of the present invention is not limited to use of the detection device 10A.

[0039] First, steps S1 to S5 are carried out in the same manner as steps S1 to S5 in the difference detection method of the first embodiment, and it is determined whether the subject's current emotional state is different from usual.

[0040] If it is determined that the subject's current emotional expression is not different from usual (S5, No), the output unit 16 outputs that there is nothing abnormal in the subject's emotional expression (S6, output step), and the process ends (END). On the other hand, if it is determined that the subject's current emotional expression is different from usual (S5, YES), the estimation unit 20 estimates the subject's emotional behavior (S7, estimation step), and the output unit 15 outputs that the subject's current emotional expression is different from usual and the grasped emotional behavior (S6, output step), and the process ends (END). A specific example of the process by the estimation unit 20 in S7 will be described later in embodiment 3.

[0041] According to the detection device of the present invention, when it is determined that the subject's current emotional state is different from usual, the estimation unit can estimate the subject's emotional state.

[0042] [Embodiment 3] Specifically, the estimation unit 20 of the detection device 10A of embodiment 2 will be described using the drawings. FIG. 6 is a block diagram showing an example of the configuration of the estimation unit 20. As shown in FIG. 6, the estimation unit 20 includes, for example, an emotion information acquisition unit 211, a reference emotion vector calculation unit 232, a comparison emotion vector calculation unit 213, a differential emotion vector calculation unit 214, a movement estimation unit 215, an output unit 216, and a storage unit 217. The estimation unit 20 is also referred to as, for example, an emotional movement estimation device 20, and the processing performed by the estimation unit 20 is also referred to as, for example, an emotional movement estimation method. The estimation unit 20 may be, for example, a device included in the detection device 10A, or may be a device separate from the detection device 10A. In the latter case, the estimation unit 20 (emotional movement estimation device 20) and the detection device 10A are capable of communicating with each other. The communication may be, for example, wired or wireless.

[0043] The estimation unit 20 (estimation device 20) may be, for example, a single device including the above-mentioned units, or a device to which the above-mentioned units can be connected via a communication network. The estimation device 20 can also be connected to an external device (described later) via the communication network. The communication network is not particularly limited and any known network can be used, for example, a wired or wireless network. Examples of the communication network include the Internet, the World Wide Web (WWW), a telephone line, a Local Area Network (LAN), a Storage Area Network (SAN), a Delay Tolerant Networking (DTN), a Low Power Wide Area Network (LPWA), and a Local 5G (L5G). Examples of wireless communication include Wi-Fi (registered trademark), Bluetooth (registered trademark), Local 5G, and LPWA. Examples of the wireless communication include direct communication between devices (Ad Hoc communication), infrastructure communication, and indirect communication via an access point. The estimation device 20 may be incorporated into a server as a system. The estimation device 20 may be, for example, a personal computer (PC, for example, a desktop or notebook type) on which the program of the present invention is installed, a smartphone, a tablet terminal, etc. The estimation device 20 may be in the form of cloud computing or edge computing, for example, in which at least one of the above-mentioned units is located on a server and the other units are located on a terminal.

[0044] 7 illustrates a block diagram of the hardware configuration of the estimation device 20. The estimation device 20 includes, for example, a CPU 201, a memory 202, a bus 203, a storage device 204, an input device 206, a display device 207, and a communication device 208. The components of the estimation device 20 are connected to each other via the bus 203 and their respective interfaces (I / F).

[0045] CPU 201 operates in conjunction with other components via a controller (system controller, I / O controller, etc.) and the like, and is responsible for overall control of estimation device 20. In estimation device 20, CPU 201 executes, for example, program 205 of the present invention and other programs, and also reads and writes various types of information. Specifically, CPU 201 functions as emotion information acquisition section 211, reference emotion vector calculation section 212, comparison emotion vector calculation section 213, differential emotion vector calculation section 214, motion estimator 215, and output section 216. This device 2 is equipped with a CPU as a computing device, but may also be equipped with other computing devices such as a GPU (Graphics Processing Unit) or APU (Accelerated Processing Unit), or may be equipped with a combination of a CPU and these.

[0046] The bus 203 can also be connected to, for example, external devices. Examples of the external devices include an external storage device (such as an external database), a printer, an external input device, an external display device, and an external imaging device. The estimation device 20 can be connected to an external network (the communication line network) by, for example, a communication device 208 connected to the bus 203, and can also be connected to other devices via the external network.

[0047] The memory 202 may be, for example, a main memory (primary storage device). When the CPU 201 performs processing, the memory 202 reads various operating programs, such as the program of the present invention, stored in the storage device 204 (described later), and the CPU 201 receives data from the memory 202 and executes the programs. The main memory may be, for example, a RAM (random access memory). The memory 202 may also be, for example, a ROM (read only memory).

[0048] The storage device 204 is also referred to as an auxiliary storage device, for example, in contrast to the main memory (primary storage device). As described above, the storage device 204 stores an operating program including the program 205 of the present invention. The storage device 204 may be, for example, a combination of a recording medium and a drive for reading and writing data from and to the recording medium. The recording medium is not particularly limited and may be, for example, an internal or external type, such as a hard disk (HD), CD-ROM, CD-R, CD-RW, MO, DVD, flash memory, or memory card. The storage device 104 may be, for example, a hard disk drive (HDD) or a solid state drive (SSD) in which the recording medium and drive are integrated. The storage device 204 functions as a memory unit 217 and includes vector transformation information 2171.

[0049] The vector conversion information 2171 is information that serves as a reference for representing emotion information on two-dimensional coordinates. For example, on the two-dimensional coordinates, the origin is set to a state where there are no emotional fluctuations, a normal state, and the type of emotion is linked to the angle from the origin, and the intensity (score) of the emotion is linked to the distance from the origin. Using the vector conversion information 2171, the estimation device 20 can convert the type of emotion of the subject into an angle from the origin, and the intensity (score) of the emotion into a distance from the origin. There are no particular restrictions on the link between the type of emotion and the angle from the origin on the two-dimensional coordinates, and any link can be used; however, it is preferable to use, for example, the Russell emotional circumplex model mentioned above.

[0050] According to Russell's circumplex model, emotions can be classified into attention, excitement, energy, happiness, satisfaction, cheerfulness, relaxation, calm, fatigue, lethargy, depression, sadness, worry, pressure, nervousness, and tension, for example, based on valence and arousal. Therefore, for example, by setting the intersection of the valence axis and the arousal axis in Russell's circumplex model as the origin, and linking the type of emotion estimated by an emotion estimation engine (described later) and the type of emotion on Russell's circumplex model, as well as the score of the emotion estimated by the emotion estimation engine and the distance from the origin, and storing these as vector conversion information 2171, it becomes possible to easily represent the emotion of a target person on two-dimensional coordinates.

[0051] In the estimation device 20, the memory 202 and the storage device 204 can also store various information such as log information, information acquired from an external database (not shown) or an external device, information generated by the estimation device 20, and information used when the estimation device 20 executes processing. In this case, the memory 202 and the storage device 204 may store at least one piece of information selected from the group consisting of, for example, emotion information of the subject, identification information of the subject, attribute information of the subject, score threshold, etc., which will be described later. Note that at least a portion of the information may be stored in an external server other than the memory 202 and the storage device 204, or may be stored in a distributed manner across multiple terminals using blockchain technology or the like.

[0052] The estimation device 20 further includes, for example, an input device 206 and a display 207. Examples of the input device 206 include pointing devices such as a touch panel, track pad, and mouse; a keyboard; imaging means such as a camera and scanner; card readers such as an IC card reader and a magnetic card reader; and audio input means such as a microphone. Examples of the display 207 include display devices such as an LED display and a liquid crystal display. In this embodiment, the input device 206 and the display 207 are configured separately, but the input device 206 and the display 207 may be configured as an integrated device, such as a touch panel display. Furthermore, the estimation device 20 may include, for example, an imaging device such as a camera as the input device 206, and the CPU 201 may control the imaging device to acquire an image.

[0053] Next, an example of the emotion movement estimation method of this embodiment will be described with reference to the flowchart of Fig. 8. The movement estimation method of this embodiment is implemented as follows, for example, using the movement estimation device 20 of Fig. 6 or Fig. 7. Note that the movement estimation method of this embodiment is not limited to use of the movement estimation device 20 of Fig. 6 or Fig. 7.

[0054] First, the emotion information acquisition unit 211 acquires emotion information of the subject (S21, emotion information acquisition step). The emotion information is, for example, as described above. The emotion information acquisition unit 211 is, for example, similar to the emotion information acquisition unit 11 in the difference detection device 10 of embodiment 1, and the description therein can be used. Alternatively, for example, the emotion information acquired in step S1 of embodiment 1 may be used.

[0055] Next, reference emotion vector calculation section 212 calculates a reference emotion vector for a first predetermined period based on the vector conversion information and the emotion information (S22, reference emotion vector calculation step). The first predetermined period is, for example, a period considered to be the subject's usual state. The usual state is also referred to as, for example, a normal emotion. For example, reference emotion vector calculation section 12 converts the emotion type and emotion score for the first predetermined period, from the acquired emotion information, into an angle and distance from the origin based on the vector conversion information, to calculate a provisional reference emotion vector for the first predetermined period. Then, it combines the provisional reference emotion vectors for the first predetermined period to calculate a reference emotion vector. Alternatively, reference emotion vector calculation section 12 may, for example, combine the provisional reference emotion vectors to calculate a combined provisional reference emotion vector, and then divide the combined provisional reference emotion vector by the number of combined provisional reference emotion vectors to calculate the average of the combined provisional reference emotion vector as the reference emotion vector. The first predetermined period is not particularly limited and may be, for example, a period longer than the second predetermined period described below, and may be measured in units of, for example, years, months, weeks, or days. Specific examples of the first predetermined period include, for example, 5 years, 3 years, 1 year, 6 months, 3 months, 1 month, 2 weeks, and 3 days. The reference emotion vector is also referred to as the subject's usual emotion vector.

[0056] Furthermore, the reference emotion vector calculation unit 212 may calculate the reference emotion vector for all emotions included in the emotion information, or may extract emotions that satisfy a condition and calculate the reference emotion vector for the extracted emotion. The condition is not particularly limited, and may be, for example, an emotion whose score per unit time in the first predetermined period exceeds a threshold, an emotion whose score per unit time in the first predetermined period is the highest, or an emotion whose priority per unit time in the first predetermined period is the highest. The unit time is not particularly limited, and any unit can be set, such as an hour or an event. If the unit time is in hours, examples include 10 minutes, 30 minutes, and 1 hour. If the unit time is in events, examples include the number of times an event occurs, or a unit for each event segment (e.g., one class period (45 minutes) in an elementary school). The event is not particularly limited, and examples include meetings such as web conferences, school classes, seminars, etc. The emotion extracted based on the condition is also referred to as a representative emotion.

[0057] For example, the storage unit 217 may store the reference emotion vector in association with the identification information of the subject. This makes it possible to create a database representing the reference emotion vector for each subject, i.e., the subject's usual emotion.

[0058] Next, the comparative emotion vector calculation unit 213 calculates a comparative emotion vector for a second predetermined period based on the vector conversion information and the emotion information (S23, comparative emotion vector calculation step). The second predetermined period is, for example, a period during which differences in the emotions of the subject are examined (also known as examination, investigation, testing, or confirmation). The comparative emotion vector calculation unit 213, for example, converts the emotion type and emotion score for the second predetermined period from the acquired emotion information into an angle and distance from the origin based on the vector conversion information, and calculates an emotion vector for the second predetermined period. Then, it combines each emotion vector for the second predetermined period to calculate a comparative emotion vector. Alternatively, the comparative emotion vector calculation unit 213 may, for example, combine the provisional comparison emotion vectors to calculate a combined provisional comparison emotion vector, and then divide the combined provisional comparison emotion vector by the number of combined provisional comparison emotion vectors to calculate the average of the combined provisional comparison emotion vectors as the comparative emotion vector. The second predetermined period is not particularly limited and may be, for example, a period shorter than the first predetermined period, and may be measured in years, months, weeks, or days. Specific examples of the second predetermined period include 5 years, 3 years, 1 year, 6 months, 3 months, 1 month, 2 weeks, 3 days, 1 day, and half a day. The comparative emotion vector may also be referred to as the subject's current emotion vector.

[0059] For example, the comparative emotion vector calculation unit 213 may calculate the comparative emotion vector for all emotions included in the emotion information, or may extract emotions that satisfy a condition and calculate the comparative emotion vector for the extracted emotion. The condition is not particularly limited, and may be, for example, an emotion whose score per unit time during the second predetermined period exceeds a threshold, an emotion whose score per unit time during the second predetermined period is the highest, or an emotion whose priority per unit time during the second predetermined period is the highest. The unit time is not particularly limited, and any unit can be set, such as an hour or an event. If the unit time is an hour, examples include 10 minutes, 30 minutes, and 1 hour. If the unit time is an event, examples include the number of times an event occurs, or a unit for each event segment (e.g., one class period (45 minutes) in an elementary school). The event is not particularly limited, and examples include meetings such as web conferences, school classes, seminars, etc. The emotion extracted based on the condition is also referred to as a representative emotion.

[0060] Next, the differential emotion vector calculation unit 214 calculates a differential emotion vector based on the reference emotion vector and the comparison emotion vector (S24, differential emotion vector calculation step). The calculation of the differential emotion vector by the differential emotion vector calculation unit 214 can use, for example, a normal vector calculation method. The differential emotion vector represents, for example, the difference between the subject's usual emotion and their current emotion, and is therefore also called an emotion vector that represents a difference from usual.

[0061] Next, the motion estimation unit 215 estimates the emotional changes of the subject between a first predetermined period and a second predetermined period based on the vector conversion information and the differential emotion vector (S5, emotion estimation step). Specifically, the motion estimation unit 215 can, for example, compare the direction of the differential emotion vector with the direction of the corresponding emotion on the two-dimensional coordinates of the vector conversion information to estimate how the type of emotion of the subject has changed. Furthermore, because the length of the differential emotion vector corresponds, for example, to the degree (amount) of emotional change, the motion estimation unit 215 can estimate the amount of emotional change of the subject based on the length of the differential emotion vector. However, the processing by the motion estimation unit 215 is not limited to this. For example, the motion estimation unit 215 may estimate the emotional changes of the subject by creating an axis that connects two emotions located at opposite poles across the origin on the two-dimensional coordinates and passes through the origin, and comparing the axis with the differential emotion vector. In this case, the movement estimation unit 215 can estimate the movement of the subject's emotions from the direction of the differential emotion vector compared with the axis on the two-dimensional coordinate system, and the length of the differential emotion vector on the two-dimensional coordinate system.

[0062] A specific example of emotional movement estimation by the motion estimator 215 will be described using Figure 9. Figure 9 is a diagram showing a specific example of vector conversion information 2171, illustrating an example of a reference emotion vector 121, a comparison emotion vector 131, and a difference emotion vector 141 on two-dimensional coordinates 1711 included in the vector conversion information 2171. Note that the vector conversion information 2171, reference emotion vector 121, comparison emotion vector 131, and difference emotion vector 141 are not limited to the specific example shown in Figure 9. For example, as shown in Figure 9, the motion estimator 215 first connects the emotions of cheerfulness and depression, which are polar opposites, on the two-dimensional coordinates 1711, to create an axis 1712 passing through the origin. Axis 1712 represents, for example, whether the subject's emotion is cheerful or depressed, and can therefore also be considered a vitality axis indicating the subject's level of vitality. If the differential emotion vector 141 has the direction and length shown in the specific example of Fig. 9, the motion estimation unit 215 can, for example, from the direction of the differential emotion vector 141 relative to the vitality axis 1712, estimate that the subject's emotion has changed in a depressed direction, i.e., that the subject has become less energetic, and from the length of the vector, estimate the extent of the change. For convenience of explanation, the axis 1712 has been described as an vitality axis connecting vitality and depression, but the processing by the motion estimation unit 215 is not limited to this. For example, the motion estimation unit 215 can create axes connecting any emotions located at opposite poles across the origin, such as excitement and lethargy, happiness and sadness, and compare each axis with the differential emotion vector to estimate the subject's emotional changes. Furthermore, the motion estimation unit 215 may, for example, estimate the subject's emotional changes based on one type of axis 1712, or may estimate the subject's emotional changes based on two or more types of axes 1712.

[0063] The output unit 216 then outputs the estimated emotional movement of the subject (S26, output step), and the process ends (END). The output unit 216 may, for example, output the reference emotion vector, comparison emotion vector, and differential emotion vector together.

[0064] In this embodiment, S22 is performed before S23, but the emotional movement estimation method of the present invention is not limited to this. S22 and S23 may be performed upstream of S24. For example, S22 may be performed after S23, or S22 and S23 may be performed simultaneously.

[0065] The movement estimation device 20 of this embodiment can estimate the emotional movement of a subject based on, for example, a differential emotion vector between a reference emotion vector and a comparison emotion vector. Therefore, the movement estimation device 20 of this embodiment can estimate how the emotional movement has changed, taking into account the individual's usual emotional state. This also makes it possible to estimate changes in the subject's state, particularly their mental state, such as how the subject's energy level has changed. The emotional movement estimation device of this embodiment can also be referred to as, for example, a subject's state estimation device.

[0066] According to the emotion difference detection device including the movement estimation device of this embodiment, when it is determined that the subject's current emotional expression is different from usual, it is possible to estimate the subject's emotional movement. Therefore, the emotion difference detection device of this embodiment can perform a detailed analysis of the subject's state.

[0067] [Embodiment 4] The program of this embodiment is a program for causing a computer to execute each step of the difference detection method described above. Specifically, the program of this embodiment is a program for causing a computer to execute an emotion information acquisition procedure, a reference emotion counting procedure, an emotion counting procedure, an outlier detection procedure, a judgment procedure, and an output procedure.

[0068] the emotion information acquisition step acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection step detects whether the number of occurrences of emotions includes an abnormal value relative to the reference number of occurrences of emotions; The determination step determines that the subject's current emotional expression is different from usual when the number of times the emotion occurs includes an abnormal value; The output step outputs the determination result.

[0069] The program of this embodiment can also be said to be a program that causes a computer to function as an emotion information acquisition procedure, a reference emotion counting procedure, an emotion counting procedure, an abnormal value detection procedure, a judgment procedure, and an output procedure.

[0070] The program of this embodiment can cite the descriptions of the difference detection device and difference detection method of the present invention. For each of the steps, for example, "step" can be read as "processing." The program of this embodiment may also be recorded on a computer-readable recording medium. The recording medium is, for example, a non-transitory computer-readable storage medium. The recording medium is not particularly limited, and examples include random access memory (RAM), read-only memory (ROM), hard disk (HD), optical disk, and floppy disk (FD).

[0071] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0072] <Additional Notes> Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Appendix 1) The system includes an emotion information acquisition unit, a reference emotion counting unit, an emotion counting unit, an abnormal value detection unit, a determination unit, and an output unit, the emotion information acquisition unit acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting unit counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting unit counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection unit detects whether the number of emotion occurrences includes an abnormal value with respect to the reference number of emotion occurrences; When the number of times of emotion occurrence includes an abnormal value, the determination unit determines that the subject's current emotion is being expressed differently than usual; The output unit outputs the determination result. (Appendix 2) a storage unit, the storage unit stores the identification information of the subject and the reference emotion occurrence count in association with each other; 2. The difference detection device according to claim 1, wherein the determination unit determines whether there is a difference between the stored reference emotion occurrence count and the emotion occurrence count. (Appendix 3) the emotion information acquisition unit acquires emotion information of the subject for each emotion type; the reference emotion counting unit counts the number of occurrences of the reference emotion for each type of emotion; 3. The difference detection device according to claim 1, wherein the emotion counting unit counts the number of occurrences of the emotion for each type of emotion. (Appendix 4) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, the reference emotion counting unit determines whether or not the score exceeds a threshold for each emotion type, and counts the number of occurrences of emotions having scores exceeding the threshold as the number of occurrences of the reference emotion; 4. The difference detection device according to claim 1, wherein the emotion counting unit determines whether the score exceeds a threshold for each type of emotion, and counts the number of occurrences of emotions having a score exceeding the threshold as the number of occurrences of the emotion. (Appendix 5) 5. The difference detection device according to claim 1, wherein the determination unit determines that the subject's current emotional expression is different from usual when the number of abnormal values ​​included in the emotion occurrence count exceeds a threshold value. (Appendix 6) 5. The difference detection device according to claim 1, wherein the determination unit determines that the subject's current emotional expression is different from usual when the proportion of abnormal values ​​included in the number of emotion occurrences exceeds a threshold value. (Appendix 7) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, 5. The difference detection device according to claim 1, wherein the determination unit determines that the subject's current emotional expression is different from usual when the score of at least one of the abnormal values ​​included in the number of emotion occurrences exceeds a threshold. (Appendix 8) an estimation unit; the estimation unit estimates the subject's emotional behavior when the determination unit determines that the subject's current emotional expression is different from usual; The difference detection device according to any one of appendices 1 to 7, wherein the output unit outputs the emotional movement of the subject. (Appendix 9) the estimation unit includes a storage unit, an emotion information acquisition unit, a reference emotion vector calculation unit, a comparison emotion vector calculation unit, a difference emotion vector calculation unit, an estimation unit, and an output unit; the storage unit includes vector transformation information; the emotion information acquisition unit acquires emotion information of a subject; the reference emotion vector calculation unit calculates a reference emotion vector for a first predetermined period based on the vector conversion information and the emotion information; the comparative emotion vector calculation unit calculates a comparative emotion vector for a second predetermined period based on the vector conversion information and the emotion information; the differential emotion vector calculation unit calculates a differential emotion vector based on the reference emotion vector and the comparison emotion vector; the estimation unit estimates a change in the subject's emotions between a first predetermined period and a second predetermined period based on the vector conversion information and the differential emotion vector; 9. The difference detection device according to claim 8, wherein the output unit outputs the emotional movement of the subject. (Appendix 10) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, The reference emotion vector calculation unit extracting emotions for which the scores exceed a threshold for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation unit extracting emotions for which the scores exceed a threshold for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 10. The detection device of claim 9, wherein the comparison emotion vector is calculated based on the provisional comparison emotion vector. (Appendix 11) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, The reference emotion vector calculation unit extracting the emotion with the highest score for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation unit extracting the emotion with the highest score for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 11. The detection device according to claim 9, wherein the comparative emotion vector is calculated based on the provisional comparative emotion vector. (Appendix 12) the emotion information includes priority emotion information, The priority emotion information is information in which the type of emotion is associated with a priority level, The reference emotion vector calculation unit extracting the emotion with the highest priority for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation unit extracting the emotion with the highest priority for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 12. The detection device according to any one of appendices 9 to 11, wherein the comparison emotion vector is calculated based on the provisional comparison emotion vector. (Appendix 13) the emotion information includes identification information of the subject; 13. The detection device according to any one of appendices 9 to 12, wherein the storage unit stores the identification information of the subject and the reference emotion vector in association with each other. (Appendix 14) The method includes an emotion information acquisition step, a reference emotion counting step, an emotion counting step, an outlier detection step, a determination step, and an output step, the emotion information acquiring step acquires emotion information of a subject, the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection step detects whether the number of occurrences of an emotion includes an abnormal value relative to the reference number of occurrences of an emotion, The determining step determines that the subject's current emotional expression is different from usual when the number of times the emotion is generated includes an abnormal value, The output step outputs the determination result. (Appendix 15) a storing step, the storing step stores the identification information of the subject and the number of occurrences of the reference emotion in association with each other; 15. The difference detection method according to claim 14, wherein the determining step determines whether or not there is a difference between the stored reference emotion occurrence count and the emotion occurrence count. (Appendix 16) the emotion information acquiring step acquires emotion information of the subject for each emotion type; the reference emotion counting step counts the number of occurrences of the reference emotion for each type of emotion; 16. The difference detection method according to claim 14, wherein the emotion counting step counts the number of occurrences of the emotions for each type of emotion. (Appendix 17) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, the reference emotion counting step determines whether or not the score exceeds a threshold for each type of emotion, and counts the number of occurrences of emotions having scores exceeding the threshold as the number of occurrences of the reference emotion; 17. The difference detection method according to claim 16, wherein the emotion counting step determines whether the score exceeds a threshold for each type of emotion, and counts the number of occurrences of emotions having a score exceeding the threshold as the number of occurrences of the emotion. (Appendix 18) 18. The difference detection method according to any one of appendices 14 to 17, wherein the determination step determines that the subject's current emotional expression is different from usual if the number of abnormal values ​​included in the emotion occurrence count exceeds a threshold. (Appendix 19) 18. The difference detection method according to any one of appendices 14 to 17, wherein the determination step determines that the subject's current emotional expression is different from usual when the proportion of abnormal values ​​included in the number of emotion occurrences exceeds a threshold. (Appendix 20) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, 18. The difference detection method according to any one of Appendices 14 to 17, wherein the determination step determines that the subject's current emotional expression is different from usual when the score of at least one abnormal value among the abnormal values ​​included in the emotion occurrence count exceeds a threshold. (Appendix 21) An estimation step is included, the estimation step estimates the subject's emotional behavior when it is determined in the determination step that the subject's current emotional expression is different from usual; A difference detection method described in any one of Appendices 14 to 20, wherein the output step outputs the emotional movements of the subject. (Appendix 22) the estimation step includes an emotion information acquisition step, a reference emotion vector calculation step, a comparative emotion vector calculation step, a differential emotion vector calculation step, an estimation step, and an output step; the emotion information acquiring step acquires emotion information of a subject, the reference emotion vector calculation step calculates a reference emotion vector for a first predetermined period based on the vector conversion information and the emotion information; the comparative emotion vector calculation step calculates a comparative emotion vector for a second predetermined period based on the vector conversion information and the emotion information; the differential emotion vector calculation step calculates a differential emotion vector based on the reference emotion vector and the comparison emotion vector; the estimation step estimates a change in the subject's emotions between a first predetermined period and a second predetermined period based on the vector transformation information and the differential emotion vector; 22. The difference detection method according to claim 21, wherein the output step outputs the emotional movement of the subject. (Appendix 23) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, The reference emotion vector calculation step includes: extracting emotions for which the scores exceed a threshold for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation step includes: extracting emotions for which the scores exceed a threshold for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 23. The detection method of claim 22, wherein the comparison emotion vector is calculated based on the provisional comparison emotion vector. (Appendix 24) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, The reference emotion vector calculation step includes: extracting the emotion with the highest score for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation step includes: extracting the emotion with the highest score for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 24. The detection method according to claim 22, wherein the comparative emotion vector is calculated based on the provisional comparative emotion vector. (Appendix 25) the emotion information includes priority emotion information, The priority emotion information is information in which the type of emotion is associated with a priority level, The reference emotion vector calculation step includes: extracting the emotion with the highest priority for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation step includes: extracting the emotion with the highest priority for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 25. The detection method according to any one of appendices 22 to 24, wherein the comparison emotion vector is calculated based on the provisional comparison emotion vector. (Appendix 26) the emotion information includes identification information of the subject; 26. The detection method according to any one of appendices 22 to 25, wherein the storage step stores the identification information of the subject and the reference emotion vector in association with each other. (Appendix 27) A program for causing a computer to execute an emotion information acquisition procedure, a reference emotion counting procedure, an emotion counting procedure, an abnormal value detection procedure, a judgment procedure, and an output procedure, the emotion information acquisition step acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time during a first predetermined period; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the number of occurrences of an emotion is the number of occurrences of the emotion of the subject per unit time in a second predetermined period, the abnormal value detection step detects whether the number of occurrences of emotions includes an abnormal value relative to the reference number of occurrences of emotions; The determination step determines that the subject's current emotional expression is different from usual when the number of times the emotion occurs includes an abnormal value; The output step outputs the determination result. (Appendix 28) Includes memory procedures, the storing step includes storing the identification information of the subject and the number of occurrences of the reference emotion in association with each other; 18. The program according to claim 17, wherein the determining step determines whether or not there is a difference between the stored reference emotion occurrence count and the emotion occurrence count. (Appendix 29) the emotion information acquisition step acquires emotion information of the subject for each emotion type; the reference emotion counting step counts the number of occurrences of the reference emotions for each type of emotion; 29. The program according to claim 27, wherein the emotion counting step counts the number of occurrences of the emotion for each type of emotion. (Appendix 30) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, the reference emotion counting step includes determining whether or not the score exceeds a threshold for each type of emotion, and counting the number of occurrences of emotions having scores exceeding the threshold as the reference emotion occurrence count; 30. The program of claim 29, wherein the emotion counting step determines whether the score exceeds a threshold for each type of emotion, and counts the number of occurrences of emotions having a score exceeding the threshold as the number of occurrences of the emotion. (Appendix 31) 32. The program described in any one of Appendices 27 to 31, wherein the determination step determines that the subject's current emotional expression is different from usual if the number of abnormal values ​​included in the emotion occurrence count exceeds a threshold. (Appendix 32) 32. The program described in any one of Appendices 27 to 31, wherein the determination step determines that the subject's current emotional expression is different from usual if the proportion of abnormal values ​​included in the emotion occurrence count exceeds a threshold. (Appendix 33) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, 32. The program described in any one of Appendices 27 to 31, wherein the determination procedure determines that the subject's current emotional expression is different from usual if the score of at least one abnormal value among the abnormal values ​​included in the emotion occurrence count exceeds a threshold. (Appendix 34) causing the computer to perform an estimation procedure; the estimation step estimates the subject's emotional behavior when it is determined in the determination step that the subject's current emotional expression is different from usual; The program described in any one of Appendices 27 to 33, wherein the output step outputs the emotional movements of the subject. (Appendix 35) the estimation procedure includes an emotion information acquisition procedure, a reference emotion vector calculation procedure, a comparison emotion vector calculation procedure, a difference emotion vector calculation procedure, an estimation procedure, and an output procedure; the emotion information acquisition step acquires emotion information of a subject; the reference emotion vector calculation step calculates a reference emotion vector for a first predetermined period based on vector conversion information and the emotion information; the comparative emotion vector calculation step calculates a comparative emotion vector for a second predetermined period based on the vector conversion information and the emotion information; the step of calculating a differential emotion vector includes calculating a differential emotion vector based on the reference emotion vector and the comparison emotion vector; the estimation step estimates a change in the subject's emotions between a first predetermined period and a second predetermined period based on the vector transformation information and the differential emotion vector; 22. The program of claim 21, wherein the output step outputs the emotional movements of the subject. (Appendix 36) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, The reference emotion vector calculation procedure includes: extracting emotions for which the scores exceed a threshold for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation procedure includes: extracting emotions for which the scores exceed a threshold for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 36. The program according to claim 35, wherein the comparative emotion vector is calculated based on the provisional comparative emotion vector. (Appendix 37) the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, The reference emotion vector calculation procedure includes: extracting the emotion with the highest score for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation procedure includes: extracting the emotion with the highest score for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 39. The program according to claim 35 or 38, wherein the comparative emotion vector is calculated based on the provisional comparative emotion vector. (Appendix 38) the emotion information includes priority emotion information, The priority emotion information is information in which the type of emotion is associated with a priority level, The reference emotion vector calculation procedure includes: extracting the emotion with the highest priority for each unit time during the first predetermined period; Calculating a provisional reference emotion vector for the extracted emotion based on the vector conversion information; calculating the reference emotion vector based on the provisional reference emotion vector; The comparative emotion vector calculation procedure includes: extracting the emotion with the highest priority for each unit time during the second predetermined period; Calculating a provisional comparison emotion vector for the extracted emotion based on the vector conversion information; 38. The program according to any one of appendices 35 to 37, wherein the comparative emotion vector is calculated based on the provisional comparative emotion vector. (Appendix 39) the emotion information includes identification information of the subject; 39. The program according to any one of appendices 35 to 38, wherein the storage step stores the target person's identification information and the reference emotion vector in association with each other. (Appendix 40) A computer-readable recording medium having recorded thereon a program according to any one of appendices 27 to 39. [Industrial Applicability]

[0073] According to the present invention, it is possible to determine whether a subject's emotional state is different from usual, and therefore the present invention is particularly useful in fields such as human resource management and health management. [Explanation of symbols]

[0074] 10, 10A difference detection device 11 Emotion information acquisition unit 12 Reference Emotion Counting Unit 13 Emotion Counting Department 14. Anomaly detection unit 15 Judgment section 16 Output section 101 CPU 102 memory 103 Bus 104 Storage device 105 Programs 106 Input Device 107 Display device 108 Communication Devices 20 Estimation unit (motion estimation device) 211 Emotion information acquisition unit 212 Reference emotion vector calculation unit 213 Emotion Vector Calculation Unit 214 Differential Emotion Vector Calculation Unit 215 Motion Estimation Unit 216 Output section 217 Memory section 201 CPU 202 memory 203 Bus 204 Storage device 205 Programs 206 Input Device 207 Display device 208 Communication Devices

Claims

1. The system includes an emotion information acquisition unit, a reference emotion counting unit, a memory unit, an emotion counting unit, an abnormal value detection unit, a determination unit, and an output unit, the emotion information acquisition unit acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting unit counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time in a first predetermined period, the storage unit stores identification information of the subject and the number of occurrences of the reference emotion for each subject in association with each other; the emotion counting unit counts the number of occurrences of emotions based on the emotion information; the emotion occurrence count is the number of times the emotion occurs per unit time of the subject during a second predetermined period; the abnormal value detection unit detects whether the number of times an emotion is generated by the subject includes an abnormal value with respect to the reference number of times an emotion is generated that is stored in association with identification information of the subject; When the number of times of emotion occurrence includes an abnormal value, the determination unit determines that the subject's current emotion is manifested differently from usual; The output unit outputs the determination result.

2. the emotion information acquisition unit acquires emotion information of the subject for each emotion type; the reference emotion counting unit counts the number of occurrences of the reference emotion for each type of emotion; 2. The difference detection device according to claim 1, wherein the emotion counting unit counts the number of occurrences of each emotion for each type of emotion.

3. the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, the reference emotion counting unit determines whether or not the score exceeds a threshold for each type of emotion, and counts the number of occurrences of emotions having scores exceeding the threshold as the number of occurrences of the reference emotion; 3. The difference detection device according to claim 1, wherein the emotion counting unit determines whether the score exceeds a threshold for each type of emotion, and counts the number of occurrences of emotions having a score exceeding the threshold as the number of occurrences of the emotion.

4. 4. The difference detection device according to claim 1, wherein the determination unit determines that the subject's current emotional expression is different from usual when the number of abnormal values ​​included in the emotion occurrence count exceeds a threshold value.

5. 4. The difference detection device according to claim 1, wherein the determination unit determines that the subject's current emotional expression is different from usual when a proportion of abnormal values ​​included in the emotion occurrence count exceeds a threshold value.

6. the emotion information includes score information, the score information is information including a score indicating the degree of each emotion type, 4. The difference detection device according to claim 1, wherein the determination unit determines that the subject's current emotional expression is different from usual when a score of at least one of the abnormal values ​​included in the emotion occurrence count exceeds a threshold.

7. an estimation unit; the estimation unit estimates the subject's emotional behavior when the determination unit determines that the subject's current emotional expression is different from usual; The difference detection device according to claim 1 , wherein the output unit outputs an emotional movement of the subject.

8. The method includes an emotion information acquisition step, a reference emotion counting step, a storage step, an emotion counting step, an outlier detection step, a determination step, and an output step, the emotion information acquiring step acquires emotion information of a subject, the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time in a first predetermined period, the storing step stores the identification information of the subject and the number of occurrences of the reference emotion for each subject in association with each other; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the emotion occurrence count is the number of times the emotion occurs per unit time of the subject during a second predetermined period; the abnormal value detection step detects whether the number of times an emotion is generated by the subject includes an abnormal value with respect to the reference number of times an emotion is generated that is stored in association with the identification information of the subject; The determining step determines that the subject's current emotional expression is different from usual when the number of times the emotion is generated includes an abnormal value, The output step outputs the determination result.

9. A program for causing a computer to execute an emotion information acquisition procedure, a reference emotion counting procedure, a storage procedure, an emotion counting procedure, an abnormal value detection procedure, a judgment procedure, and an output procedure, the emotion information acquisition step acquires emotion information of a subject; the emotion information includes an emotion of the subject and identification information of the subject; the reference emotion counting step counts the number of occurrences of a reference emotion based on the emotion information; the reference emotion occurrence count is the number of times the emotion of the subject occurs per unit time in a first predetermined period, the storing step includes storing the identification information of the subject and the number of occurrences of the reference emotion for each subject in association with each other; the emotion counting step counts the number of occurrences of emotions based on the emotion information; the emotion occurrence count is the number of times the emotion occurs per unit time of the subject during a second predetermined period; the abnormal value detection step detects whether the number of times the subject has expressed an emotion includes an abnormal value relative to the reference number of times the subject has expressed an emotion that is stored in association with the subject's identification information; The determination step determines that the subject's current emotional expression is different from usual when the number of times the emotion is generated includes an abnormal value; The output step outputs the determination result.

Images