Facial expression recognition support device, control device, control method, and program

Abstract

One aspect of the present invention is provided with: an imaging unit that captures an image of the expression of a person to be recognized and outputs the image information; a stimulus generation unit that applies a stimulus for promoting the secretion of oxytocin to the person; and a control unit that connects the imaging unit and the stimulus generation unit to each other. The control unit is configured so as to acquire the image information output from the imaging unit, then determine whether the expression of the person corresponds to a first expression corresponding to a negative emotion that is defined in advance or corresponds to a second expression that is different from the first expression on the basis of the image information, and make the stimulus generation unit output a stimulus control signal for generating the stimulus when it is determined that the expression of the person corresponds to the second expression.

Description

[Technical Field]

[0001] One aspect of the present invention relates to a facial expression recognition support device, method, and program used to support a recognizer in recognizing the facial expression of a recognized person. [Background technology]

[0002] For example, when a perceiver such as a psychological counselor recognizes the facial expression of a person being recognized such as a patient while conversing with the person being recognized, it is important that the perceiver maintain a stable mental and physical state. Therefore, for example, Non-Patent Document 1 describes a study in which oxytocin, a neurotransmitter that acts on the parasympathetic nerve and serotonin nerve, is administered to the perceiver, thereby stabilizing the perceiver's psychological state and improving the recognition accuracy of the person being recognized's facial expression. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] G. Domes, M. Heinrichs, A. Michel, C. Berger, and SC Herpertz, “Oxytocin Improves 'Mind-Reading' in Humans,” Biol. Psychiatry, vol. 61, no. 6, pp. 731-733, 2007 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the psychological state of each individual varies, and even within the same individual, the psychological state changes over time. Therefore, when administering oxytocin, the administration and dosage must be adjusted appropriately depending on the individual's state, which places a heavy burden on the individual.

[0005] Furthermore, research has shown that oxytocin administration intensifies fear experiences in mice, and it is assumed that humans may react sensitively to the facial expressions of the recipient when negative emotions such as anger or sadness are expressed, depending on the amount of oxytocin administered. Therefore, if oxytocin is administered to the recipient constantly, there is a possibility that the recipient may not be able to accurately recognize facial expressions depending on their state, which could lead to a decrease in the accuracy of facial expression recognition.

[0006] This invention was made with the above-mentioned circumstances in mind, and aims to provide a technology that eliminates the need to administer oxytocin to the perceiving person, thereby reducing the burden on the perceiving person, and that appropriately suppresses the perceiving person's reaction to the negative emotions of the person being recognized, thereby improving the accuracy of recognizing the facial expressions of the person being recognized. [Means for solving the problem]

[0007] In order to solve the above problems, one aspect of a facial expression recognition support device according to the present invention comprises an imaging unit that captures an image of a facial expression of a person to be recognized and outputs the image information, a stimulus generating unit that provides the person with a stimulus to stimulate the secretion of oxytocin, and a control unit connected to the imaging unit and the stimulus generating unit. The control unit acquires the image information output from the imaging unit and determines, based on the image information, whether the facial expression of the person to be recognized corresponds to a first facial expression corresponding to a predefined negative emotion or to another second facial expression. If the control unit determines that the facial expression of the person to be recognized corresponds to the second facial expression, it outputs a stimulus control signal to the stimulus generating unit to generate the stimulus.

[0008] According to one aspect of the present invention, oxytocin is autonomously secreted in the brain of the person being recognized by the application of an external stimulus. This eliminates the need to administer oxytocin synthesized as a drug to the person being recognized, thereby eliminating the need for management of oxytocin administration and reducing the burden on the person being recognized. Furthermore, the application of stimuli to the person being recognized is not performed when the person being recognized expresses negative emotions such as anger or sadness, but only when the person being recognized expresses other emotions. This prevents the person being recognized from overreacting to facial expressions that express negative emotions, thereby preventing a decline in the accuracy of facial expression recognition. [Effects of the Invention]

[0009] In other words, according to one aspect of the present invention, a technology can be provided that eliminates the need to administer oxytocin to the perceiver, thereby reducing the burden on the perceiver, and appropriately suppresses the perceiver's reaction to the perceived negative emotions of the perceived person, thereby improving the accuracy of recognizing the perceived person's facial expression. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing an example of the hardware configuration of a facial expression recognition support device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the software configuration of the facial expression recognition support device according to the first embodiment of the present invention. [Figure 3] FIG. 3 is a flowchart showing an example of a processing procedure and processing content of facial expression recognition support control executed by the control device of the facial expression recognition support device shown in FIG. [Figure 4] FIG. 4 is a flowchart showing an example of the processing procedure and processing contents of facial expression recognition support control executed by the control device of the facial expression recognition support device according to the second embodiment of the present invention. [Figure 5] FIG. 5 is a diagram illustrating an example of conversation determination information. [Figure 6] FIG. 6 is a diagram showing an example of facial expression determination information. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] [principle] This invention is based on the finding that when a specific external stimulus is applied to a person's skin, this stimulus is transmitted to the brain via the skin's C tactile fibers, causing the hypothalamus to secrete a neurotransmitter called oxytocin, which causes the parasympathetic nervous system to take over and relax the mind and body.When recognizing the facial expression of the person being recognized, an external stimulus that promotes the secretion of oxytocin is applied to the recognizer, thereby stabilizing the psychological state of the recognizer.

[0013] Furthermore, it is predicted that when oxytocin is secreted, the perceiver will overreact to facial expressions that correspond to negative emotions in the person being recognized. Therefore, in this invention, when performing facial expression recognition, the perceiver is controlled so that no stimulation is given to the perceiver when the person being recognized shows negative emotions, and stimulation is given only when other facial expressions are shown.

[0014] This eliminates the need for management of oxytocin administration to the person being recognized, thereby reducing the burden on the person being recognized. Furthermore, it prevents the person being recognized from overreacting to facial expressions when the person being recognized expresses negative emotions, thereby improving the accuracy of facial expression recognition.

[0015] [First embodiment] (Configuration example) The facial expression recognition support device according to the first embodiment of the present invention is used to support a medical professional in recognizing a patient's facial expression, for example, when the medical professional provides rehabilitation or psychological counseling to the patient. Note that the recognizer is not limited to a medical professional, and the recognized person is not limited to a patient.

[0016] 2 and 3 are block diagrams showing an example of a hardware configuration and a software configuration, respectively, of a facial expression recognition support device.

[0017] The facial expression recognition support device comprises a control device 1, a sound collection unit 2, an imaging unit 3, and a stimulus generation unit 4. The sound collection unit 2 has, for example, a microphone and an audio amplifier circuit, and collects the sound emitted by a person to be recognized, such as a patient, when they are having a conversation, and outputs an audio signal. The imaging unit 3 has a camera, and captures an image of a predetermined target range including the face of the person to be recognized, and outputs an image signal.

[0018] The stimulus generator 4 applies a contact stimulus to the skin of the recipient, such as a medical professional, to induce the release of oxytocin. For example, the stimulus generator 4 has a contact part made of a cloth material such as velvet that feels soft to humans, and is configured to slide at a speed of about 3 to 10 cm per second using a step motor while the contact part is in contact with the recipient's skin.

[0019] Alternatively, the stimulus generator 4 may be one that stimulates the C tactile fibers of the skin by applying electrical stimulation at a specific low frequency (5 Hz), for example.

[0020] The control device 1 has a control unit 11 that uses a hardware processor such as a central processing unit (CPU), and this control unit 11 is connected via a bus 15 to a memory unit having a program memory unit 12 and a data memory unit 13, and an input / output interface (hereinafter, interface will be abbreviated as I / F) unit 14.

[0021] Although a personal computer, for example, can be used as the control device 1, a smartphone or a tablet terminal can also be used. The control device 1 can also be a wearable terminal, in which case the wearable terminal and the stimulus generation unit 4 can be configured as an integrated unit.

[0022] The input / output I / F unit 14 has the function of converting the audio signal output from the sound collection unit 2 into digitized audio data, the function of converting the image signal output from the imaging unit 3 into digitized image data, and the function of converting the stimulus signal output from the control unit 11 into a drive signal for the stimulus generation unit 4 and outputting it to the stimulus generation unit 4.

[0023] The input / output I / F unit 14 may be provided with a wireless interface function that employs a low-power wireless data communication standard such as Bluetooth (registered trademark). By using the wireless interface function, signals can be transmitted and received between the control device 1 and the sound collection unit 2, the imaging unit 3, and the stimulus generation unit 4 in a cordless manner.

[0024] The program storage unit 12 is configured by combining, for example, a nonvolatile memory such as a solid state drive (SSD) as a storage medium that can be written to and read from at any time, and a nonvolatile memory such as a read only memory (ROM), and stores middleware such as an operating system (OS) as well as application programs required to execute various controls according to the first embodiment. Hereinafter, the OS and each application program will be collectively referred to as a program.

[0025] The data storage unit 13 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read from at any time as a storage medium, and a volatile memory such as a RAM (Random Access Memory), and the storage area thereof is provided with a conversation determination information storage unit 131 and a facial expression determination information storage unit 132 as main storage units required to implement the first embodiment of the present invention.

[0026] The conversation determination information storage unit 131 stores information representing the determination result of whether or not the person being recognized is having a conversation per unit time. The facial expression determination information storage unit 132 stores information representing the determination result of the facial expression of the person being recognized per unit time.

[0027] The control unit 11 includes, as processing functions necessary for carrying out the first embodiment of the present invention, a voice data acquisition processing unit 111, a conversation determination processing unit 112, an image data acquisition processing unit 113, a facial expression determination processing unit 114, a stimulus application determination processing unit 115, and a stimulus signal generation processing unit 116. These processing units 111 to 116 are all realized by causing a hardware processor of the control unit 11 to execute an application program stored in the program storage unit 12.

[0028] Note that some or all of the processing units 111 to 116 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0029] The voice data acquisition processing unit 111 performs processing to acquire digital voice data corresponding to a voice signal including the voice of the person being recognized collected by the sound collection unit 2 from the input / output I / F unit 14 during support control of facial expression recognition.

[0030] The conversation determination processing unit 112 determines the presence or absence of conversation for each preset unit time based on the voice data acquired by the voice data acquisition processing unit 111. Then, information representing the determination result is stored in the conversation determination information storage unit 131 in association with the unit time.

[0031] The image data acquisition processing unit 113 performs processing to acquire image data corresponding to the image signal of the target range including the face of the person to be recognized, captured by the imaging unit 3, from the input / output I / F unit 14 during the support control of facial expression recognition.

[0032] The facial expression determination processing unit 114 determines the facial expression of the person to be recognized for each unit time based on the image data acquired by the image data acquisition processing unit 113. Then, information representing the determination result is stored in the facial expression determination information storage unit 132 in association with the unit time.

[0033] The stimulus application determination processing unit 115 determines whether or not to apply a stimulus to the perceiver for each unit time based on the conversation determination information stored in the conversation determination information storage unit 131 and the facial expression determination information stored in the facial expression determination information storage unit 132. An example of this determination processing will be described in the operation example.

[0034] The stimulus signal generation processing unit 116 generates a stimulus signal for a period or timing when a stimulus should be applied based on the stimulus application determination result by the stimulus application determination processing unit 115, and outputs the generated stimulus signal to the input / output I / F unit 14. The input / output I / F unit 14 converts the stimulus signal into a drive signal for driving, for example, a step motor of the stimulus generation unit 4, and outputs the drive signal to the stimulus generation unit 4.

[0035] (Example of operation) Next, an example of the operation of the facial expression recognition support device configured as above will be described.

[0036] Prior to the execution of facial expression recognition assistance, the person to be recognized wears the stimulus generating unit 4, for example, on their wrist or forearm. The stimulus generating unit 4 is worn, for example, by a band or the like, so that the contact part made of a soft material is in light contact with the skin of the person to be recognized. In addition, the sound collecting unit 2 and the imaging unit 3 are positioned facing the direction of the person to be recognized.

[0037] When the preparation for support is completed and the recognizer inputs a support start request to the facial expression recognition support device, the control device 1 of the facial expression recognition support device executes facial expression recognition support control as follows.

[0038] FIG. 3 is a flowchart showing an example of the processing procedure and processing content of the facial expression recognition support control executed by the control device 1 of the facial expression recognition support device.

[0039] (1) Acquiring voice data and determining whether or not a conversation is occurring When the control device 1 of the facial expression recognition support device recognizes the input of the request to start support in step S10, under the control of the voice data acquisition processing unit 111, in step S11, it acquires voice data corresponding to the voice signal containing the voice of the person being recognized that is continuously output from the sound collection unit 2 from the input / output I / F unit 14.

[0040] Next, in step S12, the control device 1 of the facial expression recognition support device determines whether or not there is conversation based on the voice data at predetermined unit time intervals (for example, 10 seconds) under the control of the conversation determination processing unit 112. Then, information representing the determination result is stored in the conversation determination information storage unit 131.

[0041] For example, the conversation determination processing unit 112 determines whether the amplitude level of a voice signal included in voice data acquired during 10 seconds of a certain unit time T1 is continuous over the unit time T1. If the voice signal is continuous, the conversation determination processing unit 112 determines that the person being recognized is talking, and stores the determination result "yes" in the conversation determination information storage unit 131 in association with the next unit time T2.

[0042] Similarly, the conversation determination processing unit 112 determines whether or not there is conversation for each of the voice signals included in the voice data acquired at each unit time after unit time T2, and stores the determination results in association with the next unit times T3, T4, ... in the conversation determination information storage unit 131. Fig. 5 shows an example of the determination results of whether or not there is conversation stored in this manner.

[0043] As a method for determining whether or not a conversation is occurring, a method may be used in which, for example, morphological analysis technology is used to determine that the person being recognized is speaking if a certain number or more meaningful words are recognized from the speech signal within a unit time.

[0044] (2) Acquiring image data and determining facial expressions In addition, in parallel with the process of determining whether or not a conversation is occurring, the control device 1 of the facial expression recognition support device, under the control of the image data acquisition processing unit 113, acquires image data corresponding to the image signal output from the imaging unit 3 on a frame-by-frame basis from the input / output I / F unit 14 in step S13.

[0045] Next, in step S14, the control device 1 of the facial expression recognition support device determines the facial expression of the person being recognized based on the image data for each unit time (10 seconds) under the control of the facial expression determination processing unit 114. Then, information representing the determination result is stored in the facial expression determination information storage unit 132.

[0046] For example, the facial expression determination processing unit 114 determines the type of facial expression for the acquired image data for each of the unit times T1, T2, T3, ... using, for example, well-known face recognition technology. Possible types of facial expressions to be determined include, but are not limited to, "anger," "happiness," "fear," "surprise," "disgust," and "neutral."

[0047] The facial expression determination processing unit 114 stores the facial expression type determination results obtained in each unit time T1, T2, T3, ... in the facial expression determination information storage unit 132 in association with the next unit time T2, T3, T4, .... Fig. 6 shows an example of the facial expression type determination results stored in this manner.

[0048] (3) Judgment of stimulus application and output of stimulus signal The control device 1 of the facial expression recognition support device, under the control of the stimulus application determination processing unit 115, first reads conversation determination information, i.e., information representing the conversation history, from the conversation determination information storage unit 131 in step S15 for each unit time, and determines whether the person being recognized is engaged in a conversation based on the read conversation determination information.

[0049] If it is determined that the person is having a conversation, the stimulus application determination processing unit 115 then reads the corresponding facial expression determination information from the facial expression determination information storage unit 132 in step S16, and determines whether the type of facial expression of the person being recognized at that time is an expression corresponding to a negative emotion or something else, based on the read facial expression determination information.

[0050] If the result of this determination is that the type of facial expression of the person being recognized is an expression other than an expression corresponding to a negative emotion such as "anger," "fear," or "disgust," such as "joy," "surprise," or "neutral," the stimulus application determination processing unit 115 generates a stimulus signal in step S17. Then, the generated stimulus signal is output to the input / output I / F unit 14 during a unit time period. The input / output I / F unit 14 converts the stimulus signal into a drive signal for a step motor and outputs it to the stimulus generation unit 4.

[0051] As a result, the step motor of the stimulus generator 4 operates, causing the contact part to slide back and forth at a speed of, for example, 3 to 10 cm per second over the unit time period, providing a soft stimulus to the subject's skin. This stimulus is transmitted to the subject's brain via the subject's C tactile fibers, which stimulates the secretion of oxytocin and stabilizes the subject's psychological state.

[0052] On the other hand, if the person being recognized is not engaged in a conversation, or if the person being recognized is engaged in a conversation but the type of facial expression corresponds to a negative emotion, the control device 1 of the facial expression recognition support device determines whether or not to end the support in step S18. The end of the support is performed by determining, for example, whether or not the person being recognized has not been engaged in a conversation for two consecutive unit times.

[0053] If the result of the support end determination is that the person to be recognized is in conversation, the control device 1 of the facial expression recognition support device returns to step S11 and continues to execute a series of support controls in steps S11 to S18. On the other hand, if the person to be recognized has finished talking, the support control ends.

[0054] It is also possible to determine that support has ended when it is detected that the recognizer has input a support end request to the control device 1 of the facial expression recognition support device.

[0055] (Actions and Effects) As described above, in the first embodiment, voice data including the speech of the person to be recognized and image data including the face of the person to be recognized are acquired, and for each unit time, it is determined whether the person to be recognized is talking or not, and the type of facial expression of the person to be recognized is determined. Then, if the person to be recognized is talking and the type of facial expression at that time is an expression other than an expression corresponding to a negative emotion, a stimulation signal is output to drive the stimulation generating unit 4, which stimulates the skin of the person to secrete oxytocin.

[0056] Therefore, the person who recognizes does not need to administer oxytocin synthesized as a drug, which eliminates the management burden associated with administering oxytocin. Also, if the facial expression of the person being recognized corresponds to a negative emotion, no stimulation is given to the person being recognized. Therefore, when the person being recognized has an expression corresponding to a negative emotion, the secretion of oxytocin can prevent the person from overreacting, thereby reducing misrecognition of the facial expression of the person being recognized and improving recognition accuracy.

[0057] [Second embodiment] In the second embodiment of the present invention, a stimulus signal is output in a timely manner when the facial expression of the person being acknowledged changes from an expression corresponding to a negative emotion to another expression while the person being acknowledged is conversing.

[0058] In the second embodiment, the basic configuration of the control device 1 of the facial expression recognition support device is the same as that in the first embodiment, and therefore will be described with reference to FIGS.

[0059] 4 is a flowchart showing an example of the processing procedure and processing content of facial expression recognition support control executed by the control device 1 of the facial expression recognition support device according to the second embodiment of the present invention. In FIG. 4, the same processing parts as those in FIG. 3 are denoted by the same reference numerals and will be described.

[0060] When the control device 1 of the facial expression recognition support device detects input of a support start request in step S10, it acquires, under the control of the voice data acquisition processing unit 111, in step S11 from the input / output I / F unit 14, voice data corresponding to a voice signal including the spoken voice of the person to be recognized that is output in time series from the sound collection unit 2. Then, under the control of the conversation determination processing unit 112, the control device 1 of the facial expression recognition support device determines, in step S12, whether or not a conversation is occurring based on the acquired voice data, and sequentially stores the determination results in the conversation determination information storage unit 131 as conversation history information.

[0061] In parallel with this, the control device 1 of the facial expression recognition support device, under the control of the image data acquisition processing unit 113, acquires image data corresponding to image signals including the face of the person to be recognized that are output in time series from the imaging unit 3 in step S13 from the input / output I / F unit 14. Then, under the control of the facial expression determination processing unit 114, the control device 1 of the facial expression recognition support device continuously determines the type of facial expression of the person to be recognized in real time based on the image data in step S14, and sequentially stores changes in the determined type of facial expression in the facial expression determination information storage unit 132 as facial expression history information.

[0062] While determining whether the person being recognized is having a conversation and judging the person's facial expression, the control device 1 of the facial expression recognition support device reads conversation history information from the conversation judgment information storage unit 131 in step S20 under the control of the stimulus application judgment processing unit 115, and judges whether the person being recognized is having a conversation based on this conversation history information. At the same time, in step S21, the stimulus application judgment processing unit 115 reads facial expression history information from the facial expression judgment information storage unit 132, and judges the latest facial expression change of the person being recognized based on the read facial expression history information. Then, it is judged whether the facial expression of the person being recognized has changed to a specific facial expression other than a facial expression corresponding to a negative emotion during the conversation.

[0063] When, as a result of the judgment made in the above steps S20 and S21, the facial expression of the person being recognized changes during the conversation to a specific facial expression other than an expression corresponding to a negative emotion, the stimulus application judgment processing unit 115 instructs the stimulus signal generation processing unit 116 to generate a stimulus signal at that time.

[0064] When the stimulus signal generation processing unit 116 receives an instruction to generate a stimulus signal from the stimulus application determination processing unit 115, it generates a stimulus signal at that time and outputs it to the input / output I / F unit 14. As a result, a drive signal corresponding to the stimulus signal is output from the input / output I / F unit 14 and supplied to the stimulus generation unit 4.

[0065] As a result, the step motor of the stimulus generator 4 is operated by the drive signal, causing the contact part to slide back and forth at a speed of, for example, 3 to 10 cm per second, providing a soft stimulus to the skin of the person being recognized. This stimulus is transmitted to the brain via the C tactile fibers of the person being recognized, thereby promoting the secretion of oxytocin. In other words, when the facial expression of the person being recognized changes to a specific facial expression other than the facial expression corresponding to the negative emotion described above during the conversation, the stimulus generator 4 operates to provide a stimulus to the skin of the person being recognized, thereby promoting the secretion of oxytocin.

[0066] On the other hand, if the person being recognized is not speaking or if the most recent facial expression during a conversation is a negative emotion, the stimulus application determination processing unit 115 does not issue an instruction to generate a stimulus signal. Therefore, no stimulus is applied to the person being recognized.

[0067] As described above, according to the second embodiment, it is continuously determined in real time whether the person to be recognized is engaged in a conversation and whether the facial expression changes to a specific facial expression other than a facial expression corresponding to a negative emotion. Then, when the facial expression of the person to be recognized changes to the specific facial expression during the conversation, the stimulus generating unit 4 operates to apply a stimulus to the skin of the person to be recognized.

[0068] Therefore, the perceiver is given timely stimuli according to specific facial expression changes of the person being recognized during conversation, which allows the perceiver to stimulate the secretion of oxytocin at the optimal timing, thereby further improving the accuracy of the perceiver's recognition of the person being recognized.

[0069] [Other embodiments] (1) In the above embodiments, the functions of the control device 1 are provided in a terminal such as a personal computer. However, the functions of the control device 1 may be provided in a server computer on the web or in the cloud, thereby allowing multiple users to share the facial expression recognition support functions provided in the control device.

[0070] (2) In addition, a learning model may be created by performing machine learning based on past conversation history information and facial expression history information, and this learning model may be used to predict changes in the conversation and facial expression of the person being recognized, and the timing of providing stimuli may be determined based on the prediction results.

[0071] (3) In the above embodiments, the case where the application of stimuli is controlled by determining the change in facial expression of the person being recognized while he / she is talking has been described, but the application of stimuli may also be controlled by determining the change in facial expression of the person being recognized regardless of whether he / she is talking or not. In this way, the present invention can be applied even when the person being recognized is deaf or mute.

[0072] In addition, the types of facial expressions to which stimuli are applied and the method for determining them, the configuration of the stimulus generation unit, the functional configuration of the control unit, its processing procedures and processing contents, etc. can be modified and implemented in various ways without departing from the spirit of this invention.

[0073] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0074] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0075] 1...Control device 2…Sound collection section 3...Image capture unit 4...Stimulus generating section 11...Control unit 12...Program memory section 13...Data storage unit 14...Input / output interface 15...Bus 111...Audio data acquisition processing unit 112...conversation determination processing unit 113...Image data acquisition processing unit 114...Facial expression determination processing unit 115... Stimulus application determination processing unit 116...Stimulus signal generation processing unit 131...conversation determination information storage unit 132...Facial expression determination information storage unit

Claims

1. A facial expression recognition support device that supports a recognizer in recognizing a facial expression of a recognized person, an imaging unit that captures an image of the facial expression of the person to be recognized and outputs the image information; a stimulus generating unit that provides the perceiving person with a stimulus to promote the secretion of oxytocin; a control unit connected to the imaging unit and the stimulus generating unit; Equipped with The control unit a first processing unit that acquires the image information output from the imaging unit and determines whether the facial expression of the person to be recognized corresponds to a first facial expression corresponding to a predefined negative emotion or to another second facial expression based on the acquired image information; a second processing unit that outputs a stimulus signal to the stimulus generating unit to generate the stimulus when it is determined that the facial expression of the person to be recognized corresponds to the second facial expression; A facial expression recognition support device comprising:

2. a sound collection unit that detects a voice uttered by the person to be recognized and outputs voice information corresponding to the voice; The control unit Further provided is a third processing unit that acquires the voice information output from the sound collection unit and determines whether the person to be recognized is talking based on the acquired voice information, The second processing unit outputs a stimulus control signal to the stimulus generating unit to generate the stimulus when it is determined that the person to be recognized is engaged in conversation and the facial expression of the person to be recognized corresponds to the second facial expression. The facial expression recognition support device according to claim 1 .

3. A control device connected to an imaging unit that captures the facial expression of the person to be recognized and outputs the image information, and a stimulus generating unit that provides a stimulus to the person to be recognized that stimulates the secretion of oxytocin, a first processing unit that acquires the image information output from the imaging unit and determines whether the facial expression of the person to be recognized corresponds to a first facial expression corresponding to a predefined negative emotion or to another second facial expression based on the acquired image information; a second processing unit that outputs a stimulus signal to the stimulus generating unit to generate the stimulus when it is determined that the facial expression of the person to be recognized corresponds to the second facial expression; A control device comprising:

4. The control device according to claim 3 , wherein the first processing unit determines, as the second facial expression, whether the facial expression of the person to be recognized corresponds to an emotion of joy, surprise, or neutral.

5. The control device according to claim 3 , wherein the second processing unit outputs the stimulus signal to the stimulus generating unit to generate the stimulus during a period in which the facial expression of the person being recognized is determined to correspond to the second facial expression.

6. The control device according to claim 3, wherein the second processing unit outputs the stimulus signal to the stimulus generating unit to generate the stimulus in synchronization with the timing when the facial expression of the person being recognized changes from the first facial expression to the second facial expression.

7. A control method for facial expression recognition support is carried out by a control device connected to an imaging unit that captures the facial expression of a person to be recognized and outputs the image information, and a stimulus generating unit that provides a stimulus to the person to recognize that stimulates the secretion of oxytocin, a step of acquiring the image information output from the imaging unit, and determining whether the facial expression of the person to be recognized corresponds to a first facial expression corresponding to a predefined negative emotion or a second facial expression based on the acquired image information; outputting a stimulus control signal to the stimulus generator to generate the stimulus when the facial expression of the person to be recognized is determined to correspond to the second facial expression; A control method comprising:

8. 7. A program for causing a processor included in the control device to execute at least one of the processes executed by each processing unit included in the control device according to claim 3.

Images