Non-contact visual intelligence platform system for complex emotional exchange between robot and human and method for using same

Abstract

According to a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human and a method for using same, the system comprises: a physiological signal measurement and analysis unit which analyzes the physiological emotion of a subject by extracting a physiological signal of the subject for complex emotional exchange between a robot and a human; a complex state measurement and analysis unit which analyzes a state-like emotion of the subject by extracting a state-like complex signal, including the facial expression of the subject and gestures and the posture of the subject, for complex emotional exchange between the robot and the human; and a complex emotion analysis and recognition unit which precisely analyzes changes in and the intensity of emotions of the subject on the basis of a physiological emotion analysis signal of the physiological signal measurement and analysis unit and a state-like complex emotion analysis signal of the complex state measurement and analysis unit, and recognizes the emotion of the subject.

Description

A non-contact visual intelligence platform system for complex emotional exchange between robots and humans and its use method.

[0001] The present invention is a research result of the "Development of an artificial intelligence-based body balance diagnosis / correction platform through motion big data analysis" industry-academia R&BD collaboration commercialization project supported by the Daegu Digital Innovation Promotion Agency, and particularly relates to a non-contact visual intelligence platform system for complex emotional exchange between robots and humans and a method of using the same, and more specifically, to a non-contact visual intelligence platform system for complex emotional exchange between robots and humans, which enables mutual emotional exchange and interaction between workers or users and robots through visual intelligence that enables robots to have emotions, and a method of using the same.

[0002] The content described in this section merely provides background information for one embodiment of the present invention and does not constitute prior art.

[0003]

[0004] With the recent advancement of artificial intelligence technology, emotion recognition has emerged as an important field of research. Emotions refer to the state of mind or feelings that arise in response to a phenomenon or event. These emotions can be expressed through words, gestures, facial expressions, and other nonverbal cues, and many physiological signals can also convey information about emotional states. Facial expressions are the most widely used indicator for emotion recognition. However, because facial expressions can be controlled and manipulated, it is difficult to discern truly felt emotions solely from facial expressions.

[0005]

[0006] To address these issues, research on emotion recognition using independent biosignals that can be continuously measured and are beyond the individual's control has been actively underway. Numerous previous studies have demonstrated a strong correlation between human emotions and biosignals, and emotion recognition through biosignals is controlled by the autonomic nervous system, making it possible to obtain accurate results. However, existing emotion analysis methods, when implemented using a single analysis model, suffer from the following issues:

[0007]

[0008] First, electroencephalography (EEG) for emotion analysis achieves maximum accuracy through comprehensive analysis of EEG signals and ensures model reliability by validating multiple datasets. However, it suffers from issues such as limited availability of public datasets, performance uncertainty, and lack of adaptability. Furthermore, electrocardiography (ECG) for emotion analysis achieves maximum accuracy for ECG signals and ensures reliability by validating multiple datasets, but it also suffers from limitations in the use of hyperparameters and fusion technologies. Furthermore, galvanic skin conductance (GSR) for emotion analysis achieves maximum accuracy for GSR signals and can be validated across multiple datasets, but it suffers from issues such as performance uncertainty and lack of adaptability. Furthermore, eye tracking (ET) for emotion analysis generates simple models using datasets generated from various stimuli, but it suffers from issues such as limited public datasets and lack of adaptability. Furthermore, facial image recognition for emotion analysis can recognize emotions using facial images generated and validated across multiple datasets, but it suffers from issues such as performance uncertainty and lack of adaptability.

[0009]

[0010] As such, each existing single analysis model for sentiment analysis suffers from limitations such as limited availability of public datasets, performance uncertainty, and lack of adaptability, which in turn limits the reliability of emotion recognition. Republic of Korea Patent No. 10-1307248 is disclosed as a prior art document.

[0011]

[0012] The background technology described above is technical information that the inventor possessed for the purpose of deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be said to be publicly known technology disclosed to the general public prior to the application for the present invention.

[0013] The present invention has been proposed to solve the above-mentioned problems of the existing proposed methods, and the purpose of the present invention is to provide a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, which comprises a physiological signal measurement and analysis unit that extracts a physiological signal of a subject for complex emotional exchange between a robot and a human and analyzes the physiological emotion of the subject, a complex state measurement and analysis unit that extracts a state-specific complex signal including the subject's facial expression, gesture, and posture for complex emotional exchange between a robot and a human, and analyzes the state-specific complex emotion of the subject, and a complex emotion analysis and recognition unit that precisely analyzes the change and intensity of the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit and the state-specific complex emotion analysis signal of the complex state measurement and analysis unit, and recognizes the subject's emotion, thereby enabling mutual emotional exchange and interaction between a worker or user and a robot through visual intelligence that enables the robot to have emotions, and a method of using the same.

[0014]

[0015] In addition, the present invention provides a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, and a method of using the same, which enables mutual emotional exchange and interaction between a worker or user and a robot through visual intelligence that enables the robot to have emotions, and enables complex emotional analysis through measurement of the state-wise complex state of the physiological signals, facial expressions, gestures, and posture analysis of the subject, thereby predicting complex emotions with multiplexing emotions through complex dimensional data rather than fragmentary visual data, thereby enabling higher accuracy and smoother implementation of human-robot interaction through high recognition performance. Another object of the present invention is to provide a system for complex emotional exchange between a robot and a human, and a method of using the same.

[0016]

[0017] However, the technical problems to be solved by the present invention are not limited to the technical problems described above, and other technical problems may exist.

[0018] In order to achieve the above-mentioned purpose, a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to the features of the present invention is provided.

[0019] As a non-contact visual intelligence platform system for complex emotional exchange between robots and humans,

[0020] A physiological signal measurement and analysis unit that extracts physiological signals of the subject for complex emotional exchange between robots and humans and analyzes the subject's physiological emotions;

[0021] A complex state measurement analysis unit that extracts complex state signals including the subject's facial expressions, gestures, and postures for complex emotional exchange between robots and humans and analyzes the subject's complex state emotions; and

[0022] The composition is characterized by including a complex emotion analysis recognition unit that precisely analyzes the change and intensity of the subject's emotions based on the physiological emotion analysis signal of the physiological signal measurement analysis unit and the state-based complex emotion analysis signal of the complex state measurement analysis unit, and recognizes the subject's emotions.

[0023]

[0024] Preferably, the physiological signal measurement and analysis unit,

[0025] Extracting physiological signals of a subject for complex emotional exchange between a robot and a human, wherein the physiological signals of the subject may include one or more of electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET).

[0026]

[0027] More preferably, the physiological signal measurement and analysis unit,

[0028] It can be configured to include a camera that extracts physiological signals of the subject for complex emotional exchange between robots and humans and analyzes the subject's physiological emotions, as well as computer vision.

[0029]

[0030] Even more preferably, the physiological signal measurement and analysis unit,

[0031] It can function as a multi-scale spectrum-based multi-dimensional data analyzer capable of measuring the physiological signals of a subject in a non-contact manner and processing visual data of the measured non-contact physiological signals.

[0032]

[0033] Preferably, the composite state measurement analysis unit,

[0034] It can be configured to include a camera and computer vision to analyze the subject's state complex emotions by extracting a state complex signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human.

[0035]

[0036] More preferably, the composite state measurement analysis unit,

[0037] By analyzing the facial expressions of the subject for complex emotional exchanges between robots and humans, the subject's emotions of joy, anger, sorrow, and pleasure can be analyzed.

[0038]

[0039] More preferably, the composite state measurement analysis unit,

[0040] By analyzing the subject's facial expressions, gestures, and postures for complex emotional exchanges between robots and humans, the subject's negative and positive emotions can be analyzed.

[0041]

[0042] More preferably, the complex emotion analysis recognition unit,

[0043] Based on the physiological emotion analysis signal of the physiological signal measurement analysis unit and the state-based complex emotion analysis signal of the complex state measurement analysis unit, the change and intensity of the subject's emotion can be precisely analyzed and a complex emotion analysis algorithm for recognizing the subject's emotion can be implemented.

[0044]

[0045] A method of using a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to the features of the present invention to achieve the above-mentioned purpose is as follows.

[0046] A method of utilizing a non-contact visual intelligence platform system for complex emotional exchange between robots and humans,

[0047] (1) A step in which a physiological signal measurement and analysis unit extracts physiological signals of a subject for complex emotional exchange between a robot and a human and analyzes the subject's physiological emotions;

[0048] (2) A step in which a complex state measurement analysis unit extracts a complex state signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyzes the subject's complex state emotion; and

[0049] (3) The composite emotion analysis recognition unit precisely analyzes the change and intensity of the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement analysis unit and the state-based composite emotion analysis signal of the composite state measurement analysis unit, and includes a step of recognizing the subject's emotion.

[0050]

[0051] Preferably, the physiological signal measurement and analysis unit in the above step (1) is

[0052] Extracting physiological signals of a subject for complex emotional exchange between a robot and a human, wherein the physiological signals of the subject may include one or more of electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET).

[0053]

[0054] More preferably, the physiological signal measurement and analysis unit,

[0055] It can be configured to include a camera that extracts physiological signals of the subject for complex emotional exchange between robots and humans and analyzes the subject's physiological emotions, as well as computer vision.

[0056]

[0057] Even more preferably, the physiological signal measurement and analysis unit,

[0058] It can function as a multi-scale spectrum-based multi-dimensional data analyzer capable of measuring the physiological signals of a subject in a non-contact manner and processing visual data of the measured non-contact physiological signals.

[0059]

[0060] Preferably, the composite state measurement analysis unit in the above step (2) is

[0061] It can be configured to include a camera and computer vision to analyze the subject's state complex emotions by extracting a state complex signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human.

[0062]

[0063] More preferably, the composite state measurement analysis unit,

[0064] By analyzing the facial expressions of the subject for complex emotional exchanges between robots and humans, the subject's emotions of joy, anger, sorrow, and pleasure can be analyzed.

[0065]

[0066] More preferably, the composite state measurement analysis unit,

[0067] By analyzing the subject's facial expressions, gestures, and postures for complex emotional exchanges between robots and humans, the subject's negative and positive emotions can be analyzed.

[0068]

[0069] More preferably, the complex emotion analysis recognition unit in the above step (3) is

[0070] Based on the physiological emotion analysis signal of the physiological signal measurement analysis unit and the state-based complex emotion analysis signal of the complex state measurement analysis unit, the change and intensity of the subject's emotion can be precisely analyzed and a complex emotion analysis algorithm for recognizing the subject's emotion can be implemented.

[0071] According to the non-contact visual intelligence platform system for complex emotional exchange between a robot and a human proposed in the present invention and the method of using the same, the system comprises: a physiological signal measurement and analysis unit that extracts physiological signals of a subject for complex emotional exchange between a robot and a human and analyzes the physiological emotions of the subject; a complex state measurement and analysis unit that extracts state-based complex signals including facial expressions, gestures, and postures of the subject for complex emotional exchange between a robot and a human and analyzes the state-based complex emotions of the subject; and a complex emotion analysis and recognition unit that precisely analyzes the change and intensity of the subject's emotions based on the physiological emotion analysis signals of the physiological signal measurement and analysis unit and the state-based complex emotion analysis signals of the complex state measurement and analysis unit, and recognizes the emotions of the subject, thereby enabling mutual emotional exchange and interaction between a worker or user and a robot through visual intelligence that enables the robot to have emotions.

[0072]

[0073] In addition, according to the non-contact visual intelligence platform system for complex emotional exchange between a robot and a human of the present invention and the method of using the same, mutual emotional exchange and interaction between a worker or user and a robot are enabled through visual intelligence that enables a robot to have emotions, and by enabling complex emotional analysis through measurement of the state-specific complex state of the physiological signals, facial expressions, gestures, and posture analysis of the subject, it is possible to predict complex emotions with multiplexing emotions through complex dimensional data rather than fragmentary visual data, thereby achieving higher accuracy and enabling smoother implementation of human-robot interaction through high recognition performance.

[0074]

[0075] In addition, the various advantageous advantages and effects of the present invention are not limited to the above-described contents, and will be more easily understood in the process of explaining specific embodiments of the present invention.

[0076] FIG. 1 is a diagram illustrating the configuration of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention, with functional blocks.

[0077] FIG. 2 is a diagram illustrating the configuration of a physiological signal measurement and analysis unit of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention, as a functional block.

[0078] FIG. 3 is a diagram illustrating the configuration of a physiological signal measured by a physiological signal measurement and analysis unit of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention.

[0079] FIG. 4 is a diagram illustrating the configuration of facial expression analysis and gesture and posture analysis measured in a complex state measurement and analysis unit of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention.

[0080] FIG. 5 is a drawing illustrating an example of facial expression emotions of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention.

[0081] FIG. 6 is a diagram illustrating an implementation mechanism of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention.

[0082] FIG. 7 is a diagram illustrating a flow chart of a method of using a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to one embodiment of the present invention.

[0083] <Explanation of symbols>

[0084] 100: Non-contact visual intelligence platform system according to one embodiment of the present invention

[0085] 110: Physiological signal measurement and analysis department

[0086] 120: Complex State Measurement Analysis Unit

[0087] 130: Complex Emotion Analysis Recognition Unit

[0088] S110: A step in which the physiological signal measurement and analysis unit extracts the physiological signals of the subject for complex emotional exchange between the robot and the human and analyzes the physiological emotions of the subject.

[0089] S120: A step in which a complex state measurement analysis unit extracts a complex state signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyzes the subject's complex state emotion.

[0090] S130: A step in which the complex emotion analysis recognition unit precisely analyzes the change and intensity of the subject's emotions based on the physiological emotion analysis signal of the physiological signal measurement analysis unit and the state-based complex emotion analysis signal of the complex state measurement analysis unit, and recognizes the subject's emotions.

[0091] Below, with reference to the attached drawings, embodiments of the present invention are described in detail so that those skilled in the art can easily implement them. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, irrelevant parts have been omitted for clarity of description, and similar reference numerals have been used throughout the specification to indicate similar parts.

[0092]

[0093] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "indirectly connected" with another element in between. Furthermore, when a part is said to "include" a component, this should be understood to mean that, unless specifically stated to the contrary, it may include other components rather than excluding them, and does not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0094]

[0095] The following examples are provided as detailed explanations to aid understanding of the present invention and do not limit the scope of the invention. Therefore, inventions with the same scope and function as the present invention are also within the scope of the present invention.

[0096]

[0097] In addition, each configuration, process, procedure or method included in each embodiment of the present invention may be shared within a scope that is not technically inconsistent with each other.

[0098]

[0099] FIG. 1 is a diagram illustrating the configuration of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention as a functional block. As illustrated in FIG. 1, a non-contact visual intelligence platform system (100) for complex emotional exchange between a robot and a human according to an embodiment of the present invention may be configured to include a physiological signal measurement and analysis unit (110) that extracts physiological signals of a subject for complex emotional exchange between a robot and a human and analyzes the physiological emotions of the subject, a complex state measurement and analysis unit (120) that extracts state-based complex signals including facial expressions, gestures, and postures of the subject for complex emotional exchange between a robot and a human and analyzes the state-based complex emotions of the subject, and a complex emotion analysis and recognition unit (130) that precisely analyzes changes and intensities of the subject's emotions and recognizes the emotions of the subject based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotion analysis signal of the complex state measurement and analysis unit (120). Hereinafter, with reference to the attached drawings, the specific configuration of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention will be described in detail.

[0100]

[0101] FIG. 2 is a diagram illustrating the configuration of a physiological signal measurement and analysis unit of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention as a functional block, FIG. 3 is a diagram illustrating the configuration of a physiological signal measured by a physiological signal measurement and analysis unit of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention, FIG. 4 is a diagram illustrating the configuration of facial expression analysis and gesture and posture analysis measured by a complex state measurement and analysis unit of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention, FIG. 5 is a diagram illustrating an example of an emotion of a facial expression of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention, and FIG. 6 is a diagram illustrating an implementation mechanism of a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention.

[0102]

[0103] The physiological signal measurement and analysis unit (110) is configured to extract the physiological signals of the subject for complex emotional exchange between a robot and a human and analyze the physiological emotions of the subject. As illustrated in FIG. 3, this physiological signal measurement and analysis unit (110) extracts the physiological signals of the subject for complex emotional exchange between a robot and a human, and the physiological signals of the subject may include one or more of electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET). Here, the physiological signal measurement and analysis unit (110) may be configured to include a camera and computer vision for extracting the physiological signals of the subject for complex emotional exchange between a robot and a human and analyzing the physiological emotions of the subject, as illustrated in FIG. 2.

[0104]

[0105] In addition, the physiological signal measurement and analysis unit (110) can function as a multi-scale spectrum-based multidimensional data analyzer capable of measuring the physiological signals of the subject in a non-contact manner and processing the visual data of the measured non-contact physiological signals. This physiological signal measurement and analysis unit (110) performs non-contact computer vision biosignal analysis, and extracts physiological signals such as the subject's electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET) in a non-contact manner by using a camera and computer vision technology. This non-contact method of extracting physiological signals can improve the user experience, increase the efficiency of data collection, and reduce privacy infringement because it does not directly attach sensors to the user's body.

[0106]

[0107] In addition, the physiological signal measurement and analysis unit (110) has the advantage of physiological signals for emotional analysis, as physiological signals are activated unintentionally and cannot be easily controlled by the subject, so they can more accurately reflect the emotional state, and by analyzing these signals, it can function to identify the user's real-time emotional state.

[0108]

[0109] The complex state measurement analysis unit (120) is configured to extract a state complex signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyze the subject's state complex emotion. This complex state measurement analysis unit (120) may be configured to include a camera and computer vision for extracting a state complex signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyze the subject's state complex emotion. Here, the complex state measurement analysis unit (120) may be provided separately from the physiological signal measurement analysis unit (110) and may function to analyze the subject's complex state based on additional visual intelligence.

[0110]

[0111] In addition, the complex state measurement analysis unit (120) can analyze the subject's emotions of joy, anger, sorrow, and pleasure through facial analysis of the subject's facial expressions for complex emotional exchange between a robot and a human. The facial expression analysis of this complex state measurement analysis unit (120) is an emotional analysis of the subject's emotions of joy, anger, sorrow, and pleasure, and analyzes the subject's facial expressions using computer vision technology. Facial expressions are one of the most commonly used mechanisms for identifying emotions, and as illustrated in FIG. 5, can function to detect emotions such as joy, sadness, anger, and surprise.

[0112]

[0113] Additionally, the complex state measurement analysis unit (120) can analyze the subject's facial expressions, gestures, and postures for complex emotional exchanges between robots and humans, thereby analyzing the subject's negative and positive emotions. The gesture and posture analysis of the complex state measurement analysis unit (120) can function to understand the user's emotions by analyzing their gestures and postures. For example, a hunched posture can indicate anxiety or stress, while a relaxed posture can be expressed as indicating positive emotions.

[0114]

[0115] The complex emotion analysis recognition unit (130) is configured to precisely analyze the change and intensity of the subject's emotion and recognize the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotion analysis signal of the complex state measurement and analysis unit (120). This complex emotion analysis recognition unit (130) can be implemented as a complex emotion analysis algorithm for precisely analyzing the change and intensity of the subject's emotion and recognizing the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotion analysis signal of the complex state measurement and analysis unit (120).

[0116]

[0117] In addition, the complex emotion analysis recognition unit (130) can function to continuously map emotional states in a three-dimensional space through a 3D emotion space model, thereby precisely identifying changes in emotions and their intensity. In addition, the complex emotion analysis recognition unit (130) can recognize emotions by combining physiological signals (EEG, ECG, ET, and GSR) and physical activities (speech and facial expression) as a multi-mode emotion recognition processing, and can function to design a hierarchical artificial intelligence model and perform emotion recognition using machine learning (ML) and deep learning (DL) technologies.

[0118]

[0119] In this way, the non-contact visual intelligence platform system (100) including a physiological signal measurement and analysis unit (110) that extracts the physiological signal of the subject for complex emotional exchange between a robot and a human and analyzes the physiological emotion of the subject, a complex state measurement and analysis unit (120) that extracts a state-based complex signal including the subject's facial expression, gesture, and posture for complex emotional exchange between a robot and a human and analyzes the state-based complex emotion of the subject, and a complex emotion analysis and recognition unit (130) that precisely analyzes the change and intensity of the subject's emotion and recognizes the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotion analysis signal of the complex state measurement and analysis unit (120), can function as a large super-dimensional-based visual intelligence platform that enables mutual emotional exchange and interaction between a worker or user and a robot through visual intelligence that enables the robot to have emotions, and through this, it can be implemented to recognize the other party's multiplexing emotion through complex dimensional data rather than fragmentary visual data.

[0120]

[0121] In addition, the non-contact visual intelligence platform system (100) of the present invention is a visual intelligence platform that analyzes bio-signals using non-contact computer vision technology and can identify complex emotional states through the same, and functions as a complex emotional mapping system through multi-scale spectrum-based multi-dimensional data analysis, and in particular, can function to analyze emotions using various physiological signals (ECG, PPG, BP, SpO2, BIA, GSR, ET), and to enable the implementation of human-robot interaction by applying continuous dimensional mapping and multi-mode emotional recognition algorithms using a 3D emotional space model.

[0122]

[0123] FIG. 7 is a diagram illustrating a flowchart of a method for utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention. As illustrated in FIG. 7, a method for utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention may be implemented including a step (S110) in which a physiological signal measurement and analysis unit extracts a physiological signal of a subject for complex emotional exchange between a robot and a human and analyzes the physiological emotion of the subject, a step (S120) in which a complex state measurement and analysis unit extracts a state-specific complex signal including the subject's facial expression, gesture, and posture for complex emotional exchange between a robot and a human and analyzes the state-specific complex emotion of the subject, and a step (S130) in which a complex emotion analysis and recognition unit precisely analyzes a change and intensity of an emotion of the subject based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit and the state-specific complex emotion analysis signal of the complex state measurement and analysis unit, and recognizes the emotion of the subject.

[0124]

[0125] In step S110, the physiological signal measurement and analysis unit (110) extracts the physiological signals of the subject for complex emotional exchange between the robot and the human and analyzes the physiological emotions of the subject. The physiological signal measurement and analysis unit (110) in step S110, as illustrated in FIG. 3, extracts the physiological signals of the subject for complex emotional exchange between the robot and the human, and the physiological signals of the subject may include one or more of electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET). Here, the physiological signal measurement and analysis unit (110) may be configured to include a camera and computer vision for extracting the physiological signals of the subject for complex emotional exchange between the robot and the human and analyzing the physiological emotions of the subject, as illustrated in FIG. 2.

[0126]

[0127] In addition, the physiological signal measurement and analysis unit (110) can function as a multi-scale spectrum-based multidimensional data analyzer capable of measuring the physiological signals of the subject in a non-contact manner and processing the visual data of the measured non-contact physiological signals. This physiological signal measurement and analysis unit (110) performs non-contact computer vision biosignal analysis, and extracts physiological signals such as the subject's electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET) in a non-contact manner by using a camera and computer vision technology. This non-contact method of extracting physiological signals can improve the user experience, increase the efficiency of data collection, and reduce privacy infringement because it does not directly attach sensors to the user's body.

[0128]

[0129] In addition, the physiological signal measurement and analysis unit (110) has the advantage of physiological signals for emotional analysis, as physiological signals are activated unintentionally and cannot be easily controlled by the subject, so they can more accurately reflect the emotional state, and by analyzing these signals, it can function to identify the user's real-time emotional state.

[0130]

[0131] In step S120, the complex state measurement analysis unit (120) extracts a state complex signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyzes the subject's state complex emotion. The complex state measurement analysis unit (120) in step S120 may include a camera and computer vision for extracting a state complex signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyzing the subject's state complex emotion. Here, the complex state measurement analysis unit (120) may be provided separately from the physiological signal measurement analysis unit (110) and may function to analyze the subject's complex state based on additional visual intelligence.

[0132]

[0133] In addition, the complex state measurement analysis unit (120) can analyze the subject's emotions of joy, anger, sorrow, and pleasure through facial analysis of the subject's facial expressions for complex emotional exchange between a robot and a human. The facial expression analysis of this complex state measurement analysis unit (120) is an emotional analysis of the subject's emotions of joy, anger, sorrow, and pleasure, and analyzes the subject's facial expressions using computer vision technology. Facial expressions are one of the most commonly used mechanisms for identifying emotions, and as illustrated in FIG. 5, can function to detect emotions such as joy, sadness, anger, and surprise.

[0134]

[0135] Additionally, the complex state measurement analysis unit (120) can analyze the subject's facial expressions, gestures, and postures for complex emotional exchanges between robots and humans, thereby analyzing the subject's negative and positive emotions. The gesture and posture analysis of the complex state measurement analysis unit (120) can function to understand the user's emotions by analyzing their gestures and postures. For example, a hunched posture can indicate anxiety or stress, while a relaxed posture can be expressed as indicating positive emotions.

[0136]

[0137] In step S130, the complex emotion analysis recognition unit (130) precisely analyzes the change and intensity of the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement analysis unit (110) and the state-based complex emotion analysis signal of the complex state measurement analysis unit (120), and recognizes the subject's emotion. The complex emotion analysis recognition unit (130) in step S130 precisely analyzes the change and intensity of the subject's emotion based on the physiological emotion analysis signal of the physiological signal measurement analysis unit (110) and the state-based complex emotion analysis signal of the complex state measurement analysis unit (120), and can be implemented as a complex emotion analysis algorithm for recognizing the subject's emotion.

[0138]

[0139] In addition, the complex emotion analysis recognition unit (130) can function to continuously map emotional states in a three-dimensional space through a 3D emotion space model, thereby precisely identifying changes in emotions and their intensity. In addition, the complex emotion analysis recognition unit (130) can recognize emotions by combining physiological signals (EEG, ECG, ET, and GSR) and physical activities (speech and facial expression) as a multi-mode emotion recognition processing, and can function to design a hierarchical artificial intelligence model and perform emotion recognition using machine learning (ML) and deep learning (DL) technologies.

[0140]

[0141] As described above, the non-contact visual intelligence platform system for complex emotional exchange between a robot and a human according to an embodiment of the present invention and the method of using the same include a physiological signal measurement and analysis unit that extracts physiological signals of a subject for complex emotional exchange between a robot and a human and analyzes the physiological emotions of the subject, a complex state measurement and analysis unit that extracts state-based complex signals including facial expressions, gestures, and postures of the subject for complex emotional exchange between a robot and a human and analyzes the state-based complex emotions of the subject, and a complex emotion analysis and recognition unit that precisely analyzes the change and intensity of the subject's emotions based on the physiological emotion analysis signal of the physiological signal measurement and analysis unit and the state-based complex emotion analysis signal of the complex state measurement and analysis unit, thereby enabling mutual emotional exchange and interaction between a worker or user and a robot through visual intelligence that enables a robot to have emotions, and in particular, enabling mutual emotional exchange and interaction between a worker or user and a robot through visual intelligence that enables a robot to have emotions, through measurement of state-based complex states of physiological signals, facial expressions, gestures, and posture analysis of the subject. By enabling complex emotional analysis, it is possible to predict complex emotions with higher accuracy through multiplexing emotions using complex dimensional data rather than fragmentary visual data, and to implement smoother human-robot interaction through high recognition performance.

[0142]

[0143] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will readily appreciate that the present invention can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single entity may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined manner.

[0144]

[0145] The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention.

Claims

1. A non-contact visual intelligence platform system (100) for complex emotional exchange between robots and humans, A physiological signal measurement and analysis unit (110) that extracts physiological signals of a subject for complex emotional exchange between a robot and a human and analyzes the subject's physiological emotions; A complex state measurement analysis unit (120) that extracts a complex state signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyzes the subject's complex state emotion; and A non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that it includes a complex emotional analysis recognition unit (130) that precisely analyzes the change and intensity of the subject's emotions based on the physiological emotional analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotional analysis signal of the complex state measurement and analysis unit (120), and recognizes the subject's emotions.

2. In the first paragraph, the physiological signal measurement and analysis unit (110) A non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that the physiological signals of the subject are extracted for complex emotional exchange between a robot and a human, and the physiological signals of the subject include one or more of electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET).

3. In the second paragraph, the physiological signal measurement and analysis unit (110) A non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized by including a camera that extracts physiological signals of the subject for complex emotional exchange between the robot and the human and analyzes the subject's physiological emotions, and a computer vision system.

4. In the third paragraph, the physiological signal measurement and analysis unit (110) A non-contact visual intelligence platform system for complex emotional exchange between robots and humans, characterized by functioning as a multi-scale spectrum-based multi-dimensional data analyzer capable of measuring physiological signals of a subject in a non-contact manner and processing visual data of the measured non-contact physiological signals.

5. In any one of the first to fourth clauses, the composite state measurement analysis unit (120) A non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized by including a camera and computer vision for extracting complex emotional signals including facial expressions, gestures, and postures of the subject for complex emotional exchange between the robot and the human, and analyzing the complex emotional states of the subject.

6. In the fifth paragraph, the composite state measurement analysis unit (120) A non-contact visual intelligence platform system for complex emotional exchange between robots and humans, characterized by analyzing the subject's emotions of joy, anger, sorrow, and pleasure through facial analysis of the subject's facial expressions for complex emotional exchange between robots and humans.

7. In the fifth paragraph, the composite state measurement analysis unit (120) A non-contact visual intelligence platform system for complex emotional exchange between robots and humans, characterized by analyzing the subject's negative and positive emotions through gesture and posture analysis of the subject's facial expressions for complex emotional exchange between robots and humans.

8. In paragraph 5, the complex emotion analysis recognition unit (130) A non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that it precisely analyzes the change and intensity of the subject's emotions based on the physiological emotional analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotional analysis signal of the complex state measurement and analysis unit (120), and is implemented with a complex emotional analysis algorithm for recognizing the subject's emotions.

9. A method of using a non-contact visual intelligence platform system (100) for complex emotional exchange between a robot and a human, (1) A step in which the physiological signal measurement and analysis unit (110) extracts the physiological signals of the subject for complex emotional exchange between a robot and a human and analyzes the physiological emotions of the subject; (2) A step in which a complex state measurement analysis unit (120) extracts a complex state signal including the subject's facial expression, gestures, and posture for complex emotional exchange between a robot and a human, and analyzes the subject's complex state emotion; and (3) A method of using a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that it includes a step of precisely analyzing the change and intensity of the subject's emotion based on the physiological emotional analysis signal of the physiological signal measurement analysis unit (110) and the state-based complex emotional analysis signal of the complex state measurement analysis unit (120) by the complex emotional analysis recognition unit (130), and recognizing the subject's emotion.

10. In the 9th paragraph, the physiological signal measurement and analysis unit (110) in the step (1) A method of utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that the physiological signals of the subject are extracted for complex emotional exchange between a robot and a human, and the physiological signals of the subject include one or more of electrocardiogram (ECG), plethysmogram (PPG), blood pressure (BP), oxygen saturation (SpO2), bioimpedance (BIA), galvanic skin conduction (GSR), and eye tracking (ET).

11. In the 10th paragraph, the physiological signal measurement and analysis unit (110) A method of utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that it comprises a camera that extracts physiological signals of the subject for complex emotional exchange between the robot and the human and analyzes the subject's physiological emotions, and a computer vision.

12. In the 11th paragraph, the physiological signal measurement and analysis unit (110) A method of utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that it functions as a multi-scale spectrum-based multi-dimensional data analyzer capable of measuring a subject's physiological signal in a non-contact manner and processing visual data of the measured non-contact physiological signal.

13. In any one of the 9th to 12th clauses, the composite state measurement analysis unit (120) in the step (2) A non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized by including a camera and computer vision for extracting complex emotional signals including facial expressions, gestures, and postures of the subject for complex emotional exchange between the robot and the human, and analyzing the complex emotional states of the subject.

14. In the 13th paragraph, the composite state measurement analysis unit (120) A method of utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized by analyzing the subject's emotions of joy, anger, sorrow, and pleasure through facial analysis of the subject's facial expressions for complex emotional exchange between a robot and a human.

15. In the 13th paragraph, the composite state measurement analysis unit (120) A method of utilizing a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized by analyzing the subject's negative and positive emotions through analysis of the subject's facial expressions, gestures, and postures for complex emotional exchange between a robot and a human.

16. In the 13th paragraph, the complex emotion analysis recognition unit (130) in the step (3) is A method of using a non-contact visual intelligence platform system for complex emotional exchange between a robot and a human, characterized in that it precisely analyzes the change and intensity of the subject's emotions based on the physiological emotional analysis signal of the physiological signal measurement and analysis unit (110) and the state-based complex emotional analysis signal of the complex state measurement and analysis unit (120), and is implemented as a complex emotional analysis algorithm for recognizing the subject's emotions.

Images