Emotion perception and empathetic interaction adjustment method and system for embodied agent
Patent Information
- Application Number
- CN202611124123.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-28
- Publication Date
- 2026-10-09
- Estimated Expiration
- 2046-07-28
AI Technical Summary
[0005]现有研究发现,上述现有技术至少存在以下缺陷:其一,面部表情与语音情绪易被主观掩饰、受遮挡与噪声干扰,且仅反映被表达出来的情绪,许多用户在不适或紧张时并不会通过表情或语言明显表露,导致机器人难以感知其真实内在状态;其二,接触式生理监测需要长期佩戴设备,对老人、儿童、患者而言佩戴困难、舒适度差,难以在日常陪护或教学中持续使用;其三,脚本化、规则化的行为无法根据用户实时生理状态动态调节语速、动作幅度、靠近距离与对话内容,无法形成感知—调节—再感知的共情闭环;其四,以任务完成或显式反馈为奖励的强化学习,其优化目标并非让用户在交互过程中生理上更放松、更舒适,且显式反馈稀疏、滞后、易被礼貌性应答掩盖;其五,单一使用毫米波雷达或单一使用摄像头 rPPG时,在用户运动、遮挡、光照变化等真实复杂环境下鲁棒性不足
(1)以毫米波雷达与摄像头 rPPG 为主、可穿戴为辅实现无需佩戴的持续生理感知,特别适合难以长期佩戴设备的陪护/教育对象,舒适度与可用性优于接触式方案;
Smart Images

Figure CN122624077B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and human-computer interaction technology, specifically to a method and system for an embodied intelligent agent to perceive a user's emotional and stress state in a non-contact manner and adjust its own embodied interactive behavior in real time based on this. In particular, it relates to a method and system for incorporating the user's physiological arousal into a reinforcement learning reward function, enabling the embodied intelligent agent to learn embodied behaviors that improve user comfort. Background Technology
[0002] With the aging population and the growing demand for personalized education, embodied intelligent agents such as companion robots and educational robots are gradually entering homes, hospitals, elderly care facilities, and educational settings. These robots not only need to complete predetermined tasks, but also need to pay attention to the user's emotions and psychological comfort during the interaction process, avoiding causing users' tension, anxiety, resistance, or fatigue due to excessively fast interaction pace, excessively large movements, too close distance, or inappropriate topics.
[0003] In existing technologies, robots perceive user emotions mainly by: facial expression-based emotion recognition, which uses cameras to capture facial images and uses expression classification models to determine discrete emotion categories; speech or language-based emotion recognition, which uses speech prosody features or semantic sentiment analysis of dialogue text to determine the user's emotional tendency; and contact-based physiological sensing for emotion / stress assessment, which uses wearable electrodes, finger-clip pulse oximeters, chest straps, and other contact sensors to collect signals such as electrocardiograms and electrodermal activity to assess autonomic nervous activity.
[0004] In terms of behavior control, the interaction behavior of existing embodied intelligent agents is mostly pre-scripted or triggered based on fixed rules; a few schemes that use reinforcement learning to optimize interaction strategies usually have reward signals from explicit indicators such as whether the task is completed, whether the user gives explicit feedback, or whether the dialogue is interrupted.
[0005] Existing research has found that the aforementioned technologies have at least the following drawbacks: First, facial expressions and vocal emotions are easily masked by subjective bias, obstructed, and subject to noise interference, and only reflect the expressed emotions. Many users do not explicitly express discomfort or tension through facial expressions or language, making it difficult for robots to perceive their true inner state. Second, contact-based physiological monitoring requires long-term wearing of devices, which is difficult and uncomfortable for the elderly, children, and patients, making it difficult to use continuously in daily care or teaching. Third, scripted and rule-based behaviors cannot dynamically adjust speech rate, movement amplitude, proximity distance, and dialogue content according to the user's real-time physiological state, failing to form an empathic closed loop of perception-regulation-reperception. Fourth, reinforcement learning that rewards task completion or explicit feedback does not aim to make users physiologically more relaxed and comfortable during interaction, and explicit feedback is sparse, delayed, and easily masked by polite responses. Fifth, when using millimeter-wave radar or a single camera rPPG, robustness is insufficient in real and complex environments such as user movement, obstruction, and changes in lighting. Summary of the Invention
[0006] To address the aforementioned problems in existing technologies, this invention provides a method and system for regulating emotion perception and empathic interaction in an embodied intelligent agent. This method and system estimate the user's emotional / stress state, particularly physiological arousal, in a non-contact manner in real time and robustly, and use this estimate as the regulating input for the robot's embodied behavior strategy. Furthermore, physiological arousal is incorporated as a component of the reinforcement learning reward function, enabling the robot to learn embodied behaviors that enhance user comfort.
[0007] To achieve the above objectives, the technical solution adopted by this invention is a method for regulating emotion perception and empathic interaction in an embodied intelligent agent, comprising: acquiring the user's physiological signals through non-contact means, wherein the non-contact means include at least measuring the user's respiratory rate and heart rate using millimeter-wave radar, and measuring the user's heart rate and / or heart rate variability using a camera with remote photoplethysmography (rPPG); constructing a real-time estimate of the user's emotional and / or stress state based on the acquired physiological signals, wherein the real-time estimate includes at least the user's physiological arousal level; using the real-time estimate as the adjustment input of the embodied intelligent agent's behavioral strategy, wherein the behavioral strategy outputs embodied behavioral parameters including at least two or more of the following: speech rate, movement amplitude, proximity distance, and dialogue content; the embodied intelligent agent executing the adjusted embodied behavior, and continuing to acquire the user's physiological signals to form a closed loop; wherein the behavioral strategy is a reinforcement learning strategy, and the physiological arousal level is used as a component of the reward function of the reinforcement learning strategy.
[0008] Furthermore, the signal quality of each modality of the acquired physiological signal is calculated in real time, and each modality is adaptively weighted and fused according to the signal quality or fused using Kalman filtering related to observation noise and signal quality, so as to maintain robust physiological estimation under conditions such as user movement, occlusion, changes in illumination, or wearable devices not being worn.
[0009] Furthermore, the user's individual physiological baseline is collected and updated online while the user is in a calm state. The physiological arousal level is calculated based on the standardized deviation of physiological characteristics relative to the individual physiological baseline, thereby achieving personalized and transferable emotion estimation across users.
[0010] Furthermore, the real-time estimation also includes emotional valence and stress index, wherein the stress index is determined by physiological arousal and emotional valence, such that high arousal and negative valence correspond to high stress, while high arousal and positive valence are not counted as high stress.
[0011] Furthermore, the reward function is a multi-objective weighted combination, including at least one of a physiological comfort reward item directly derived from physiological arousal and an optimal arousal reward item, and may include a task reward item, a safety reward item, and a behavior stability reward item; the physiological comfort reward item increases as the user's stress index decreases, and / or is penalized when the physiological arousal exceeds a set upper limit; the optimal arousal reward item takes the optimal value when the physiological arousal is within a set optimal range, and is penalized according to the deviation amount when it deviates from the range; the weights of each item are configured according to the interaction scenario.
[0012] Furthermore, the reinforcement learning strategy employs at least one of the following methods for training or updating: offline reinforcement learning from interaction logs, learning user response models and performing forward planning based on them, and constrained online adaptation under hard safety constraints, thereby avoiding high-risk exploration on vulnerable users.
[0013] The present invention also provides an embodied intelligent agent emotion perception and empathic interaction regulation system and a computer-readable storage medium for implementing the above method.
[0014] Compared with the prior art, the present invention has the following beneficial effects: (1) Using millimeter-wave radar and camera rPPG as the main means and wearable devices as the auxiliary means to achieve continuous physiological perception without wearing them, it is especially suitable for caregivers / educational subjects who have difficulty wearing devices for a long time. The comfort and usability are better than contact solutions. (2) Physiological signals are difficult to conceal subjectively, and compared with solutions that rely solely on facial expressions / voice, they can more realistically capture concealed tension, anxiety and discomfort; (3) By using signal quality assessment and adaptive fusion, the failure problem of single mode under illumination, motion, occlusion, and missing wear conditions is overcome, and it is stable and usable in real environment; (4) Introduce physiological arousal into reinforcement learning rewards, so that the robot’s optimization goal changes from completing tasks or obtaining explicit praise to making users more physiologically relaxed and comfortable, with dense, objective and real-time reward signals; (5) Simultaneously adjust the speech rate, movement range, proximity and dialogue content to form a holistic and comfortable interaction; (6) Personalized interaction is achieved based on individual baselines and online adaptation, and security and privacy are guaranteed through hard security constraints and edge de-identification processing. Attached Figure Description
[0015] Figure 1 is a block diagram of the overall structure of the system of the present invention; Figure 2 is a flowchart of the non-contact multimodal physiological perception and feature extraction of the present invention; Figure 3 is a schematic diagram of the signal quality assessment and adaptive fusion of the present invention; Figure 4 is a schematic diagram of the real-time emotion / stress estimation module of the present invention; Figure 5 is a block diagram of the reinforcement learning behavior strategy based on physiological arousal reward of the present invention; Figure 6 is a schematic diagram of the reward function structure of the present invention; Figure 7 is a flowchart of the empathic interaction closed-loop regulation of the present invention; Figure 8 is a schematic diagram of the application scenario of the present invention.
[0016] Figure labeling: 100 - Non-contact multimodal physiological perception module; 110 - Millimeter-wave radar unit; 120 - Camera unit; 130 - Wearable physiological signal access unit; 200 - Signal quality assessment and adaptive fusion module; 300 - Personalized baseline calibration module; 400 - Real-time emotion / stress estimation module; 500 - Reinforcement learning behavior decision-making module; 600 - Embodied behavior execution module; 610 - Speech synthesis unit; 620 - Action execution unit; 630 - Mobile chassis unit; 640 - Dialogue management unit; 700 - Safety constraint and supervision module; 800 - Data storage and privacy protection module. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The parameters, thresholds, network structures, and frequency bands given are all exemplary values and can be replaced according to actual solutions.
[0018] I. System Overall Structure. Referring to Figure 1, the embodied intelligent agent emotion perception and empathic interaction regulation system of the present invention includes: a non-contact multimodal physiological perception module 100, a signal quality assessment and adaptive fusion module 200, a personalized baseline calibration module 300, a real-time emotion / stress estimation module 400, a reinforcement learning behavior decision-making module 500, an embodied behavior execution module 600, a safety constraint and supervision module 700, and a data storage and privacy protection module 800. The non-contact multimodal physiological perception module 100 includes a millimeter-wave radar unit 110, a camera unit 120, and an optional wearable physiological signal access unit 130; the embodied behavior execution module 600 includes a speech synthesis unit 610, an action execution unit 620, a mobile chassis unit 630, and a dialogue management unit 640. These modules constitute a closed loop of empathic interaction: perception—estimation—decision—execution—re-perception.
[0019] II. Non-contact multimodal physiological sensing and feature extraction. Referring to Figure 2, the millimeter-wave radar unit 110 uses a frequency-modulated continuous wave (FMCW) radar (e.g., 24 GHz, 60 GHz, or 77 GHz) to transmit a linear frequency-modulated signal to the user and receive the reflected echo from the human body. A range-dimensional fast Fourier transform is performed on the echo to determine the user's location within a specific range cell. The phase signal of this range cell is extracted to obtain the micro-displacement of the thoracic cavity. Bandpass filtering is performed in the respiratory band (approximately 0.1 to 0.5 Hz) and the heart rate band (approximately 0.8 to 2.5 Hz), respectively. Separation is achieved using methods such as phase unwinding, differential cross-multiplication, or variational mode decomposition to obtain the respiratory rate (RR), heart rate (HR), and beat-by-beat interval.
[0020] Camera unit 120 acquires user facial video, performs face detection and tracking, and selects regions of interest (ROIs) such as the forehead and cheeks. For subtle changes in color channels within these ROIs over time, it uses chromaticity algorithms such as CHROM and POS, or a deep learning-based rPPG network, to extract pulse wave signals. These signals are then bandpass filtered and peaked to obtain heart rate (HR) and beat-by-beat pulse interval, used to estimate heart rate variability (HRV). An optional wearable physiological signal access unit 130 accesses signals such as PPG, electrical skin activity (EDA), skin temperature, and acceleration, with EDA directly reflecting sympathetic nerve arousal. The physiological features extracted from each channel are aligned to a unified time axis and calculated within a sliding time window (e.g., 10 to 60 seconds, with a step size of 1 to 5 seconds). The extracted features include at least heart rate-related, heart rate variability-related, and respiratory-related features, as well as optional EDA and skin temperature features.
[0021] III. Signal Quality Assessment and Adaptive Fusion. Refer to Figure 3 for each mode. Real-time calculation of signal quality The basis for this includes the signal-to-noise ratio of millimeter-wave radar, target range cell stability, and user motion; face visibility, lighting conditions, motion artifacts, and signal periodicity of rPPG; and the wearable contact status and signal-to-noise ratio. For physiological quantities measured jointly by multiple modalities, a quality-adaptive weighted fusion method is used. .in, For modality exist Signal quality at any given moment For modality exist The signal at each time point. Alternatively, a Kalman filter with quality-adaptive observation noise can be used for time-series fusion, where the modal... Observation noise variance Its quality Inversely proportional, that is This approach assigns greater weight to high-quality modalities; automatically reduces the weight or removes a modality when its quality falls below a threshold; and triggers a conservative default behavior when the quality of all modalities is insufficient. Alternatively, a deep fusion network based on an attention mechanism can be used, with the quality of each modality serving as the modulation input for the attention weights.
[0022] IV. Personalized Baseline Calibration and Mood / Stress Estimation. See Figure 4 for individual physiological baseline data collection during a user's calm state. The baseline mean of each feature is obtained. Compared with baseline standard deviation The data is updated slowly online using an exponential moving average. The fused physiological characteristics are standardized based on individual baselines. .in, For a standardized baseline, Features exist Characteristics of time, Features baseline mean and Features The baseline standard deviation. Physiological arousal. The features are weighted and combined based on sympathetic influence and normalized to [0,1] using Sigmoid: in, As a weight for physiological arousal, This is a physiological arousal bias. For the Sigmoid function, Physiological arousal level. Features that increase with increasing arousal (such as HR, RR, skin conductance level and response, LF / HF) are given a positive contribution, while parasympathetic indicators that decrease with increasing arousal (such as RMSSD, HF) are given a negative contribution, with weights... With bias Trained from labeled data or initialized from physiological priors. Emotional valence. The stress index is estimated by combining explicit cues such as facial expressions, facial movement units, and phonological prosody with physiological characteristics. The stress index is defined as: High arousal with negative valence corresponds to high stress, while high arousal with positive valence is not considered high stress. The system simultaneously outputs the trends of various quantities, such as... As an optional implementation method , and Alternatively, it can be directly provided by an end-to-end multimodal temporal network, with individual baselines as the conditional inputs to the network.
[0023] V. Reinforcement Learning Behavioral Strategies Based on Physiological Arousal Rewards. See Figure 5, where the reinforcement learning agent at time... state At least including physiological arousal valence Stress index Its changing trends and the quality of each mode The semantic embedding of the interaction context and dialogue history, the duration and time period of the interaction, and the robot's current behavioral parameters; its actions It is a set of embodied behavioral parameters that can be continuously or hierarchically adjusted, including at least the speech rate / volume / tone of speech, the amplitude / speed of movement, the spatial proximity / direction, and the topic / information density / whether the conversation is paused.
[0024] See Figure 6 reward function For multi-objective weighted combination: .in, The weighting of physiological comfort rewards. As a reward for physiological comfort, To determine the optimal weight for wake-up rewards, For optimal wake-up reward, As the weight of task rewards, As a reward for the task, Weighting of security rewards As a safety reward, The weight of the stability reward, For stability rewards. Physiological comfort rewards. Rewards reduce excessive user arousal and stress. One implementation is to reward a decrease in the stress index and penalize arousal levels exceeding the comfort limit. .in, The weighting of the stress index is reduced. For the weight of the penalty for exceeding the limit, The wakefulness threshold, This is a penalty. Optimal wake-up reward. Based on the inverted U-shaped pattern, maintain user wake-up within the optimal engagement range. Within the specified range, the optimal value is selected; values outside the range are penalized based on the deviation amount. Where, is the distance. function, This represents the lower limit of the optimal range for wakefulness. This represents the upper limit of the optimal range for wakefulness. Task Rewards Related to the established mission objectives; safety rewards Strong negative rewards are given for dangerous behaviors such as approaching too closely, moving too fast, or suddenly closing in; stable behavior is rewarded. Drastic changes in the parameters of the punishment behavior. Weights. Configuration is based on scenario, with a focus on companionship or comfort scenarios. Educational or incentive scenarios tend to be more focused and It can be further amplified when the valence is negative and arousal is increased. The weight.
[0025] Considering that those receiving care / education are often vulnerable groups such as the elderly, children, and patients, it is not advisable to conduct high-risk online explorations on real users. Therefore, the decision strategy function... The preferred learning methods are: offline reinforcement learning from recorded interaction logs; and learning a user response model. The strategy involves predicting user physiological responses to given behaviors and performing prospective planning accordingly, selecting behaviors expected to minimize discomfort; and constrained online adaptation with small steps under hard safety constraints. These strategies can be implemented using an Actor-Critic architecture or by employing value-based methods to make decisions on discretized behavioral parameters. This represents the change in physiological arousal level. This represents the change in the pressure index.
[0026] VI. Safety Constraints and Empathic Closed Loop. The safety constraint and supervision module 700, independent of the learning strategy, imposes hard constraints on the final executed behavior: the minimum approach distance is not less than a safety threshold, the action and movement speed / acceleration do not exceed upper limits, and the range of behavioral parameter changes per unit time is limited; when the perceived quality remains too low or an abnormal physiological event is detected, the robot reverts to a conservative default behavior or issues a care prompt and can notify the guardian. Referring to Figure 7, the above modules constitute a continuously operating empathic interaction closed loop: non-contact multimodal perception, quality assessment and fusion, personalized emotion / stress estimation, reinforcement learning strategy output of embodied behavioral parameters, safety constraints, embodied behavior execution, user generating new physiological responses, re-perception and reward calculation, and online strategy updates, thereby enabling the robot to continuously verify whether its behavior makes the user more comfortable and adjust subsequent behaviors accordingly.
[0027] Example 1 (Home Companion Robot). Referring to Figure 8(A), a mobile home companion robot is equipped with a 60GHz millimeter-wave radar, an RGB-D camera, a microphone array, a speaker, a head capable of facial expressions and head turning, and a movable chassis. It can optionally pair with a user's wristband. After the user sits down or lies down, the system performs a personalized baseline calibration for approximately 1 to 3 minutes. During interaction, when the system detects an increase in the user's arousal level and stress index, the reinforcement learning strategy accordingly slows down the speech rate, lowers the volume, adopts a soothing tone, reduces and slows down movements, slightly retracts the chassis to increase social distance, and switches the conversation to more relaxed topics or offers reassurance. If necessary, it pauses and asks if the user needs a rest. After the robot executes these actions, it continues to perceive the user's physiological response. If the stress index decreases accordingly, the behavior receives a positive physiological comfort reward and is reinforced. For vulnerable users, the strategy primarily uses offline reinforcement learning and user response model planning, with small-step online adaptation under hard safety constraints.
[0028] Example 2 (Educational Robot). Referring to Figure 8(B), a desktop or tutoring educational robot is equipped with millimeter-wave radar and an RGB camera. Its reward configuration emphasizes optimal wake-up rewards. With task rewards Based on the inverted U-shaped pattern, when user arousal is too low (distracted, bored), the strategy speeds up the pace, increases intonation, and introduces interactive or interesting content to improve arousal; when user arousal is too high (anxious, stumped), the strategy slows down the speech, reduces information density, breaks down steps, provides encouraging words, and appropriately increases distance, thereby maintaining arousal within the optimal engagement range and improving learning engagement and experience.
[0029] Example 3 (Look-ahead Planning Implementation). The system trains a user response model to predict changes in user arousal / stress after taking candidate actions in the current state. During decision-making, several candidate actions are prospectively evaluated, and the action with the highest expected physiological comfort reward and that meets safety constraints is selected for execution. This allows for comfort-oriented behavioral decisions without risk exploration.
[0030] Example 4 (Perception Degradation and Privacy Protection). When the camera rPPG quality remains consistently low, the system automatically increases the weight of the millimeter-wave radar and wearable channels in the fusion process; when the quality of all non-contact channels is insufficient, the strategy switches to a conservative default behavior. Physiological feature extraction and emotion estimation are preferentially completed on the device side; the original video and original physiological waveforms do not leave the device, and the de-identified features or estimation results are only saved or uploaded with user authorization.
[0031] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention. For example, millimeter-wave radar frequency bands, camera types, wearable forms, emotion estimation model forms, reinforcement learning algorithms, reward items and weights, and embodied behavior channels can all be replaced or added or removed according to the scenario, and external cues such as facial expressions and voice emotions can also be integrated as supplementary inputs.
Claims
1. A method for regulating emotion perception and empathic interaction in an embodied intelligent agent, characterized in that, Includes the following steps: Acquiring the user's physiological signals through non-contact methods, the non-contact methods including at least measuring the user's respiratory rate and heart rate using millimeter-wave radar, and measuring the user's heart rate and / or heart rate variability using a camera with remote photoplethysmography (rPPG). Based on the acquired physiological signals, a real-time estimate of the user's emotional and / or stress state is constructed. The real-time estimate includes at least the user's physiological arousal, emotional valence, and stress index. The stress index is jointly determined by the physiological arousal and the emotional valence, such that high arousal and negative valence correspond to high stress. The real-time estimation of the emotional and / or stress state is used as the adjustment input of the embodied agent's behavior strategy, and the behavior strategy outputs embodied behavior parameters, which include at least two or more of the following: speech rate, movement amplitude, proximity distance, and dialogue content. The embodied intelligent agent executes the regulated embodied behavior and continues to acquire the user's physiological signals to form a closed loop; The behavioral strategy is a reinforcement learning strategy, and the physiological arousal is used as a component of the reward function of the reinforcement learning strategy, so as to enable the embodied agent to learn embodied behaviors that reduce user discomfort and improve user comfort. The reward function includes a physiological comfort reward item, which increases as the stress index decreases, and / or a penalty is imposed when the physiological arousal level exceeds a set upper limit; The reward function also includes an optimal arousal reward item, which takes the optimal value when the physiological arousal level is within the set optimal range, and is penalized according to the deviation amount when it deviates from the range; The reward function further includes at least one of task reward, safety reward, and behavior stability reward, and the weights of each item in the reward function are configured according to the interaction scenario; wherein, the task reward is related to the predetermined task objective; the safety reward provides a strong negative reward for dangerous behaviors such as approaching too close, moving too fast, or suddenly approaching; the behavior stability reward penalizes drastic changes in the embodied behavior parameters.
2. The method according to claim 1, characterized in that, The non-contact method also includes accessing physiological signals from the user's wearable device, wherein the physiological signals from the wearable device include at least one of photoplethysmography (PPG), electrical skin activity (EDA), and skin temperature.
3. The method according to claim 1, characterized in that, The signal quality of each modality of the acquired physiological signal is calculated in real time, and each modality is adaptively weighted and fused according to the signal quality, or fused using Kalman filtering that is related to the observation noise and signal quality; when the signal quality of any modality is lower than the threshold, the modality is downweighted or eliminated.
4. The method according to claim 1, characterized in that, The user's individual physiological baseline is collected and updated online when the user is in a calm state. The physiological arousal level is calculated based on the standardized deviation of the physiological characteristics relative to the individual physiological baseline.
5. The method according to claim 1, characterized in that, The reinforcement learning strategy is trained or updated in at least one of the following ways: offline reinforcement learning from interaction logs; learning user response models and performing look-ahead planning based on them; and constrained online adaptation under hard safety constraints.
6. The method according to claim 1, characterized in that, It also includes a safety constraint step, which imposes hard constraints on the final execution approach distance, action speed, and the magnitude of change of behavioral parameters per unit time, and causes the embodied agent to revert to a conservative default behavior or issue a care prompt when the signal quality is continuously below the threshold or an abnormal physiological event is detected; and the feature extraction of the physiological signal and the real-time estimation are completed on the device side of the embodied agent, and the original video and original physiological waveform do not leave the device, only the de-identified features or estimation results are saved or uploaded.
7. A system for regulating the emotion perception and empathy interaction of an embodied intelligent agent, characterized in that, include: The non-contact multimodal physiological sensing module includes a millimeter-wave radar unit and a camera unit, as well as a wearable physiological signal access unit; and a signal quality assessment and adaptive fusion module. Personalized baseline calibration module; Real-time mood / stress estimation module; The system includes a reinforcement learning behavior decision-making module, an embodied behavior execution module, and a safety constraint and supervision module; the system is configured to perform the method as described in any one of claims 1 to 6; the embodied intelligent agent is a companion robot or an educational robot.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.
Citation Information
Patent Citations
AI digital human interaction system
CN120045069A
Interaction system based on YOLO-CNN face emotional state model and multi-modal behavior response
CN121148029A