An emotional accompanying robot based on five-dimensional morphological changes and an emotional closed-loop interaction method thereof
Patent Information
- Application Number
- CN202610671295.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-05-15
- Publication Date
- 2026-08-21
AI Technical Summary
[0006]针对现有技术中陪伴机器人缺乏触觉与物理形态共情能力、情感闭环交互不足的问题,本发明提供了一种情感陪护机器人及其情感交互方法
(1)实现了创新的“物理共情”交互体验:本发明突破了传统机器人仅靠视听交互的局限,独创了五维形态变化系统(刚度、体积、温度、形状、节律)。机器人能够模拟生命体的真实反应(如体温变化、呼吸起伏、心跳节律以及伸展拥抱),给予用户多维度的物理慰藉,极大提升了情感陪伴的心理疗愈效果。
Smart Images

Figure CN122606664A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of robotics and human-computer interaction, and more specifically, to an emotional companion robot based on five-dimensional morphological changes and its emotional closed-loop interaction method. Background Technology
[0002] With the fast pace of modern life, mental health issues such as loneliness, anxiety, and depression are becoming increasingly prominent, leading to a surge in demand for emotional support and psychological counseling. Traditional psychological interventions are limited by time, space, and human resources, while the emergence of emotional companion robots offers a new approach to addressing this problem.
[0003] However, existing emotional companionship products have the following obvious drawbacks: Lack of deep physical tactile empathy: Most companion robots on the market currently rely on "audiovisual" interactions such as voice dialogue and screen expressions. For humans, when feeling sad or anxious, physical hugs, the transmission of warmth, and the resonance of heartbeats are often more comforting than words. Most existing robots are made of rigid materials and cannot change their physical form (such as softness, temperature, and shape) according to the user's emotional dynamics.
[0004] Limited perception dimensions prevent the formation of an effective closed loop: Existing interactive dolls or robotic pets can usually only passively respond to single presses or voice commands, lacking multimodal comprehensive perception of user physiological signals (such as heart rate), implicit touch patterns, and environmental characteristics. They cannot accurately assess the user's true psychological state, and it is even more difficult to adjust the companionship strategy in real time based on the user's status feedback.
[0005] Therefore, there is an urgent need for a companion robot that can accurately identify user emotions based on multimodal perception and provide "physical empathy" through dynamic changes in touch and physical form (such as temperature, heartbeat, softness and hardness). Summary of the Invention
[0006] To address the problems of existing companion robots lacking tactile and physical empathy capabilities and insufficient emotional closed-loop interaction, this invention provides an emotional companion robot and its emotional interaction method.
[0007] First, the present invention provides an emotional companion robot based on five-dimensional morphological changes, including a robot body, a five-dimensional morphological change system, a multimodal sensing system, and a control system; The robot body adopts a square pillow-shaped structure with an elastic silicone outer skin; A five-dimensional morphological transformation system is installed inside the robot body, including: A variable stiffness subsystem is used to change the surface stiffness of different regions of the robot body. The volume change subsystem is used to change the overall or partial volume of the robot body to achieve overall expansion of the robot body, hugging action, or support height adjustment. A temperature regulation subsystem is used to change the surface temperature of different areas of the robot body; The shape transformation subsystem is used to change the external contour shape of the robot body to achieve curling or stretching shape changes; The rhythmic motion subsystem is used to generate periodic movements that mimic the heartbeat of a living organism. A multimodal sensing system is installed on the robot body to sense information related to the user's emotional state; The control system is located inside the robot body and is connected to the five-dimensional morphological transformation system and the multimodal sensing system. The control system includes an emotion state estimation engine and a morphological mapping decision model. The emotion state estimation engine estimates the user's current emotion state based on the perception data of the multimodal sensing system. The morphological mapping decision model maps the emotion state estimation results to the control parameters of each subsystem to coordinate the control of the five-dimensional physical morphological output of the five-dimensional morphological transformation system and perform physical empathy response.
[0008] Preferably, the variable stiffness subsystem includes flexible bags arranged in sections and a vacuum control device connected to each flexible bag; the flexible bags include bags arranged in the outer region and bags arranged in the central region, and the flexible bags are filled with granular filler; by using the vacuum control device to evacuate or ventilate and repressurize each flexible bag, the granular filler is switched between a blocked state and a free-flow dynamic state, so as to independently adjust the stiffness of each region of the robot body.
[0009] Preferably, the volume change subsystem includes a miniature air pump, a solenoid valve assembly, a pressure limiting valve, and multiple internal airbags; Internal airbags are located in the center, sides, and bottom of the robot body; the central internal airbag is used to achieve overall inflation; the internal airbags on both sides serve as arm-like structures of the robot body to achieve hugging movements; and the internal airbag at the bottom is used to adjust the support height of the robot body. A miniature air pump inflates and deflates each internal airbag, a solenoid valve assembly independently controls the inflation and deflation pathways of each internal airbag, and a pressure relief valve automatically releases pressure when the internal airbag pressure exceeds a set value.
[0010] Preferably, the shape deformation subsystem includes multiple transverse drive wires arranged along the length direction of the robot body and multiple longitudinal drive wires arranged along the width direction of the robot body; the drive wires are arranged at the edge of the robot body and together form a ring-shaped drive structure around the robot body; the drive wires achieve longitudinal or transverse deformation of the robot body by shrinking through heating.
[0011] Preferably, the emotional state estimation engine uses a hybrid method combining a valence-arousal two-dimensional continuous space model with discrete emotion classification to estimate the user's emotional state, and predicts the evolution direction of the emotional state based on the valence-arousal trajectory of multiple consecutive frames. The multimodal fusion model is implemented using deep learning or machine learning methods.
[0012] Preferably, the method for estimating the user's emotional state is as follows: based on the distance between the user's current valence-arousal coordinate value and the center point of each preset emotion category, calculate the matching score of each emotion category, and determine the emotion category with the highest matching score as the user's current emotional state.
[0013] Preferably, the morphological mapping decision model includes basic mapping rules and an online optimization strategy; the basic mapping rules define the correspondence between emotional state categories and control parameters of each subsystem of the five-dimensional morphological change system; the online optimization strategy adaptively adjusts the control parameters according to the changes in the user's emotional state after physical empathy response.
[0014] Furthermore, this invention also provides an emotional closed-loop interaction method based on the aforementioned emotional companion robot, comprising the following steps: S1: Real-time acquisition of interaction and environmental data through a multimodal sensing system; control system preprocesses the acquired raw data. S2: Based on the data collected in S1, the current emotional state is estimated through the emotional state estimation engine based on the multimodal fusion model and the valence-arousal two-dimensional continuous space model, and discrete classification and evolution prediction are performed. S3: Based on the emotional state estimation results, control parameters of each subsystem in the five-dimensional morphological change system are generated through the preset basic mapping rules in the morphological mapping decision model. S4: The control system coordinates and drives the five-dimensional morphological change system to perform physical morphological changes and provide a physical empathic response; S5: Continuously monitor changes in the user's emotional state. If a change in emotional state is detected, regenerate the control parameters of each subsystem in the five-dimensional morphological change system. If the emotional state remains unchanged, increase the response intensity of each subsystem in the five-dimensional morphological change system to form a closed-loop support system. Use the emotional state change as a reward signal to optimize the control parameters of the morphological mapping decision model online. Compared with existing technologies, this invention has the following advantages: (1) Achieved an innovative "physical empathy" interactive experience: This invention breaks through the limitations of traditional robots that rely solely on audiovisual interaction and creates a unique five-dimensional morphological change system (stiffness, volume, temperature, shape, and rhythm). The robot can simulate the real reactions of living organisms (such as changes in body temperature, breathing fluctuations, heart rhythm, and stretching and hugging), providing users with multi-dimensional physical comfort and greatly enhancing the psychological healing effect of emotional companionship.
[0015] (2) A high-precision multimodal emotion perception network was constructed: combining flexible pressure sensing, voice, photoplethysmography pulse wave and inertial measurement, it can not only understand the user's voice emotion, but also perceive whether the user is "gently stroking" or "violently hitting", and even read the user's heart rate physiological indicators, making the emotional state estimation more objective and accurate.
[0016] (3) An adaptive closed-loop companionship strategy was implemented: A morphological mapping decision model and an online optimization strategy were introduced. The robot can not only change the current emotional output morphology, but also monitor the user's emotional trend after the change. If the user's negative emotions are relieved, the system will reinforce this strategy as a positive reward, so that the robot understands the user better and better with use, and achieves highly personalized emotional companionship.
[0017] (4) Improved safety and user-friendliness: The pillow-shaped design combined with a high-elasticity silicone outer skin and a replaceable fleece fabric cover ensures an extremely soft touch when users hug or press it. At the same time, the cover is removable and washable, meeting the hygiene needs of daily use. Attached Figure Description
[0018] Figure 1 This is a block diagram of the overall system architecture of the emotional companion robot provided in an embodiment of the present invention; Figure 2 A cross-sectional schematic diagram of the five-dimensional morphological transformation system of the emotional companion robot provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the multimodal sensor system of the emotional companion robot provided in an embodiment of the present invention; Figure 4 A flowchart of the emotional interaction method provided in the embodiments of the present invention; Figure 5 This is a logical relationship diagram between the emotion state estimation engine and the morphological mapping decision model in an embodiment of the present invention.
[0019] The annotations in the attached figures are explained as follows: 100-Emotional Companion Robot; 110 - Robot body; 111 - High-elasticity silicone outer skin; 112 - Fleece fabric outer cover; 200-Five-Dimensional Morphological Transformation System; 210 - Variable stiffness subsystem; 211 - Flexible bag body; 212 - Vacuum control device; 213 - Granular filler; 220 - Volume change subsystem; 221 - Internal airbag; 222 - Miniature air pump; 223 - Solenoid valve assembly; 224 - Pressure relief valve; 230 - Temperature regulation subsystem; 231 - Flexible heating film; 232 - Cooling element; 240 - Shape transformation subsystem; 241 - Shape memory alloy drive unit; 250 - Rhythmic Motion Subsystem; 251 - Heartbeat Simulation Unit; 300 - Multimodal sensing system; 310 - Tactile sensing module; 320 - Voice sensing module; 330 - Physiological sensing module; 340 - Motion sensing module; 350 - Environmental sensing module; 400 - Control system; 410 - Emotional state estimation engine; 420 - Morphological mapping decision model. Detailed Implementation
[0020] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.
[0021] I. Overall Structure of the Robot An embodiment of the present invention discloses an emotional companion robot 100 based on five-dimensional morphological changes, including a robot body 110, a five-dimensional morphological change system 200, a multimodal sensing system 300, and a control system 400.
[0022] The robot body 110 adopts a square pillow-shaped structure, with an overall rounded rectangular soft pillow structure. In this embodiment, its dimensions are approximately 45cm long × 35cm wide × 20cm thick, making it easy for users to hug and cuddle. The main structural layer of the robot body 110 is made of a high-elasticity silicone outer skin 111. This high-elasticity silicone outer skin 111 is made of food-grade silicone material with a Shore hardness of 10A-20A, which has excellent elastic recovery, biocompatibility, and durability, while providing a smooth force transmission interface for the deformation of the internal shape change subsystem 240.
[0023] The high-elasticity silicone outer skin 111 is fitted with a replaceable fleece fabric cover 112. The fleece fabric cover 112 is made of short-pile or coral fleece fabric and is detachably connected to the high-elasticity silicone outer skin 111 via a zipper or magnetic closure, facilitating cleaning, replacement, and customization. The fleece fabric cover 112 provides a warm and soft touch without affecting the signal acquisition of the internal sensing system and the deformation output of the drive system within the robot body 110.
[0024] The robot body 110 has an internal support frame, which is a mesh structure made of flexible material (such as thermoplastic polyurethane) to fix and support the internal functional modules while maintaining the flexibility of the overall structure.
[0025] II. Five-Dimensional Morphological Change System Reference Figure 2 The five-dimensional shape transformation system 200 is disposed inside the robot body 110 and includes a variable stiffness subsystem 210, a volume transformation subsystem 220, a temperature regulation subsystem 230, a shape transformation subsystem 240, and a rhythmic motion subsystem 250. The five subsystems correspond to the five physical dimensions of stiffness perception, volume / pressure perception, temperature perception, shape perception, and rhythm perception in tactile interaction, and achieve multi-dimensional physical empathy through coordinated control.
[0026] (1) Variable stiffness subsystem 210 The variable stiffness subsystem 210 uses the particle clogging principle to achieve variable surface stiffness of the robot body 110. Specifically, the variable stiffness subsystem 210 includes multiple partitioned flexible bags 211, vacuum control devices 212 connected to each flexible bag 211, and each flexible bag 211 is filled with particulate filler 213.
[0027] The flexible bag 211 is made of elastic silicone film or thermoplastic polyurethane film, and is flat and sac-shaped with a thickness of 10mm-30mm. In this embodiment, the surface of the robot body 110 is divided into six stiffness control areas, including a top area, a bottom area, a left side area, a right side area, a front end area, and a rear end area, with an independent flexible bag 211 set in each area.
[0028] The particulate filler 213 consists of spherical or irregularly shaped particles with a particle size of 1mm-5mm, and the material is selected from coffee grounds, polystyrene microspheres, or glass microspheres, etc. The filling rate of the particulate filler 213 is 60%-80% of the volume of the flexible bag 211.
[0029] The vacuum control device 212 includes a miniature vacuum pump and a proportional solenoid valve, which are connected to each flexible bag 211 via flexible tubing. When the vacuum control device 212 evacuates the flexible bag 211, the external atmospheric pressure causes the flexible bag 211 to contract and compress the internal granular filler 213, generating friction and geometric locking effects between the particles (i.e., a blockage state), which significantly increases the surface stiffness of the corresponding area, resulting in a harder feel. When the vacuum control device 212 ventilates the flexible bag 211 to return to normal pressure, the constraints between the granular filler 213 are released, restoring a free-flowing state, and the corresponding area becomes soft and deformable.
[0030] By precisely controlling the vacuum level (negative pressure value) of each flexible bag 211, the stiffness can be continuously adjusted, with a vacuum level range of 0 kPa to -90 kPa, corresponding to a surface stiffness variation range of 0.1 N / mm to 5 N / mm. Each stiffness control area can be controlled independently to achieve differentiated stiffness distribution. For example, when a soft hugging sensation is needed, the vacuum in all areas is released to soften the overall structure; when a sense of support and boundary is needed, the vacuum in some flexible bags 211 is selectively evacuated to harden the corresponding areas of the robot body 110.
[0031] (2) Volume change subsystem 220 The volume change subsystem 220 includes a micro air pump 222, a solenoid valve group 223, a pressure limiting valve 224, and multiple internal airbags 221.
[0032] The internal airbags 221 are made of thermoplastic polyurethane film material and are located at different positions inside the robot body 110. In this embodiment, there are four internal airbags 221, namely a central main airbag, a left wing airbag, a right wing airbag, and a bottom airbag. The central main airbag is the largest volume airbag and is located in the center of the robot body 110. The volume of the central main airbag can increase by about 40% after inflation, which is used to achieve an overall expansion effect. The left and right wing airbags are located on the left and right sides of the robot body 110, forming the arm-like structure of the robot body 110. The left and right wing airbags are used to cooperate with the shape change subsystem 240 to realize the deployment and wrapping action of the arm-like structure. The bottom airbag is used to adjust the height and support posture of the robot body 110.
[0033] The miniature air pump 222 is a brushless DC miniature diaphragm pump with a maximum flow rate of 3L / min, a maximum pressure of 35kPa, and an operating noise level below 40dB, ensuring minimal noise interference in quiet environments. Two miniature air pumps 222 are used: one for inflation and one for rapid deflation, to meet the needs of rapid expansion and contraction.
[0034] The solenoid valve assembly 223 includes normally closed solenoid valves corresponding to each internal airbag 221, used to independently control the inflation and deflation passages of each airbag. The solenoid valve assembly 223 uses miniature low-power solenoid valves with a response time of less than 20ms.
[0035] The pressure relief valve 224 is installed in the air circuit. When the system pressure exceeds the set opening pressure value, it automatically releases pressure to prevent the internal airbag 221 from rupturing due to overcharging or to generate excessive pressure on the user, thus ensuring safe use. In this embodiment, the opening pressure of the pressure relief valve 224 is set to 30 kPa.
[0036] By controlling the working state of the micro air pump 222 and the opening and closing combination of the solenoid valve group 223, the following volume change modes can be achieved: slow expansion mode (simulating deep breathing, with a cycle of 3-6 seconds), fast expansion mode (simulating the force effect of a tight hug, with a response time of about 0.5 seconds), and slow contraction mode (smooth release).
[0037] (3) Temperature regulation subsystem 230 The temperature regulation subsystem 230 includes a flexible heating film 231 and a cooling plate 232 distributed on the inner surface of the robot body 110.
[0038] The flexible heating film 231 is made of polyimide electrothermal film or graphene flexible heating film, with a thickness of 0.1mm-0.3mm. It has excellent flexibility and can be bent according to the deformation of the robot body 110 without affecting the overall flexibility. The flexible heating film 231 is distributed on the side of the robot body 110 facing the user and the left and right sides. The surface facing the user is the main heating area, and its heating area accounts for about half of the total area of the robot body 110 facing the user. The left and right sides are auxiliary heating areas, and their heating areas are adapted to the contact area of an adult's palm. The zoned distribution of the flexible heating film 231 divides the inner surface of the robot body 110 into three independent heating areas. Each heating area has a power of 3W-8W and can raise the surface temperature of the corresponding area from room temperature to 38℃-42℃ within 30 seconds, simulating the warm touch of human body temperature.
[0039] The cooling chip 232 is a miniature semiconductor cooling chip, located in specific areas of the robot body 110, mainly distributed on the left and right sides and the side facing away from the user. This partitioned distribution divides the inner surface of the robot body 110 into three independent cooling zones. The side facing away from the user is the primary cooling zone, with a cooling area approximately half the total area of that side. The left and right sides are auxiliary cooling zones, with cooling areas adapted to the contact area of an adult's palm. Each cooling zone can reduce the surface temperature of its corresponding area by 5°C-10°C within 60 seconds, providing a cool touch. The robot body 110 also contains a flexible thermally conductive silicone pad and an internal heat dissipation structure. The cold side of the cooling chip 232 faces the inner surface of the robot body 110, while the hot side of the cooling chip 232 is connected to the internal heat dissipation structure via the flexible thermally conductive silicone pad.
[0040] Therefore, the temperature regulation subsystem 230 divides the inner surface of the robot body 110 into six independent temperature control zones. Each zone is equipped with an NTC thermistor temperature sensor for closed-loop temperature control, with a control accuracy of ±0.5℃. Each zone can be independently set with a target temperature to achieve differentiated temperature distribution. For example, when soothing sadness, the area in contact with the user's arm can be heated to 37℃-39℃ to simulate the experience of being warmly hugged; when relieving anxiety or anger, the area in contact with the user's arm can be cooled to 22℃-30℃ to provide a calming tactile feedback.
[0041] In terms of safety, the temperature regulation subsystem 230 is equipped with a dual over-temperature protection mechanism: when the temperature of any area exceeds 45°C, the control system 400 immediately cuts off the heating power supply to the corresponding area and triggers an alarm; when the temperature is below 15°C, it cuts off the cooling power supply to the corresponding area.
[0042] (4) Shape change subsystem 240 The shape change subsystem 240 includes a shape memory alloy driving unit 241.
[0043] The shape memory alloy drive unit 241 is constructed from nickel-titanium alloy (NiTi) wires or springs. In this embodiment, the shape memory alloy drive unit 241 includes two sets of transverse drive wires arranged along the length direction of the robot body 110 and two sets of longitudinal drive wires arranged along the width direction, to form a ring-shaped drive structure surrounding the robot body 110.
[0044] When the lateral drive wire is heated and contracts, it causes the robot body 110 to bend or curl up along its length, achieving a large deformation from a flattened state to a curled state. The bending angle range is 0°-120°, used to simulate the action of curling up and wrapping. When the longitudinal drive wire is heated and contracts, it causes the robot body 110 to contract or bend in the width direction. In conjunction with the inflation of the left and right wing airbags, the two wings of the robot body 110 bend inward, forming an arm-like hugging action, simulating the feeling of being hugged.
[0045] In this embodiment, the diameter of each driving wire is 0.3mm-1.0mm, the length of the transverse driving wire is 35-40cm, the length of the longitudinal driving wire is 30-35cm, the phase transition temperature is 60℃-70℃, and the maximum recoverable strain is 4%-6%. The deformation can be precisely controlled by adjusting the magnitude and duration of the energizing current. The shape memory alloy driving unit 241 has a relatively slow deformation speed, requiring 5-15 seconds to complete a full curling or hugging motion. This slow deformation characteristic is well-suited to the gentle and gradual interaction rhythm in emotional companionship scenarios.
[0046] The robot body 110 is equipped with an elastic silicone structure and an elastic bias spring. When no power is applied, the shape memory alloy drive unit 241 is provided with a restoring force by the elastic silicone structure and the elastic bias spring, so that the robot body 110 returns to its initial shape.
[0047] (5) Rhythmic Motion Subsystem 250 The rhythmic motion subsystem 250 includes a heartbeat simulation unit 251.
[0048] The heartbeat simulation unit 251 employs an eccentric rotor vibration motor (ERM motor) or a linear vibration motor (LRA motor). In this embodiment, the heartbeat simulation unit 251 is positioned slightly to the left of the center of the robot body 110, simulating the approximate location of the human heart. The heartbeat simulation unit 251 consists of an eccentric brushed DC vibration motor, driven by the control system 400 according to a specific timing pattern, generating a "thump-thump" double-beat rhythmic vibration sensation to simulate the feeling of a real heartbeat.
[0049] The heartbeat simulation unit 251 can generate a heartbeat frequency ranging from 40 to 120 beats per minute, with adjustable amplitude. The control system 400 automatically adjusts the heartbeat frequency and amplitude based on the emotional state estimation results: for example, when soothing sadness and anxiety, it outputs a gentle heartbeat at a frequency of 55-65 beats per minute to guide the user to relax; in motivational and arousal scenarios, the frequency can be appropriately increased to 80-100 beats per minute.
[0050] III. Multimodal Sensing System Reference Figure 3 The multimodal sensing system 300 is mounted on the robot body 110 and includes a tactile sensing module 310, a voice sensing module 320, a physiological sensing module 330, a motion sensing module 340, and an environmental sensing module 350.
[0051] (1) Tactile sensing module 310 The tactile sensing module 310 includes a flexible pressure sensing array. This array utilizes FSR flexible resistive thin-film pressure sensors, arranged in a matrix on the inner surface of the highly elastic silicone outer skin 111. In this embodiment, the flexible pressure sensing array contains approximately 64 sensing nodes, forming a sensing area of approximately 240 mm × 240 mm, resulting in a resolution of approximately 30 mm × 30 mm, covering the main surface area of the robot body 110.
[0052] The tactile sensing module 310 can perceive the following characteristics of user touch interactions: contact location, contact area, pressure, touch duration, and spatiotemporal dynamic patterns of the touch. Through analysis and pattern recognition of these characteristics, the tactile sensing module 310 can distinguish the following touch interaction methods: stroking (low pressure, large area, moving contact), pressing (medium pressure, small area, static contact), rubbing (medium pressure, dynamic alternating contact), patting (high pressure, short duration, impact contact), hugging (large area, medium pressure, continuous enveloping contact), and gripping (high pressure, continuous concentrated contact), etc. Different touch interaction methods correspond to different emotional expression intentions of users, providing important clues for emotional state estimation and laying a data foundation for the deepening and expansion of related research.
[0053] (2) Voice perception module 320 The voice perception module 320 includes a microphone array. In this embodiment, the microphone array consists of four MEMS digital microphones distributed at the four corners of the robot body 110 facing the user, and has beamforming and sound source direction-oriented capabilities.
[0054] The speech perception module 320 is used to collect the user's speech signal, including speech content, intonation, speech rate, volume, and emotional prosodic features in the speech. The control system 400 performs speech emotion recognition on the collected speech signal, extracting acoustic features such as Mel-frequency cepstral coefficients (MFCC), fundamental frequency (F0), and energy envelope, and classifying emotions based on a pre-trained speech emotion recognition model. The speech perception module 320 also has a keyword detection function, which can identify keywords expressing specific emotions.
[0055] (3) Physiological perception module 330 The physiological sensing module 330 includes a photoplethysmography (PPG) sensor. In this embodiment, the PPG sensor is embedded in a specific contact area on the surface of the robot body 110 (such as a position frequently touched by the user's palm), and the PPG sensor includes a green LED light source and a photodetector.
[0056] When a user's palm or fingers come into contact with the PPG sensor, the following physiological parameters can be measured non-invasively: heart rate (HR), heart rate variability (HRV), and pulse waveform characteristics. HR and HRV are important physiological indicators reflecting the activity state of the autonomic nervous system and are closely related to emotional state. For example, when the sympathetic nervous system is excited (such as in states of anxiety or anger), HR increases and HRV decreases; when the parasympathetic nervous system is active (such as in states of relaxation or calm), HR decreases and HRV increases.
[0057] (4) Motion sensing module 340 The motion sensing module 340 includes an inertial measurement unit (IMU), which consists of a three-axis accelerometer and a three-axis gyroscope, and is located at the center of the robot body 110.
[0058] The motion sensing module 340 is used to detect the user's motion operations on the robot body 110, including: shaking (periodic acceleration changes), flipping (attitude angle changes), throwing (instantaneous high acceleration followed by weightlessness), hugging and shaking (low-frequency, small-amplitude compound movements), etc. Different motion operations reflect different emotional and behavioral intentions of the user: for example, gentle hugging and shaking usually express intimacy and dependence, while violent shaking or throwing may express anger or irritability.
[0059] (5) Environmental perception module 350 The environmental perception module 350 is disposed on the surface of the robot body 110 and is used to acquire environmental information. The environmental perception module 350 includes: an ambient light sensor for detecting ambient light intensity and spectral characteristics (such as sunlight / artificial light / dark light); and a temperature and humidity sensor for detecting ambient temperature and humidity.
[0060] Environmental information provides contextual support for estimating emotional states. For example, a user hugging the robot body 110 in low-light conditions at night is more likely to be in a state of loneliness or insomnia; a user may be more irritable in high-temperature environments. Environmental information can also be used to adjust the reference temperature of the temperature regulation subsystem 230, making the surface temperature output more reasonable.
[0061] IV. Control System The control system 400 is located inside the robot body 110 and is connected to the five-dimensional morphological change system 200 and the multimodal sensing system 300. The control system 400 includes a main controller, an emotion state estimation engine 410, a morphological mapping decision model 420, a drive control interface, and a communication interface.
[0062] The main controller uses a low-power embedded processor (such as ARM Cortex-M7 or Cortex-A series) and is responsible for the coordination, management, data processing and real-time control of each module.
[0063] (1) Emotional State Estimation Engine 410 Reference Figure 5 The emotional state estimation engine 410 uses a hybrid method combining a valence-arousal two-dimensional continuous space model with discrete emotion classification to estimate the user's emotional state.
[0064] The workflow of the emotion state estimation engine 410 is as follows: First, the data from each channel acquired by the multimodal sensing system 300 are preprocessed and feature extracted. Specifically: touch pattern feature vectors (including contact area, average pressure, pressure gradient, dynamic rate of change, etc.) are extracted from the tactile sensing module 310; acoustic emotion feature vectors (including Mel-frequency cepstral (MFCC) coefficients, fundamental frequency statistics, energy envelope features, etc.) are extracted from the voice sensing module 320; physiological feature vectors (including mean HR, HRV frequency domain and time domain features, etc.) are extracted from the physiological sensing module 330; motion feature vectors (including root mean square acceleration, angular velocity, motion period, etc.) are extracted from the motion sensing module 340; and environmental feature vectors (including light intensity, ambient temperature, etc.) are extracted from the environmental sensing module 350.
[0065] Then, the feature vectors of each channel are input into a multimodal fusion model for feature-level or decision-level fusion, and the coordinates of the user's current emotional state in the valence-arousal two-dimensional space are output. Among them, valence Used to characterize the positive or negative degree of a user's emotions, arousal level Valence is used to characterize the degree of activation of a user's emotions. and wakefulness Normalizable to The interval, in which Indicates strong negative emotions. It expresses a strong positive emotion; Indicates a low wake-up state. This indicates a high arousal state. Multimodal fusion models can be implemented using deep learning methods (such as multi-head attention mechanism fusion networks) or classical machine learning methods (such as Bayesian fusion decision).
[0066] Next, discrete emotion classification is performed based on the valence-arousal coordinate values. A corresponding emotion centroid and matching scoring function are set for each emotion category, and the user's current valence-arousal coordinate values are used as the basis for the classification. The matching score for each emotion category is calculated based on the distance between each emotion center point and the emotional center point. Specifically, in this embodiment, the valence-arousal space is divided into the following emotion category regions: Sadness: Low valence, low arousal, coordinates (-0.7, -0.6) Anxiety: Low valence, high arousal, coordinates (-0.6, 0.6) Anger: Low valence, high arousal, with coordinates (-0.8, 0.8). Fear: Low valence, medium to high arousal, coordinates (-0.7, 0.5) Calm: Neutral valence, low arousal, coordinates (0, -0.6) Satisfies: high efficiency, low wake-up rate, with coordinates (0.6, -0.4). Joy: High efficacy, medium to high wake-up value, coordinates (0.7, 0.5) Excitement: High valence, high arousal, coordinates (0.8, 0.8).
[0067] The above emotional category classification is only one optional implementation method. In practical applications, emotional categories can be added or deleted according to specific needs, and the classification range of coordinate values can be adjusted accordingly. This invention does not impose specific limitations on this.
[0068] In one alternative implementation, let the first... The emotional center of this type of emotion is Then the user's current emotional state With the The distance between different types of emotions Represented as: Corresponding matching score The Gaussian matching score can be expressed using the following formula: in, For the first The region width parameter corresponding to an emotion category is used to characterize the coverage of that emotion category in the valence-arousal space. The system selects the emotion category with the highest matching score as the user's current emotion category (i.e., the user's emotional state) and generates a corresponding soothing control strategy based on the emotion category.
[0069] Finally, the emotional state estimation engine 410 also predicts the direction of emotional state evolution based on the valence-arousal trajectory of consecutive multi-frames: by analyzing the time series change trend of (V, A) coordinates, it predicts the direction of user emotion evolution in the near future (such as from anxiety to calm, or from deepening sadness), providing feedforward information for the morphological mapping decision model 420, making physical empathy response more predictable and guiding.
[0070] (2) Morphological mapping decision model 420 The morphological mapping decision model 420 includes basic mapping rules and online optimization strategies, which are used to map the emotional state estimation results into control parameters of each subsystem of the five-dimensional morphological change system 200.
[0071] The basic mapping rules define the default correspondence between each emotional state category and the control parameters of each subsystem of the five-dimensional morphological change system 200. In this embodiment, the basic mapping rules are defined as follows: ① Strategies for coordinating and controlling grief: Variable stiffness subsystem 210: All areas are ventilated and repressurized, so that the entire surface is in the lowest stiffness state (soft), providing a gentle enveloping feeling; Volume change subsystem 220: The central main airbag, left wing airbag and right wing airbag slowly inflate, increasing the volume by about 20%-30%, causing the robot body 110 to expand and provide a full and cuddly feeling. Temperature regulation subsystem 230: The entire surface heating area is heated to 37℃-39℃, simulating the warm body temperature of the human body; Shape transformation subsystem 240: Drives the drive wire to contract, and in conjunction with the inflation of the left and right wing airbags, causes the robot body 110 to bend inward on both sides to form an embracing motion, providing a feeling of being tightly hugged and wrapped. Rhythmic movement subsystem 250: Heartbeat simulation unit 251 outputs a gentle, low-amplitude heartbeat rhythm at a frequency of 55-65 beats / minute, conveying a sense of security and companionship.
[0072] ② Strategies for coordinating and controlling anxiety: Variable stiffness subsystem 210: The flexible bag 211 in the top, bottom, left and right regions is vacuumed to keep the outer region of the robot body 110 with moderate stiffness, providing a clear sense of tactile boundaries and presence, and helping anxious users feel certainty. Volume change subsystem 220: Maintains the current volume or slightly inflates, simulating a stable breathing rhythm (frequency 5-7 times / minute) through slow, periodic, micro-amplitude expansion and contraction, guiding the user to follow the breathing rhythm; Temperature regulation subsystem 230: Surface temperature is regulated to 28℃-30℃ (slightly cool body temperature), providing a cool and calming touch; Shape transformation subsystem 240: Drives the lateral drive wire to contract moderately, causing the robot body 110 to curl up slightly, forming a compact body shape and providing a safe sense of enclosure; Rhythmic movement subsystem 250: Heart rate simulation unit 251 outputs a slow and stable heart rate rhythm at a low frequency of 45-55 beats / minute to guide the user to lower their heart rate.
[0073] ③ Strategies for coordinating and controlling anger: Variable stiffness subsystem 210: The flexible bags 211 in the top, bottom, left and right areas are ventilated and pressurized, while the flexible bags 211 in the front and rear areas are vacuumed, so that the outer area of the robot body 110 remains soft (allowing users to knead to release pressure), while the inner or core area remains relatively hard (providing resistance feedback so that users feel a certain support). Volume change subsystem 220: Each airbag deflates to a moderately contracted volume of about 10%-15%, so that the robot body 110 is slightly tightened and not over-expanded to avoid increasing the feeling of irritability. Temperature regulation subsystem 230: The surface temperature of the entire surface cooling area is reduced to 22℃-25℃, providing a cool touch and helping to cool down anger; Shape change subsystem 240: Avoids large deformations to prevent overstimulation; Rhythmic movement subsystem 250: Heartbeat simulation unit 251 outputs heartbeat rhythm at a stable frequency of 50-60 beats / minute with moderate amplitude, guiding the user to gradually calm down from a high arousal state.
[0074] ④ Strategies for coordinating and controlling calm / satisfied emotions: Variable stiffness subsystem 210: Each region maintains a comfortable state with medium to low stiffness; Volume change subsystem 220: Maintains natural volume by outputting low-frequency micro-breathing simulation through slow, periodic micro-expansion and contraction; Temperature regulation subsystem 230: Maintains the surface temperature at a comfortable range of 32℃-35℃; Shape transformation subsystem 240: Maintains the natural unfolded shape; Rhythmic movement subsystem 250: Heart rate simulation unit 251 operates gently at a normal frequency of 60-70 beats / minute to maintain a sense of presence.
[0075] The online optimization strategy is used to adaptively adjust control parameters based on the basic mapping rules, taking into account individual user differences and real-time feedback. Specifically, after executing a physical empathy response, the control system 400 continuously monitors changes in the user's emotional state through the multimodal sensing system 300. If the user's emotional state is detected to be shifting in a positive direction (increased effectiveness or arousal moving towards the target direction), the current combination of control parameters is marked as positively effective, and the weight of this parameter combination is increased in subsequent similar scenarios. If the user's emotional state is detected to be not significantly improved or to be deteriorating in a negative direction, the control parameters are adjusted (e.g., increasing the temperature offset, adjusting the stiffness value, changing the heart rate, etc.) to find a more effective combination.
[0076] The online optimization strategy can be implemented using a reinforcement learning framework. Changes in emotional state (especially positive changes in efficacy value) serve as reward signals, the control parameter combinations of the five-dimensional morphological change system 200 form the action space, and the current emotional state forms the state space. The optimization strategy network is accumulated through continuous interaction. Furthermore, the online optimization strategy can learn the personalized preferences of different users; for example, some users are more sensitive to temperature changes, while others are more sensitive to stiffness changes, and the system can automatically adjust the response weights for each dimension.
[0077] V. Power Supply and Safety System The emotional companion robot 100 also includes a power management module and a safety protection system (not shown in detail in the figure). The power management module is powered by a rechargeable lithium polymer battery with a capacity of 5000mAh-10000mAh, supporting Type-C interface charging and wireless charging. The safety protection system includes: temperature over-limit protection (as mentioned above, the temperature regulation subsystem 230 has a dual over-temperature protection mechanism), pressure over-limit protection (achieved through the pressure limiting valve 224), battery overcharge / over-discharge protection, and drop detection protection (when a drop event is detected by the IMU of the motion sensing module 340, the control system 400 enters the safety protection mode and cuts off or restricts the driving of high-power actuators such as the flexible heating film 231, the cooling chip 232, the vibration motor, the micro air pump 222, and the solenoid valve group 223).
[0078] VI. Specific Implementation of Emotional Interaction Methods Reference Figure 4 The present invention provides an emotional closed-loop interaction method based on an emotional companion robot 100, the specific implementation steps of which are as follows: S1: Multimodal data acquisition When the emotional companion robot 100 is in standby mode, the tactile sensing module 310 continues to operate with low power consumption. When the tactile sensing module 310 detects that the user touches or picks up the robot body 110 (triggered by pressure change and motion detection), the system wakes up and enters working mode.
[0079] Subsequently, the multimodal sensing system 300 is fully activated, acquiring the following data in real time: the tactile sensing module 310 acquires pressure sensor array data at a sampling rate of 10 Hz to 100 Hz to obtain tactile features; the voice sensing module 320 acquires audio data at a sampling rate of 8 kHz to 48 kHz to obtain voice features; the physiological sensing module 330 acquires PPG signals at a sampling rate of 25 Hz to 200 Hz to obtain physiological features; the motion sensing module 340 acquires acceleration and angular velocity data at a sampling rate of 50 Hz to 200 Hz to obtain motion features; and the environmental sensing module 350 acquires environmental data at a sampling rate of 0.1 Hz to 10 Hz to obtain environmental features.
[0080] The control system 400 preprocesses the raw data from each channel, including filtering and denoising, signal segmentation, and feature extraction. The preprocessing window is 2-5 seconds, and the feature extraction results are updated every 1-2 seconds.
[0081] S2: Emotional State Estimation and Prediction The sentiment state estimation engine 410 receives preprocessed multimodal feature data and performs the following processing: Multimodal fusion: It integrates tactile features, speech features, physiological features, motion features and environmental features to output the coordinates (V, A) of the user's current emotional state in the valence-arousal space.
[0082] Discrete classification: based on the user's current valence-arousal coordinate value. The distance between the user and the center point of each preset emotion category is used to calculate the emotion matching score corresponding to each emotion category, and the emotion category with the highest matching score is determined as the user's current discrete emotion category (such as sadness, anxiety, anger, calmness, joy, etc.).
[0083] Confidence assessment: Output the confidence level of the sentiment classification result. When the confidence level is lower than the preset threshold (e.g., 0.6), the system maintains the current response mode without making any changes and waits for more data to accumulate.
[0084] Evolution prediction: Based on the trajectory change trend of (V, A) in the most recent N frames (e.g., N=5, corresponding to 5-10 seconds), predict the direction and trend of the evolution of the emotional state.
[0085] S3: Morphological Mapping Decision Based on the emotional state estimation results, the morphological mapping decision model 420 generates control parameters for each subsystem of the five-dimensional morphological change system 200. Specifically: Based on the discrete sentiment classification results, the corresponding basic mapping rules are invoked to obtain the baseline values of the five-dimensional control parameters.
[0086] Based on the valence-arousal continuous coordinate values, interpolation adjustments are made on the baseline values to match the control parameters with the intensity of emotion. For example, deep grief (with lower valence) triggers a higher temperature rise and a greater volume expansion than mild grief.
[0087] Feedforward compensation is performed based on the predicted emotional evolution. For example, when it is predicted that the user's emotion is transitioning from anxiety to calm, the frequency of breathing guidance (volume change subsystem 220 expansion and contraction) is gradually reduced in advance.
[0088] Based on the historical learning results of the online optimization strategy, the control parameters are fine-tuned individually.
[0089] The final output control parameters include: the vacuum value of each flexible bag 211 in the variable stiffness subsystem 210, the target air pressure and inflation / exhaust sequence of each airbag in the volume change subsystem 220, the target temperature value of each temperature zone defined by the temperature regulation subsystem 230, the current value and duration of each drive wire in the shape change subsystem 240, and the heart rate, amplitude and respiratory rate simulated by the rhythmic motion subsystem 250.
[0090] S4: Physical Empathy Response Execution The control system 400 sends control commands to each subsystem of the five-dimensional morphological change system 200 through the drive control interface to coordinate and drive the physical morphological changes in the five dimensions.
[0091] The response execution of each subsystem takes into account timing coordination to ensure a natural and smooth overall experience. In this embodiment, the typical response timing for sadness is as follows: t=0 seconds: Sadness is detected. First, the temperature regulation subsystem 230 is activated to start heating (it takes a long time due to temperature changes). At the same time, the variable stiffness subsystem 210 is activated to ventilate and repressurize to reduce stiffness. t=0-3 seconds: Stiffness reduction is complete, and the surface of the robot body 110 becomes soft; t=2-8 seconds: The volume change subsystem 220 begins to slowly inflate and expand, while the shape change subsystem 240 drives the drive filament to begin to slowly contract; t=5-15 seconds: The embracing action is gradually completed, and the robot body 110's two wings wrap around and bend inward; Starting at t=10 seconds: the rhythmic motion subsystem 250 initiates heartbeat simulation output; t=20-30 seconds: The surface temperature reaches the target value (37℃-39℃), all five-dimensional morphological changes are in place, and the companion state is maintained continuously.
[0092] The morphological changes of each subsystem should be smooth and gradual to avoid sudden changes that may cause discomfort or fright to users.
[0093] S5: Closed-Loop Care and Online Optimization During and after the execution of the physical empathy response, the multimodal sensing system 300 continuously monitors changes in the user's emotional state. The emotional state estimation engine 410 continuously updates the user's (V, A) coordinate values, forming a trajectory of emotional state changes.
[0094] The control system 400 performs the following closed-loop operation based on the trajectory of emotional changes: Real-time adjustment: If a change in emotional state (such as from sadness to anxiety) is detected during the execution of the physical empathy response, the control system 400 and the control morphology mapping decision model 420 regenerate the control parameters and smoothly switch to the new response strategy.
[0095] Response intensity adjustment: If the emotional state does not improve significantly for more than 30 seconds, the control system 400 will moderately increase the response intensity of each subsystem in the five-dimensional morphological change system 200 (such as further heating, increasing volume expansion, reducing heart rate, etc.).
[0096] Companionship Exit: When the system detects that the user's emotional state has returned to a calm or positive range and has remained so for a period of time (e.g., 2 minutes), the control system 400 gradually reduces the output of each subsystem and smoothly transitions to standby mode.
[0097] Online learning: The changes in emotional state during the interaction (especially the change in effectiveness value ΔV before and after the physical empathy response) are used as reward signals and input into the online optimization strategy model for parameter updates. ΔV>0 indicates a positive reward (emotional improvement), ΔV<0 indicates a negative reward (emotional deterioration or ineffectiveness), and ΔV≈0 indicates neutrality. Through cumulative learning from multiple interactions, the morphological mapping decision model 420 is gradually optimized into a personalized strategy suitable for specific users.
[0098] The above steps S1 to S5 are executed cyclically during each interaction between the user and the emotional companion robot 100, forming a complete closed loop of "perception-estimation-decision-execution-feedback" to achieve continuous and adaptive emotional companionship.
[0099] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.
Claims
1. An emotional companion robot based on five-dimensional morphological changes, characterized in that, This includes the robot body, a five-dimensional shape transformation system, a multimodal sensing system, and a control system; The robot body adopts a square pillow-shaped structure with an elastic silicone outer skin; A five-dimensional morphological transformation system is installed inside the robot body, including: A variable stiffness subsystem is used to change the surface stiffness of different regions of the robot body. The volume change subsystem is used to change the overall or partial volume of the robot body to achieve overall expansion of the robot body, hugging action, or support height adjustment. A temperature regulation subsystem is used to change the surface temperature of different areas of the robot body; The shape transformation subsystem is used to change the external contour shape of the robot body to achieve curling or stretching shape changes; The rhythmic motion subsystem is used to generate periodic movements that mimic the heartbeat of a living organism. A multimodal sensing system is installed on the robot body to sense information related to the user's emotional state; The control system is located inside the robot body and is connected to the five-dimensional morphological transformation system and the multimodal sensing system. The control system includes an emotion state estimation engine and a morphological mapping decision model. The emotion state estimation engine estimates the user's current emotion state based on the perception data of the multimodal sensing system. The morphological mapping decision model maps the emotion state estimation results to the control parameters of each subsystem to coordinate the control of the five-dimensional physical morphological output of the five-dimensional morphological transformation system and perform physical empathy response.
2. The emotional companion robot according to claim 1, characterized in that, The variable stiffness subsystem includes flexible bags arranged in sections and a vacuum control device connected to each flexible bag. The flexible bags include bags located in the outer region and bags located in the central region, and the flexible bags are filled with granular filler. The vacuum control device is used to evacuate or ventilate and repressurize each flexible bag, so that the granular filler switches between a blocked state and a free-flow dynamic state, thereby independently adjusting the stiffness of each region of the robot body.
3. The emotional companion robot according to claim 1, characterized in that, The volume change subsystem includes a miniature air pump, a solenoid valve assembly, a pressure limiting valve, and multiple internal airbags. Internal airbags are located in the center, sides, and bottom of the robot body; the central internal airbag is used to achieve overall inflation; the internal airbags on both sides serve as arm-like structures of the robot body to achieve hugging movements; and the internal airbag at the bottom is used to adjust the support height of the robot body. A miniature air pump inflates and deflates each internal airbag, a solenoid valve assembly independently controls the inflation and deflation pathways of each internal airbag, and a pressure relief valve automatically releases pressure when the internal airbag pressure exceeds a set value.
4. The emotional companion robot according to claim 1, characterized in that, The temperature regulation subsystem includes a flexible heating film and a cooling plate partitioned on the inner surface; the temperature regulation subsystem divides the surface of the robot body into multiple independent temperature control zones to achieve zoned heating or cooling.
5. The emotional companion robot according to claim 1, characterized in that, The shape transformation subsystem includes multiple transverse drive wires arranged along the length of the robot body and multiple longitudinal drive wires arranged along the width of the robot body; the drive wires are arranged at the edge of the robot body and together form a ring-shaped drive structure around the robot body; the drive wires achieve longitudinal or transverse deformation of the robot body by being heated and contracted by electricity.
6. The emotional companion robot according to claim 1, characterized in that, The multimodal sensing system includes: The tactile sensing module includes a flexible pressure sensor array for detecting the user's touch interaction patterns; The voice perception module includes a microphone array for acquiring voice signals and performing voice emotion recognition; The physiological sensing module includes a photoplethysmography (PPG) sensor, used to measure the user's heart rate and related physiological indicators. The motion sensing module, including an inertial measurement unit, is used to detect the user's motion manipulation of the robot body. The environmental sensing module is used to acquire environmental information.
7. The emotional companion robot according to claim 1, characterized in that, The emotional state estimation engine uses a hybrid approach combining a valence-arousal two-dimensional continuous space model with discrete emotion classification to estimate the user's emotional state. It also predicts the evolution direction of the emotional state based on the valence-arousal trajectory of multiple consecutive frames. The multimodal fusion model is implemented using deep learning or machine learning methods.
8. The emotional companion robot according to claim 7, characterized in that, The method for estimating a user's emotional state is as follows: based on the distance between the user's current valence-arousal coordinate value and the center point of each preset emotion category, calculate the matching score for each emotion category, and determine the emotion category with the highest matching score as the user's current emotional state.
9. The emotional companion robot according to claim 1, characterized in that, The morphological mapping decision model includes basic mapping rules and online optimization strategies; the basic mapping rules define the correspondence between emotional state categories and the control parameters of each subsystem of the five-dimensional morphological change system; The online optimization strategy adaptively adjusts the control parameters based on changes in the user's emotional state after a physical empathy response.
10. A closed-loop emotional interaction method based on the emotional companion robot of claim 9, characterized in that, Includes the following steps: S1: Real-time acquisition of interaction and environmental data through a multimodal sensing system; control system preprocesses the acquired raw data. S2: Based on the data collected in S1, the current emotional state is estimated through the emotional state estimation engine based on the multimodal fusion model and the valence-arousal two-dimensional continuous space model, and discrete classification and evolution prediction are performed. S3: Based on the emotional state estimation results, control parameters of each subsystem in the five-dimensional morphological change system are generated through the preset basic mapping rules in the morphological mapping decision model. S4: The control system coordinates and drives the five-dimensional morphological change system to perform physical morphological changes and provide a physical empathic response; S5: Continuously monitor changes in the user's emotional state. If a change in emotional state is detected, regenerate the control parameters of each subsystem in the five-dimensional morphological change system. If the emotional state remains unchanged, increase the response intensity of each subsystem in the five-dimensional morphological change system to form a closed-loop support. Use the change in emotional state as a reward signal to optimize the control parameters of the morphological mapping decision model online.