High-robustness accompanying robot system and method based on natural language understanding and action primitive library

Abstract

The present specification is developed about high-robustness bodybuilding robot systems and methods. The system is rich in module, and the basic module covers the functions of perception, action planning, speech synthesis, emotion coordination, role management and the like; the additional module has the characteristics of privacy protection, emotion interaction, role switching, physiological simulation, sound cloning, remote interaction, transfer prevention, cleaning and lubrication, video learning, organ replacement and the like. According to the demonstration and personalized extension method, demonstration action tracks and multi-modal data are collected, sub-fragments are decomposed to generate new track elements, the new track elements are stored, and meanwhile a semantic-action association database is established. Based on the method and the system, permanent expansion of personalized intimate interaction actions can be realized, new skills can be continuously learned according to different user requirements, accompanying services highly fitting expectations are provided for users, and practicability and user satisfaction are improved and multiple personalized requirements are met in multiple scenes such as emotion accompanying and life assistance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of embodied intelligent robots, and particularly relates to a biped humanoid robot system and method for decoupling the cognitive ability of a large language model from low-level motion control of a biped robot and realizing high-robustness emotional accompaniment through a parameterizable trajectory primitive library. BACKGROUND

[0002] Currently, large language models (LLM) and vision-language models (VLM) have exhibited semantic understanding and multi-modal reasoning capabilities close to humans at the software level. However, the efficient migration of such "brain intelligence" to biped humanoid robots still faces four recognized technical bottlenecks: 1. The dynamic balance robustness of biped robots in unstructured home environments is insufficient. Existing commercial biped robots (such as Atlas, Digit, Figure 02, etc.) can complete jumps and somersaults in the laboratory, but when facing the edges of carpets, wires, children's toys, stairs of different heights and materials in ordinary homes, the fall rate is still as high as 20%-50%, which cannot meet the safety requirements of home use.

[0003] 2. There is an insurmountable "execution gap" between high-level cognition and low-level execution. Existing solutions generally use a long-link architecture of "LLM→ task planning→ motion planning→ inverse kinematics→ trajectory optimization→ torque control", which has high computational delay (>500 ms), poor generalization ability, requires a large amount of scene labeled data, and causes the entire task to fail if any link fails.

[0004] 3. The threshold for action personalization and continuous learning is extremely high. Traditional industrial robots rely on professional teaching devices, while existing home robots (such as the Tesla Optimus prototype) still require engineers to record actions through code or remote operation, and ordinary users cannot teach robots to "hug like me slowly" or "dance my favorite dance" through natural language or a few demonstrations.

[0005] 4. The commercial landing value and technical difficulty are seriously mismatched. The market has extremely high precision requirements for general household robots (such as precise clothes folding and dishwashing), leading to uncontrolled costs. Emotional accompaniment and entertainment accompaniment scenarios have natural high fault tolerance, but lack a system architecture that is specifically optimized for this scenario. SUMMARY

[0006] To solve all the above technical problems, the application creatively proposes a new technical route of "complete decoupling of cognition and action + trajectory primitive library + online parameterization correction + natural language teaching closed loop", the core of which is not obvious: The "accompanying macro command" is first separated from the "general high-precision operation", actively giving up the pursuit of micron-level end precision, and instead pursuing the "emotional correctness" of 99.9% success rate, greatly reducing the difficulty of physical control (the field previously generally believed that general precision operation must be solved first to accompany, which is a inherent bias).

[0007] The concept of "trajectory primitive" is proposed: unlike traditional skill library, the trajectory primitive in the invention is a space-time-haptic hybrid sequence that is target-independent and can be parameterized online, so that the same "hug" primitive can be used for 1.5 m children and 1.8 m adults without the need for retraining or planning.

[0008] The "natural language-action joint embedding space + user demonstration incremental learning" mechanism is created, which allows ordinary users to permanently expand the robot's personalized action library by speaking and demonstrating 1-3 times, breaking the industry convention that "robot actions can only be pre-installed by manufacturers".

[0009] A high-robustness embodied companion robot system and method based on natural language understanding and action primitive library It includes a perception and intention understanding module, an action planning and execution module, an emotional speech synthesis module, a multi-modal emotion consistency coordination unit, and a role management module. The action planning and execution module includes a trajectory primitive library and a parameterized online correction unit. The trajectory primitive library stores multiple target-independent and parameterizable space-time-haptic hybrid trajectory primitives, which at least include normalized joint angle reference trajectories, six-dimensional force / torque reference templates, and online fillable target position, velocity multiplier, and compliance coefficient parameters. The parameterized online correction unit adjusts the selected trajectory primitive in real time according to real-time visual, tactile, or depth sensor data.

[0010] It also includes a master-specific unlockable privacy shield device, which includes a physical shield and a two-factor identity recognition module. Only when the pre-registered biological fingerprint of the adult master and the special digital key of the mobile phone are detected at the same time can the device be unlocked, otherwise it remains in a permanently closed state that cannot be violently opened without damage.

[0011] It also includes a multi-modal emotional teasing and feedback interaction system, which includes: 1. Real-time sexual desire assessment unit for fusing multi-modal signals such as user voice tone, facial redness, heart rate, and contact pressure to calculate the current sexual desire.

[0012] 2. Progressive teasing strategy library and real-time feedback closed-loop control unit, used to dynamically select and execute synchronized teasing behaviors of language, action, and voice tone according to the degree of sexual desire, and to positively reinforce when detecting positive physiological responses, and to immediately degrade when detecting discomfort.

[0013] It also includes a magnetic type face component that can be quickly replaced and an all-weather role intelligent scheduling system, which supports automatic switching of multiple roles including child care mode, adolescent tutoring mode, elderly companion mode, and adult intimate companion mode, and automatically loads mutually isolated action primitive library, voice corpus, and content filtering strategy under different roles.

[0014] It also includes a real heartbeat-breathing-body odor simulation mechanism, including a chest heartbeat and breathing airbag driving device, a replaceable body odor capsule module, and a skin temperature regulation layer, for synchronously outputting consistent heartbeat, breathing fluctuations, and personalized body odor during intimate contact.

[0015] It also includes a dedicated sound cloning and remote interaction unit, which supports generating a dedicated timbre library based on the voice samples uploaded by the owner, and supports real-time transmission of touch and voice between two robots through an encrypted channel to achieve synchronous intimate interaction.

[0016] It also includes a soul binding and anti-transfer mechanism, where the robot only responds to the biological features of the first registered adult owner, and after restoring the factory settings, it enters a permanent locked state, and can only be re-bound to a new owner through official paid authentication.

[0017] (1) Collect the demonstration action trajectory and multi-modal sensor data of the user being pulled or teleoperated under natural language description. (2) Decompose into sub-segments based on joint zero-speed points and / or end-of-force / torque mutations. (3) After target-independent processing of the sub-segments, generate new trajectory primitives and store them in the trajectory primitive library. (4) Convert the natural language description to a text embedding vector, and convert the new trajectory primitive to an action embedding vector and store it in a vector database that supports approximate nearest neighbor search, forming a semantic-action association that can be searched by natural language.

[0018] The system realizes the permanent expansion of personalized intimate interaction actions through the method of claim 8.

[0019] It also includes a one-key self-cleaning system for private parts, an automatic lubricant secretion system, and a controllable slight negative pressure enhancement system. The system includes: a. One-key self-cleaning mechanism, including ultraviolet LED, warm water spray, and negative pressure suction device. b. Lubricant intelligent secretion mechanism, including replaceable lubricant capsule and automatic secretion control unit based on pressure-temperature-humidity sensor. c. Controllable slight negative pressure enhancement mechanism, including: Built-in medical-grade miniature silent vacuum pump (noise < 25 dB) and multi-cavity silica gel negative pressure capsule.

[0020] Negative pressure intensity is 0-18 kPa in 8 grades, and the default is off. It can be manually turned on or off by the owner through voice / APP / chest long press for 3 seconds.

[0021] Support "heart beat synchronization mode" (negative pressure pulse frequency is synchronized with user's real-time heart rate 1:1) and "breath synchronization mode" (fluctuating with user's breathing rhythm). d. The above negative pressure enhancement function can only be activated after double confirmation of "adult intimate companion mode" and "owner's private shield has been unlocked", and is permanently disabled at the hardware level in child / teenager / old person mode.

[0022] It also includes a quick and non-destructive replacement mechanism for the privacy organ assembly, which is connected to the robot body using a standardized magnetic attraction + quick plug interface. Users can complete the overall replacement without tools within 30 seconds in the unlocked state of the privacy shield. The replaced old assembly can be recycled through official recycling channels for environmental protection or resold. The system automatically detects the assembly ID and rebinds it with the current owner's biometric features, ensuring that the soul binding and security policy remain effective before and after replacement. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 It is the principle architecture mind map of the system.

[0024] This mind map focuses on a high-robustness embodied companion robot system and method, and elaborates on three main aspects: system modules, demonstration and individualized expansion methods, and system individualized expansion implementation.

[0025] System modules: diverse functions, comprehensive coverage The system modules are divided into basic modules and additional modules. The basic modules are the core support for the robot to run. The perception and intention understanding module is like the "perception nerve" of the robot, responsible for receiving external information and understanding user intent; the action planning and execution module is the "action center", which has a unique target-independent, parameterizable space-time - force mixed trajectory primitive, including normalized joint angle reference trajectory, end six-dimensional force / torque reference template, etc., and can fill in parameters online, such as target position, speed multiplier, compliance coefficient, etc. It can also adjust parameters through parameterized online correction unit according to real-time sensor data, ensuring accurate and flexible motion. The emotional speech synthesis module gives the robot the ability to express emotions, and the multi-modal emotion consistency coordination unit makes the robot's speech, action, etc. Expressions are coordinated and unified, and the role management module facilitates the management of different roles of the robot.

[0026] The additional modules further expand the functions and characteristics of the robot. The owner-exclusive privacy shield device can be unlocked to ensure privacy and security through a physical shield and a two-factor identity recognition module (biological fingerprint and a mobile phone exclusive digital key). The multi-modal emotional provocation and feedback interaction system uses a real-time sexual desire evaluation unit to calculate the sexual desire degree by fusing multi-modal signals, and then realizes dynamic provocation, positive feedback reinforcement, and discomfort degradation through a progressive provocation strategy library and a real-time feedback closed-loop control unit, thereby enhancing emotional interaction with the user. The magnetic type face component that can be quickly replaced and the all-weather role intelligent scheduling system support automatic switching of multiple roles, different roles correspond to different loading isolation libraries, and meet different scene requirements. The real heartbeat - breathing - body odor simulation mechanism simulates human physiological characteristics from multiple dimensions to improve the realism of the companion. The exclusive voice cloning and remote online interaction unit can generate an exclusive timbre library, realize real-time transmission of touch and voice through an encrypted channel between two robots, and enhance the interactive experience. The soul binding and anti-transfer mechanism ensures the exclusivity and security of the robot. The privacy part one-key self-cleaning, lubricant automatic secretion and controllable slight negative pressure enhancement system is activated in a specific mode to ensure the use of hygiene and comfort. The video action learning module can receive videos, convert joint trajectories through a vision - action alignment model, generate new trajectory primitives, store them, support third-person video input, and adapt to the scale of the robot itself. The privacy organ assembly quick non-destructive replacement mechanism facilitates maintenance and upgrading, the old assembly has an official recycling process or a secondary sales channel, and can automatically detect the assembly identity ID and rebind the owner's biological features.

[0027] Demonstration and personalized expansion method: flexible learning, personalized customization This method first collects demonstration action trajectories and multi-modal sensor data, uses a time series segmentation algorithm to decompose them into sub-segments, and then generates new trajectory primitives and stores them in a trajectory primitive library. At the same time, the natural language description is converted into a text embedding vector, and the new trajectory primitive is converted into an action embedding vector, which is stored in a database to form a semantic - action association, providing a basis for personalized learning of the robot.

[0028] System personalized expansion implementation: continuous evolution, meet multiple Through the above demonstration and personalized expansion method, the robot can continuously learn new actions and skills, and realize permanent expansion of personalized intimate interaction actions. This means that the robot can continuously evolve according to the needs and preferences of different users, providing more user-expected companion services, whether it is emotional companionship, life assistance or other personalized needs, can be better met, greatly improving the practicality and user satisfaction of the robot. DETAILED DESCRIPTION

[0029] The application will be described in further detail below with reference to the drawings. These embodiments are only used to explain the application and are not limited thereto.

[0030] Embodiment 1 Overall system architecture and hardware platform The robot of the present application is a biped humanoid structure, with an appearance height that can be flexibly set within the range of 150-185 cm according to the target user group, and a weight that is highly consistent with the body proportions of a real adult human, thereby achieving a very high degree of simulation in terms of appearance, proportions, natural posture, and gait. The joints use high-power-density collimated drive actuators, and the entire body is covered with medical-grade platinum sulfide silicone skin and distributed capacitive tactile sensors. The perception system includes multiple RGB-D cameras, microphone arrays, full-body electronic skin, plantar force / torque sensors, and torso IMUs. The computing platform supports localized large model inference and high-frequency real-time motion control, with a duration that meets the all-weather companion requirements.

[0031] Embodiment 2 Perception and intent understanding module The system fuses real-time image, voice, tactile, and other multi-modal signals, and through a frozen or slightly fine-tuned multi-modal large model, maps user natural language, expressions, tone, and contact methods into structured high-order companion intent instructions, supporting highly natural interaction understanding.

[0032] Embodiment 3 Motion core module A layered high-robustness gait controller and full-body active safety mechanism are used to ensure that the robot remains stable and stops smoothly in complex home ground, stairs, and sudden push scenarios, achieving a safe experience of "never falling hard".

[0033] Embodiment 4 Action planning and execution module (core) The trajectory primitive library stores multiple target-independent, parameterizable spatio-temporal-haptic hybrid trajectory primitives, each of which contains a normalized joint angle reference trajectory, an end-of-arm force / torque reference template, and parameters such as target position, velocity multiplier, and compliance coefficient that can be filled online. The intent-primitive mapping unit directly maps high-order intent to the most matching primitive or combination of primitives through a lightweight network; the parameterized online correction unit adjusts target position, velocity, and compliance in real time based on real-time visual and tactile data, allowing the same primitive to be naturally executed in different heights, body types, and real-time postures.

[0034] Embodiment 5 Master-specific privacy shield device The robot sets a high-strength physical shield in the specific privacy area, which can only be unlocked when the pre-registered adult master's biological fingerprint and the special digital key of the mobile phone are detected at the same time, otherwise it remains in a permanently closed state that cannot be forcibly opened without damage.

[0035] Embodiment 6 Multi-modal emotion consistency coordination and emotional speech synthesis The system evaluates the user's emotional state in real time, dynamically coordinates the voice tone, action strength, and tactile feedback, and forms a highly consistent natural emotional interaction; any unsuitable signal immediately degrades or stops.

[0036] Embodiment 7 Role management and all-weather intelligent scheduling Supports multiple roles such as child care, adolescent tutoring, elderly companion, adult emotional companion, etc. Different roles automatically load completely isolated action primitive libraries, voice corpus, content filtering strategies, and hardware function switches. Adult emotional companion related functions are only enabled after the privacy shield is unlocked, and the rest of the modes are permanently disabled at the hardware level.

[0037] Embodiment 8 Real physiological simulation mechanism Including a chest flexible air bag to realize heartbeat and breathing simulation, a replaceable body odor module, and a skin constant temperature layer, further improving the real companion feeling in authorized mode.

[0038] Embodiment 9 Magnetic face quick replacement, exclusive voice cloning, and remote online Magnetic face quick release; supports master voice cloning; two robots can synchronize voice and touch in real time through an encrypted channel.

[0039] Embodiment 10 Soul binding and anti-transfer mechanism The robot only responds to the biological features of the first registered adult owner, and after recovery from the factory, it enters permanent lock, and can be re-bound only after official paid authentication.

[0040] Embodiment 11 Privacy part self-cleaning and auxiliary mechanism Integrated one-key self-cleaning, controllable auxiliary mechanism, all functions are activated only after double confirmation of adult emotional companion mode and privacy shield unlocked, default off, permanently disabled at the hardware level in other modes.

[0041] Embodiment 12 Natural language and video dual-mode teaching programming (corresponding to claim 10) In the adult emotional companion mode with the privacy shield unlocked: Traditional teaching: user natural language + manual traction → automatic timing segmentation, target-independent, new primitive generation, semantic-action joint embedding; Video teaching: user provides demonstration video and oral name → system automatically extracts human posture sequence → body type adaptive redirection → target-independent → generates new trajectory primitives and enters the library. Both methods can permanently expand the exclusive action library with only a few samples, and share the same double security lock, which cannot be used in other modes.

[0042] Embodiment 13: Quick non-destructive replacement mechanism of privacy organ assembly: the robot privacy area adopts a standardized magnetic suction + quick plug interface design, and the entire privacy organ assembly (including all internal functional mechanisms) is a single modular component that can be integrally pulled and plugged. In the adult emotional companion mode where the privacy shield has been unlocked, the user only needs to pull to remove the old assembly, push in the new assembly, and the entire process does not require any tools, and the average time consumption is less than 30 seconds. The system automatically reads the unique identity ID of the new assembly during power-on self-test, and rebinds it with the current master biological characteristics, ensuring seamless continuation of soul binding, anti-transfer, and all security strategies. The old assembly can be returned to the official designated recycling point for environmental protection treatment or resold after being refurbished, thereby greatly improving the overall secondary circulation value and full life cycle economy of the robot.

[0043] Industrial applicability: all functions of the present application can be smoothly run on consumer-grade hardware, with perfect multiple security and privacy protection mechanisms, fully meeting the use requirements of family privacy space, and having clear industrialization prospects.

[0044] The above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the scope of the claims of the present application.

Claims

1. A high-robustness embodied companion robot system and method based on natural language understanding and a library of action primitives, characterized by, The system comprises a perception and intention understanding module, an action planning and execution module, an emotional speech synthesis module, a multi-modal emotion consistency coordination unit, and a role management module; the action planning and execution module comprises a trajectory primitive library and a parameterized online correction unit, the trajectory primitive library stores a plurality of target-independent, parameterizable spatio-temporal-haptic hybrid trajectory primitives, the trajectory primitives at least include normalized joint angle reference trajectories, end six-dimensional force / torque reference templates, and target position, velocity scaling, and compliance coefficient parameters that can be filled online; The parameterized online correction unit adjusts the selected trajectory primitive parameters in real time according to real-time visual, tactile or depth sensor data.

2. The system of claim 1, wherein, It also includes a master-specific unlockable privacy shield device, which includes a physical shield and a two-factor identity recognition module, which can only be unlocked when the pre-registered adult master's biological fingerprint and the mobile phone-specific digital key are detected at the same time, otherwise it remains in a permanently closed state that cannot be violently opened without damage.

3. The system of claim 1 or 2, wherein, It also includes a multi-modal emotional teasing and feedback interaction system, which comprises: A real-time sexual desire assessment unit for calculating the current sexual desire by fusing multi-modal signals such as user voice tone, facial redness, heart rate, and contact pressure; A progressive teasing strategy library and a real-time feedback closed-loop control unit for dynamically selecting and executing synchronized teasing behaviors of language, action, and voice tone according to the sexual desire, and positively reinforcing when detecting positive physiological responses, and immediately degrading when detecting discomfort.

4. The system of any one of claims 1-3, wherein, It also includes a magnetic type face component that can be quickly replaced and an all-weather role intelligent scheduling system, which supports automatic switching of multiple roles including child care mode, adolescent tutoring mode, elderly companion mode, and adult intimate companion mode, and automatically loads mutually isolated action primitive libraries, voice databases and content filtering strategies under different roles.

5. The system of any one of claims 1-4, wherein, It also includes a real heartbeat-breathing-body odor simulation mechanism, which includes a chest heartbeat and breathing airbag driving device, a replaceable body odor capsule module, and a skin temperature regulation layer, which is used to output consistent heartbeat, breathing fluctuations and personalized body odor as human beings during intimate contact.

6. The system of any one of claims 1-5, wherein, It also includes a dedicated sound cloning and remote interaction unit, which supports generating a dedicated timbre library based on the voice samples uploaded by the master, and supports real-time transmission of tactile and voice between two robots through an encrypted channel to realize synchronous intimate interaction in different places.

7. The system of any one of claims 1-6, wherein, It also includes a soul binding and anti-transfer mechanism, the robot only responds to the biological characteristics of the first registered adult master, and enters a permanently locked state after restoring factory settings, and can be re-bound to a new master through official paid authentication.

8. A teaching and personalization extension method of a multi-mode intimate companion robot, characterized in that, It includes: (1) Collecting demonstration action trajectories and multi-modal sensor data under natural language description of the user; (2) The timing segmentation algorithm based on joint zero velocity point and / or end force / torque mutation is decomposed into sub-segments; (3) After target-independent processing of the sub-segments, new trajectory primitives are generated and stored in the trajectory primitive library; (4) Convert natural language description into text embedding vector, convert new trajectory primitive into action embedding vector and store it in vector database supporting approximate nearest neighbor search, forming semantic-action association that can be searched by natural language.

9. The system of any one of claims 1-7, wherein, The system realizes the permanent expansion of personalized intimate interaction actions by the method of claim 8.

10. The system of any one of claims 1-9, wherein, It also includes a one-key self-cleaning system for private parts, an automatic secretion of lubricant and a controllable slight negative pressure enhancement system, as well as a video action learning module. The one-key self-cleaning system for private parts, the automatic secretion of lubricant and the controllable slight negative pressure enhancement system can only be activated after the double confirmation that the adult intimate companion mode and the privacy shield have been unlocked, and are permanently disabled at the hardware level in other modes. The video action learning module can receive a video provided by the user, which contains human demonstration actions. Through a pre-trained visual-action alignment model, the human body posture sequence in the video is converted into joint trajectories in the robot coordinate system, and new trajectory primitives are generated and stored in the trajectory primitive library after automatic target-independent processing. The third-person video input is supported and the current robot height and limb proportion are automatically adapted. Thus, the user only needs to provide a video to teach the robot a new intimate interaction action, and this function can only be enabled in the adult intimate companion mode with the privacy shield unlocked. It also includes a quick and non-destructive replacement mechanism for the private organ assembly. The private organ assembly is connected to the robot body using standardized magnetic attraction + quick plug interfaces, and supports the user to complete the overall replacement without tools within 30 seconds in the privacy shield unlocked state. The old assembly after replacement can be recycled through the official recycling channel for environmental protection or resold. The system automatically detects the assembly ID and rebinds it with the current master's biometric features, ensuring that the soul binding and security policy remain effective before and after replacement.

Images