Intelligent composite humanoid robot and control method

Abstract

The application provides a smart composite humanoid robot and a control method, the smart composite humanoid robot comprising a mobile chassis, a lifting device arranged above the mobile chassis, a trunk of the humanoid robot connected to the top of the lifting device, one mechanical arm arranged on each of the left and right sides of the humanoid robot, and the humanoid robot integrated with a main control module, a router module and a sensor module; an operator remotely develops through the router module, remotely controls the robot through VR, and demonstrates actions, a diffusion model is constructed, motion trajectory generation is performed through conditional diffusion, joint angle instructions of executed actions are output according to the VR demonstrated actions, and compliant control and kinematic feedforward compensation are continuously performed during the execution of the actions to adjust motor torque instructions.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligent robot technology, and particularly relates to an intelligent composite humanoid robot and its control method. Background Technology

[0002] A hybrid robot is a comprehensive robotic system that integrates multiple robotic technologies, such as robotic arms, mobile platforms, sensors, vision systems, artificial intelligence, and control systems. It possesses complex motion control and a high degree of intelligence, enabling it to adapt to various complex environments and complete diverse tasks.

[0003] The prior art patent CN120791770A discloses a composite mobile robot and a composite mobile robot control system, including a control terminal, a wireless communication module, and a sensor module. The sensor module comprehensively acquires the real-time position of the mobile robot, the surrounding environment, and the position information of the robotic arm's movement points. The data processing module processes the received information in real time. The obstacle point generation module accurately generates and marks obstacle points based on the processed data. The movement path planning module plans a reasonable route by combining the obstacle points and the real-time position information of the mobile robot. The robotic arm positioning module generates the real-time outer contour information of the robotic arm. The robotic arm posture adjustment module plans the movement trajectory of the robotic arm and adjusts its posture based on the outer contour information and obstacle point information. Through the close cooperation of these modules, the robot's environmental perception ability in complex environments is improved, avoiding inaccurate positioning, navigation errors, and untimely obstacle avoidance, thereby improving work efficiency and safety. Summary of the Invention

[0004] Current hybrid robots are mostly robot systems consisting of a wheeled mobile platform and a single robotic arm. However, single-arm robots are insufficient in terms of operational flexibility, task efficiency, and adaptability, and cannot perform complex tasks that require the cooperation of both arms. Furthermore, traditional hybrid robots only have planar movement capabilities and lack movement along the Z-axis (height), which severely limits their operational range.

[0005] Meanwhile, traditional composite robots typically use wired connections to the main controller for development and code modification, which is extremely inconvenient and increases safety risks. In terms of motion teaching, traditional composite robots generally use teaching schemes to collect data. These schemes usually require two identical robots and necessitate human intervention by manually moving the robotic arms. This approach is costly and inherently dangerous.

[0006] This invention discloses an intelligent composite humanoid robot, including a mobile chassis with a lifting device on top of the chassis; the top of the lifting device is connected to the torso of the humanoid robot; a mechanical arm is provided on each of the left and right sides of the humanoid robot; the humanoid robot integrates a main control module, a router module, and a sensor module; the operator performs remote development through the router module and controls the robot's movements by transmitting data and VR remote control commands through the router module.

[0007] Specifically, the robotic arm communicates with the main control module via Ethernet. The robotic arm's Ethernet cable is first connected to the router, and then combined with the router's Ethernet cable to connect to the main control module.

[0008] Specifically, the robotic arm has a gripper at its far end, which communicates with the robotic arm via RS485. The main control module indirectly controls the gripper by controlling the robotic arm signals.

[0009] Specifically, the sensor module includes several depth cameras, which communicate with the main control module via USB; depth cameras are also installed on the head, chest cavity, and two grippers of the robotic arm of the humanoid robot.

[0010] Specifically, the lifting device uses a lead screw slide, driven by a servo motor at the bottom. The lead screw slide consists of a slider and a rail, with a fixed ring on the slider on which the humanoid robot is mounted. The lifting device controls the robot's height; the mobile chassis controls the robot's planar movement via motors and wheels.

[0011] This invention also discloses a control method for an intelligent composite humanoid robot, applied to the aforementioned intelligent composite humanoid robot, comprising the following steps: S1. The operator uses VR to remotely control the robot and teach it to move. During the movement teaching, the VR system obtains the homogeneous transformation matrix of the VR controller relative to the VR headset in real time. S2. Perform inverse kinematic analysis on the taught movement and calculate the joint angle commands during the taught movement; S3. Input the joint angle command and sensor data into the diffusion model; S4. The diffusion model uses conditional diffusion to generate motion trajectories, performs motion calculations, and outputs joint angle commands. S5. The humanoid robot performs actions and continuously performs compliant control and kinematic feedforward compensation to adjust motor torque commands.

[0012] Specifically, when the neural network diffusion model in S4 generates actions, it includes the following steps: S4-1. Collect low-dimensional data of the robot, including body perception data of the robotic arm and gripper, visual perception data collected by sensors, and verbal commands from the operator. Map the low-dimensional data to a unified feature space through an independent feature encoder, perform cross-modal fusion, and generate conditional vectors. S4-2. The forward diffusion model adds noise to the action, and then the conditional vector is used as the conditional input of the diffusion model for reverse denoising. The denoising direction is adjusted according to the conditional vector to generate the predicted action sequence and motion trajectory.

[0013] Specifically, after generating the predicted action sequence and motion trajectory, the environmental point cloud model is scanned before the robot performs the action. If a collision risk is detected, the path is immediately replanned.

[0014] Specifically, the planned path is solved using the damped least squares inverse kinematics solution mechanism, and joint angle commands are output.

[0015] Specifically, in S5, the contact force information is fed back in real time by the six-dimensional force sensor at the wrist of the robotic arm. The drive impedance controller dynamically adjusts the joint stiffness and damping coefficient to perform compliant control on the joint angle command. The adjusted joint angle command is then subjected to dynamic feedforward compensation, including gravity compensation, Coriolis force compensation and friction compensation. The motor torque command is generated from the joint angle command and sent to the motor driver.

[0016] The beneficial effects of this invention are as follows: the structural design of dual robotic arms and an elevator greatly expands the operating range; the use of the open-source ROS system for development greatly facilitates secondary development; the router module enables wireless connection without relying on an external wireless network, making development safer and more convenient; the use of VR for remote robot operation and motion teaching is more in line with human habits, making it convenient and safe; at the same time, the trajectory detection and compliant control methods facilitate development, requiring less effort from developers to focus on safety and trajectory smoothness. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the structure of the present invention.

[0018] Figure 2 This is a flowchart of the control method of the present invention.

[0019] Figure 3 This is a structural diagram of the lifting device.

[0020] In the diagram, 1 is the mobile chassis, 2 is the lifting device, 3 is the robotic arm, 4 is the gripper, 5 is the fixed ring, 6 is the slider, and 7 is the slide rail. Detailed Implementation

[0021] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] Example 1: An intelligent composite humanoid robot includes a mobile chassis with a lifting device on top of it; the top of the lifting device is connected to the torso of the humanoid robot; a robotic arm is provided on each of the left and right sides of the humanoid robot; the humanoid robot integrates a main control module, a router module, and a sensor module; the operator performs remote development through the router module and transmits data and VR remote control commands through the router module to control the robot's movements.

[0023] A control method for an intelligent composite humanoid robot, applied to the aforementioned intelligent composite humanoid robot, includes the following steps: S1. The operator uses VR to remotely control the robot and teach it to move. During the movement teaching, the VR system obtains the homogeneous transformation matrix of the VR controller relative to the VR headset in real time. S2. Perform inverse kinematic analysis on the taught movement and calculate the joint angle commands during the taught movement; S3. Input the joint angle command and sensor data into the diffusion model; S4. The diffusion model uses conditional diffusion to generate motion trajectories, performs motion calculations, and outputs joint angle commands. S5. The humanoid robot performs actions and continuously performs compliant control and kinematic feedforward compensation to adjust motor torque commands.

[0024] The humanoid robot consists of three parts: arms, torso, and head; the two arms are composed of robotic arms with control grippers attached to their ends. Each gripper also integrates a depth camera as a sensor.

[0025] The torso consists of a lifting mechanism, an abdominal cavity, and a head. The lifting mechanism uses a lead screw slide and is driven by a servo motor at the bottom. The rear panel houses the battery and voltage conversion module, a router module, and the main control module. Depth cameras are also located in the middle of the chest cavity and the head for perception.

[0026] The robot's sensors consist of four depth cameras, communicating directly with the main controller via USB. The arms communicate with the main controller via Ethernet; the Ethernet cables from the arms are first connected to a router, which then merges them into a single Ethernet cable connected to the main controller. The router also serves as the robot's internal network signal transmitter. The two grippers communicate with the robotic arms via RS485; the main controller indirectly controls the grippers by controlling signals from the robotic arms. The chassis uses the CAN communication protocol, so a CAN-to-USB converter is needed to establish a USB interface for communication with the main controller. Similarly, the lift's RS485 communication needs to be converted to USB for communication with the main controller.

[0027] The power supply is divided into two parts: one part is powered by a 24V battery built into the chassis, and the other part is powered by an additional 24V 30Ah battery.

[0028] The main control module requires 19V, so a 24V to 19V transformer is needed to power it. The four sensors are directly powered by the main control's USB 5V 1A current. The router module requires 5V but needs 3A current to operate at full power, so a 24V to 5V converter needs to be drawn from the chassis. The robotic arm and lift motors are both 24V, so they are directly connected to the battery. The gripper is directly connected to the robotic arm and is powered by 24V from the RS485 at the end of the robotic arm.

[0029] During development, developers can remotely connect to the main control module via the hotspot network sent by the router module to modify the code, avoiding the inconvenience of wired connections and the danger of wires getting stuck in wheels.

[0030] When teaching motion via VR telecontrol, the VR system first acquires the homogeneous transformation matrix of the VR controller relative to the VR headset in real time, collecting the current pose Tt and the previous pose Tt-1 at a sampling frequency of 90Hz. Based on these two poses, the inter-frame relative transformation ΔT=Tt·Tt-1-1 is calculated, decoupled into a three-dimensional translation increment Δp=[Δx,Δy,Δz] and a rotation increment (quaternion representation) Δq. These increments are then processed using a second-order Butterworth low-pass filter: the translation component is filtered using a 2Hz cutoff frequency, and the rotation component is filtered using spherical linear interpolation (Slerp), effectively eliminating physiological hand tremors >5Hz while maintaining the continuity of the motion intention. The filtered pose increment ΔT~ is mapped to the robotic arm end effector through the pose composite operation xarmt=xarmt-1∘ΔT~, achieving millimeter-level motion tracking with a positioning accuracy <0.5mm.

[0031] Then, inverse kinematics analysis is performed, calculating the joint angle command θtcmd=K-1(xarmt) based on the robotic arm's geometric model. Analytical solutions are used to avoid singular configurations. This command is executed with a one-control-cycle delay (Δt=2ms), during which command smoothing is performed: joint position is calculated using linear interpolation, and attitude is calculated using spherical linear interpolation (Slerp), ensuring the joint space C... 1 Continuity (velocity continuity). Ultimately, safety is ensured through impedance control strategies based on admittance models. Dynamically adjust virtual stiffness: During normal operation, maintain a high stiffness state of Kd = 5000 N / m; when the wrist six-dimensional force sensor detects a contact force > 5N (collision state), switch to a compliant mode of Kd < 200 N / m within 5ms, so that the end of the robotic arm conforms to the displacement of the contact surface (typical retraction amount 2-5mm) instead of continuously applying force, limiting the maximum contact force to within the 20N safety threshold.

[0032] Simultaneously, data can be collected remotely via VR devices, with the movements of the arms, lift, and chassis all linked to two controllers. The arms move in accordance with the end-effector poses emitted by the VR controllers, and the opening and closing of the grippers are controlled by the trigger buttons on the controllers. The lift's vertical movement is controlled by the joystick on the right controller, while the chassis's linear and angular velocities are controlled by the joystick on the left controller. This system allows for remote control of complex robot systems by a single person, and it is simpler and more convenient than traditional methods.

[0033] After the operator demonstrates remote operation using VR, the collected sensor data (motor joint position, joint speed, camera image, depth point cloud information) and action commands (joint motor commands) are input into the generative neural network diffusion model. A conditional diffusion model is used, with sensor information such as image information at a certain time stamp as a condition, to perform a diffusion denoising process.

[0034] When the diffusion model generates motion, it first collects low-dimensional data of the robot, including proprioceptive data of the robotic arm and gripper, visual perception data collected by sensors, and verbal commands from the operator. The low-dimensional data is then mapped to a unified feature space through an independent feature encoder, and cross-modal fusion is performed to generate conditional vectors.

[0035] The diffusion model adds noise to the action through forward diffusion, and then uses the conditional vector as the conditional input of the diffusion model for reverse denoising. The denoising direction is adjusted according to the conditional vector to generate the predicted action sequence and motion trajectory.

[0036] Specifically, this embodiment uses multimodal conditional information to constrain the motion generation of the diffusion model. This multimodal information includes visual images, verbal commands, and the robot's current low-dimensional state (joint angles, angular velocities, torque information, etc.). The data sources and statistical structures of different modalities differ significantly; therefore, they must first be mapped to a unified feature space using independent feature encoders. The visual modality is processed by a visual encoding network to extract spatial structure, key object information, and geometric relationships in the scene; the verbal modality is processed by a verbal encoding network to obtain semantic representations such as task semantics, target descriptions, and operational constraints; and the robot's state modality is processed by a state encoding network to obtain low-dimensional dynamic features containing the current posture, joint configuration, and actuator state.

[0037] After being encoded, the features of each modality are aligned and integrated through a cross-modal fusion module. This fusion module can employ vector concatenation, linear mapping, cross-modal attention mechanisms, or gating-based feature interaction methods. Its core objective is to establish semantic relationships between vision, language, and state in a unified latent space. This allows language instructions to selectively focus on task-related regions within visual features, while simultaneously using state features to constrain the potential action space, preventing the generation of actions inconsistent with the current physical state or unexecutable. Through this fusion mechanism, the model obtains a multimodal conditional representation that combines environmental perception, task understanding, and physical feasibility constraints.

[0038] The fused conditional vector serves as the conditional input to the diffusion model, continuously participating in noise prediction and action updates throughout the reverse denoising process of action generation. In each reverse step, the diffusion model adjusts the denoising direction based on this conditional vector, ensuring that the generated action simultaneously satisfies scene semantics, task intent, and robot state constraints. This mechanism ensures global consistency in the action generation process, enabling the model to generate executable, stable, and task-compliant control commands in complex environments.

[0039] In the forward propagation process (noising process) of the diffusion model, Gaussian noise is progressively added to the real action a_t(k), forming a series of intermediate forms from "real action" to "random noise action". The forward propagation relationship can be described as: a_t(k) = sqrt(alpha_bar_k) * a_t + sqrt(1 - alpha_bar_k) * noise Where k is the number of diffusion steps, alpha_bar_k is the cumulative product of a predefined sequence of noise attenuation coefficients, and noise is Gaussian noise with zero mean. As k increases, the action a_t(k) will gradually be submerged in noise, eventually approaching complete randomness.

[0040] In the backdiffusion process, the model starts with random noise actions, progressively predicts and removes the noise to recover actions that are close to reality. The backdiffusion denoising process can be described as follows: a_t(k-1) = (1 / sqrt(alpha_k)) * ( a_t(k) - (beta_k / sqrt(1 - alpha_bar_k)) * predicted_noise ) + optional_noise The predicted_noise is predicted by the neural network based on the current a_t(k), the diffusion step k, and the condition vector c_t; the optional_noise is an optional random term used to improve the diversity of generation. After multiple backward iterations, the noise-free action a_t(0) is finally obtained, which is the action instruction generated by the model.

[0041] To ensure that the generated actions are consistent with the actions in the real demonstration in the temporal dimension, the diffusion model employs a follow-matching (time-step action supervision) mechanism. Specifically, after the diffusion process is complete, the model generates an action sequence {a_hat_1, a_hat_2, ..., a_hat_T} and compares it with the real action sequence {a_1, a_2, ..., a_T} in the same demonstration at the same time step, calculating their differences. By minimizing the error at each time step, the model can learn the temporal structure, action amplitude, and behavioral logic of the real actions during training.

[0042] The final training objective of the diffusion model consists of two parts: first, noise prediction error, which constrains the model to accurately denoise during the diffusion reverse process; and second, follow-matching motion supervision, which ensures that the generated motion and the demonstrated motion are strictly matched in the time dimension. The combination of these two aspects enables the model to both stably recover motion from noise and generate executable, near-realistic robot control commands.

[0043] Relying on the diffusion model, the robot can learn to autonomously complete tasks by using a small number of high-quality demonstration actions. It also exhibits a certain degree of generalization (object position generalization, object shape generalization, task scenario generalization). It is particularly suitable for flexible control tasks in variable environments, such as folding clothes and cleaning in a home environment. Furthermore, the adaptive force-controlled gripper of this invention can adaptively and with high precision adjust the gripping force, enabling autonomous and precise titration operations in a clean and sterile laboratory. Combined with a movable chassis and lifting device, it can be put into use directly without considering changes to the working environment or renovations.

[0044] After generating the predicted action sequence and trajectory, the environmental point cloud model is scanned to detect potential collision risks. Collision detection employs a combination of the mature algorithms GJK (Gilbert-Johnson-Keerthi) and CCD (Continuous Collision Detection) to determine whether the robot's trajectory comes into contact with obstacles in the environment or poses a potential collision risk. The GJK algorithm can quickly calculate the shortest distance between convex geometries, while CCD is used to detect whether the motion path crosses obstacle surfaces when the robot's actions change continuously. The combination of the two enables real-time collision assessment for discrete actions, continuous trajectories, and high-speed motion.

[0045] Specifically, the geometry of the robot's end effector, joints, or tools can be represented using convex hulls or simplified models, and obstacles in the environment can also be converted into convex bodies or approximated using several convex bodies. During collision detection, the system uses the GJK algorithm to calculate the minimum distance between each component of the robot and obstacles in its current state; if the minimum distance is less than a set safety margin, a potential risk is identified. In this invention, the safety margin δ_safe is set to 50 mm to trigger obstacle avoidance strategies before actual physical contact occurs. For example, when the shortest distance between the robot's end effector and a table or wall decreases to 50 mm, even before actual contact occurs, the system will still adjust collision avoidance based on this distance threshold, thereby improving the safety of task execution.

[0046] For situations involving large movement steps, rapid actions, or high-degree-of-freedom continuous trajectories, collision detection relying solely on instantaneous position may fail to capture "intersecting" collision risks. Therefore, this invention also employs CCD technology to detect continuous space between two adjacent action states of the robot. For example, during the robot arm's movement from position A to position B, even if neither state A nor B touches an obstacle, the CCD can still identify whether the movement path crosses the obstacle surface at an intermediate moment, thus avoiding missed detections caused by rapid movement. The combination of CCD and GJK ensures that the system can not only detect proximity collisions in static positions but also capture potential intersections during continuous movement.

[0047] The planned path is solved using a damped least squares inverse kinematics mechanism, and joint angle commands are output. In the main control module of this embodiment, the target pose of the robot's end effector is determined by the desired Cartesian space quantity. Representation (e.g., position and orientation). To enable the robotic arm to reach this end-effector pose, it is necessary to... Convert to the corresponding joint space increment or velocity command Since the kinematic relationship of a robotic arm is typically a nonlinear mapping K(q), representing the relationship between joint angle q and end-effector pose, it is necessary to solve for joint velocities using numerical inverse kinematics. Therefore, this invention employs the damped least squares (DLS) method to solve the inverse kinematics, ensuring stability in singular positions or ill-conditioned regions.

[0048] The objective of the damped least squares method can be described as: minimizing the weighted sum between the end-effector pose error and the joint velocity magnitude at the current joint angle q, which can be written in the form of: , Where K(q) is the current end pose, For the desired target pose, The joint velocity command to be solved. is the damping coefficient, used to suppress excessive velocities of the joint near singular points. The optimization objective can be understood as: to allow the end effector to follow the target pose as closely as possible, while avoiding drastic changes in joint velocity due to poor kinematic matrix condition numbers.

[0049] In engineering implementation, the Cartesian space error can be obtained by subtracting the target pose from the current pose. Then, the current Jacobian matrix J(q) of the robotic arm is used to represent the linear relationship between pose change and joint change. The joint velocity command given by the damped least squares solution is: .

[0050] Where JᵀJ is the quadratic form of the Jacobian matrix, adding... This term improves the matrix condition number, thereby avoiding solution instability caused by approaching singular regions. In this way, the present invention stably maps the end-effector pose error to the joint velocity command, making the joint trajectory smooth and controllable.

[0051] In the actual control cycle, the obtained The system will perform discrete integration based on the control frequency to obtain new joint angle commands. The robot's drive system updates the position of each joint accordingly, thereby gradually bringing the end effector closer to the target pose.

[0052] For example, when the target position of the robotic arm's end effector moves forward by 20 mm, this invention first calculates the error e between the current end effector position and the target position, determines the sensitive direction of this error in joint space using the Jacobian matrix, and obtains an appropriate joint angular velocity using the aforementioned damped least squares solution. This allows the end effector of the robotic arm to move in the correct direction while maintaining the stability and controllability of the joint movements.

[0053] After calculating the joint angle commands, they need to be converted into motor torque commands and issued to the motors. During the conversion of joint angle commands into motor torque commands, and throughout the robot's execution of actions, compliant control and dynamic feedforward compensation are continuously performed to adjust the final motor torque commands issued to the motors. The six-dimensional force sensor on the humanoid robot's wrist provides real-time feedback of contact force information, driving the impedance controller to dynamically adjust the joint stiffness (adjustable from 200-5000 N / m) and damping coefficient. When performing precision tasks (such as mating tolerances <0.05 mm), a high-stiffness mode (>4500 N / m) is activated. All joint commands generate motor torque commands through the dynamic feedforward compensation module (including gravity / Coriolis force / friction compensation), while the online parameter identifier (RLS algorithm) continuously updates the inertial parameters to compensate for model errors caused by load changes (accuracy up to ±1.5%).

[0054] A six-dimensional force sensor can simultaneously measure three-axis forces and three-axis torques acting on a robot's end effector, and its output includes: F_x, F_y, F_z: External forces along the X, Y, and Z directions. M_x, My, M_z: External torques about the X, Y, and Z axes The measurement of six-dimensional force and torque reflects the interaction between the end effector and the environment. For example, in contact tasks, F_z can reflect vertical pressure, M_x and My can characterize the tilting force on the end effector, and F_x / F_y is often used for lateral contact detection.

[0055] The information collected by the sensors is transformed into the end-effector coordinate system or joint coordinate system and input into the drive impedance / admittance controller for compliant control, which is used to adjust the dynamic behavior of the robotic arm.

[0056] The main idea of ​​impedance control is to establish a "virtual spring-damper-mass" model in Cartesian space, so that the end motion conforms to the following dynamic relationship: Where Δx = x_d - x: the deviation between the expected and actual end positions. M_d, B_d, K_d: Desired mass, damping, and stiffness parameters (all adjustable). F_ext: External force / torque measured by a six-dimensional force sensor Through impedance control, the greater the external force, the greater the end-effector offset, thus allowing the controller to automatically adjust the motion and produce compliant behavior. The desired mass, damping, and stiffness parameters can all be adaptively adjusted; for example, when the external force exceeds 10 N, the system reduces damping or stiffness to achieve a softer interaction.

[0057] Impedance or admittance models output a desired velocity or displacement change in Cartesian space. or To drive the robot, the parameters need to be converted into joint space quantities using the Jacobian matrix or damped least squares method. Torque commands for the motors are then generated through dynamic feedforward compensation.

[0058] In this embodiment, the generation of the robot's driving torque not only relies on classical dynamics feedforward compensation but also includes an online compensation mechanism for real-time correction of dynamic model errors. This compensation module does not employ the traditional recursive least squares method; instead, it uses a reinforcement learning policy network to continuously learn factors such as external force disturbances, friction changes, and modeling inaccuracies during task execution, thereby achieving online correction of dynamic model errors. This approach provides greater adaptability under complex environments and nonlinear contact conditions.

[0059] In the implementation process, the reinforcement learning module receives the robot's current joint angles, joint velocities, desired acceleration, external force information from the six-dimensional force sensors, and the execution deviation of the most recent action as state input information. Based on these real-time observations, the policy network outputs a compensation term to correct the feedforward torque. This compensation term is superimposed with the traditional dynamics feedforward term to form the final torque command. Since the policy network has learned how to reduce tracking errors, offset dynamic uncertainties, and improve compliance through the reward function during the training phase, its output can adaptively respond to the current environmental state.

[0060] The training process of the compensation network is typically carried out in a simulation environment. The system construction includes training scenarios involving dynamic disturbances, frictional changes, collisions, and sensor noise. The reward function uses minimizing joint angle tracking errors, stability after sudden changes in external forces, and smoothness of control inputs as optimization objectives, enabling the policy network to automatically adjust compensation terms according to actual task requirements. For example, when the robot's end effector contacts an unknown flexible object, causing an increase in six-dimensional force, the policy network can output a corresponding compensation torque within one control cycle, making the end effector exhibit more compliant motion. When the inertia estimation of a joint in the dynamic model has a large deviation, the compensation output of the policy network will gradually tend to offset the deviation, thereby maintaining trajectory tracking performance.

[0061] During the online execution phase, the reinforcement learning policy no longer updates parameters but runs as a real-time function. Its inputs include the actual joint torque, the predicted dynamic feedforward, feedback generated by the Jacobian matrix, end-effector information, and the current task context. The compensation output by the policy network is added to the dynamic feedforward and then combined with a feedback control term based on admittance or impedance control to form the final reference torque. This reference torque is fed into the motor drive module, becoming the motor's torque command.

[0062] The reinforcement learning compensation mechanism enables the control system of this invention to continuously respond to dynamic errors that cannot be accurately modeled during actual execution. For example, when changes in joint reducer clearance, friction coefficient, or load disturbances cause the trajectory error to increase, the policy network can adjust the compensation torque in real time to restore the robot's motion to the desired trajectory; when changes in environmental stiffness cause fluctuations in external forces, the compensation term can suppress vibrations and improve contact stability. This online compensation method has higher flexibility and adaptability than traditional parameter identification methods and is an important improvement of this invention in compliant control and dynamic execution.

[0063] Ultimately, the dynamics feedforward term, reinforcement learning compensation term, and impedance control feedback term together form the motor torque command, which is used to control the robot's movements, enabling it to maintain trajectory accuracy, compliance, and stability in complex scenarios.

[0064] During the robot's actions, the main control module also performs safety monitoring. If there is a 5nm error between the expected torque and the actual torque monitored by the motor, or if a collision is detected at the collision detection layer, an emergency stop command is triggered to protect the safety of the user and the robot.

[0065] This invention constructs conditional constraints through multimodal fusion, enabling the conditional representations built during condition diffusion to possess high abstraction and cross-modal consistency. This allows the action generation model to fully utilize the richness of visual information, the task-oriented nature of linguistic information, and the physical constraints of state information, thereby significantly improving the reliability, generalization ability, and task success rate of robot behavior generation. Through model inference using the diffusion model, the model starts with random noise actions and gradually generates conditional action predictions through multiple steps of back diffusion. These predicted actions can be directly sent as control commands to the robotic arm for execution. The model of this invention can generate continuous and stable robot actions based on visual and linguistic tasks and real-time states, thereby improving the robot's autonomous control capability under complex task conditions. In trajectory detection, by introducing a GJK-CCD joint detection mechanism and a safety margin δ_safe, this invention can monitor the spatial relationship between the robot and the environment in real time during action generation and trajectory execution, predicting and avoiding possible collisions in advance. This mechanism reduces the risk of accidents when the robot performs tasks in complex environments and improves the stability and safety of motion planning and control.

Claims

1. An intelligent composite humanoid robot, characterized in that, The system includes a mobile chassis with a lifting device on top; the top of the lifting device is connected to the torso of the humanoid robot; a robotic arm is located on each of the left and right sides of the humanoid robot; the humanoid robot integrates a main control module, a router module, and a sensor module; operators can perform remote development through the router module and control the robot's movements by transmitting data and VR remote control commands through the router module.

2. The intelligent composite humanoid robot according to claim 1, characterized in that, The robotic arm communicates with the main control module via Ethernet. The robotic arm's Ethernet cable is first connected to the router, and then combined with the router's Ethernet cable to connect to the main control module.

3. The intelligent composite humanoid robot according to claim 1 or 2, characterized in that, The robotic arm has a gripper at its far end, which communicates with the robotic arm via RS485. The main control module indirectly controls the gripper by controlling the robotic arm through control signals.

4. The intelligent composite humanoid robot according to claim 1, characterized in that, The sensor module includes several depth cameras, which communicate with the main control module via USB; depth cameras are also installed on the head, chest cavity, and two grippers of the robotic arm of the humanoid robot.

5. The intelligent composite humanoid robot according to claim 1, characterized in that, The lifting device uses a lead screw slide table, driven by a servo motor at the bottom. The lead screw slide table consists of a slider and a slide rail. A fixed ring is provided on the slider, and the humanoid robot is installed on the fixed ring. The lifting device controls the robot's height increase and decrease. The mobile chassis controls the robot's planar movement through a motor and wheels.

6. A control method for an intelligent composite humanoid robot, applied to the intelligent composite humanoid robot described in claims 1-5, characterized in that, Includes the following steps: S1. The operator uses VR to remotely control the robot and teach it to move. During the movement teaching, the VR system obtains the homogeneous transformation matrix of the VR controller relative to the VR headset in real time. S2. Perform inverse kinematic analysis on the taught movement and calculate the joint angle commands during the taught movement; S3. Input the joint angle command and sensor data into the diffusion model; S4. The diffusion model uses conditional diffusion to generate motion trajectories, performs motion calculations, and outputs joint angle commands. S5. The humanoid robot performs actions and continuously performs compliant control and kinematic feedforward compensation to adjust motor torque commands.

7. The intelligent composite humanoid robot control method according to claim 6, characterized in that, When generating actions using the diffusion model in S4, the specific steps include: S4-1. Collect low-dimensional data of the robot, including body perception data of the robotic arm and gripper, visual perception data collected by sensors, and verbal commands from the operator. Map the low-dimensional data to a unified feature space through an independent feature encoder, perform cross-modal fusion, and generate conditional vectors. S4-2. The forward diffusion model adds noise to the action, and then the conditional vector is used as the conditional input of the diffusion model for reverse denoising. The denoising direction is adjusted according to the conditional vector to generate the predicted action sequence and motion trajectory.

8. The intelligent composite humanoid robot control method according to claim 7, characterized in that, After generating the predicted action sequence and motion trajectory, the environmental point cloud model is scanned before the robot performs the action to detect whether there is a collision risk in the motion trajectory; if a collision risk is detected, the path is immediately replanned.

9. The intelligent composite humanoid robot control method according to claim 8, characterized in that, The planned path is solved using the damped least squares inverse kinematics solution mechanism, and joint angle commands are output.

10. The intelligent composite humanoid robot control method according to claim 6, characterized in that, In S5, the contact force information is fed back in real time by the six-dimensional force sensor at the wrist of the robotic arm, and the driving impedance controller dynamically adjusts the joint stiffness and damping coefficient to perform compliant control of the joint angle command. The adjusted joint angle command is then subjected to dynamic feedforward compensation, including gravity compensation, Coriolis force compensation, and friction compensation. The motor torque command is generated from the joint angle command and sent to the motor driver.

Images