Humanoid robot compliant collaborative handling method and system based on virtual safety mechanism

Abstract

The present application belongs to the field of robot carrying technology, and discloses a kind of based on virtual security mechanism humanoid robot compliant collaborative carrying method and system, the method includes steps: the automatic identification and spatial position accurate positioning of the object to be carried;Based on the identification and positioning result of the object to be carried, the position of the optimal space contact point of the dual-arm end effector of humanoid robot is calculated to execute carrying operation;The position of the optimal space contact point is dynamically adjusted, the motion trajectory of humanoid robot from starting state to complete carrying task is generated, and the virtual space constraint domain is constructed;The interactive force feedback information of humanoid robot and environment and the object to be carried is continuously perceived, and the motion posture and force degree of humanoid robot dual-arm are dynamically adjusted in real time.The present application realizes compliant anti-collision control in carrying process through visual and force perception fusion across modalities and high-reliability low-latency communication guarantee, and achieves the synchronous optimization of safety and work efficiency.

Description

Technical Field

[0001] This invention belongs to the field of robotics technology, specifically relating to robot handling technology, and more specifically to a humanoid robot compliant collaborative handling method and system based on a virtual safety mechanism. Background Technology

[0002] In the field of industrial material handling, traditional automated equipment is limited by the requirements of structured environments and is ill-suited for complex unstructured scenarios such as narrow passages, irregularly stacked goods, or dynamic production lines. While manual handling offers flexibility, it comes with high labor intensity, safety hazards, and high costs. Existing humanoid robots' compliant control strategies can passively adapt to accidental contact to some extent, but they generally lack proactive prediction and intrinsically constraining safety mechanisms for robot movement. They cannot provide reliable protection in dynamic processes involving close human-robot collaboration, thus limiting the actual effectiveness of humanoid robots in collaborative material handling tasks requiring high flexibility and safety. Summary of the Invention

[0003] Technical objective: To address the aforementioned technical problems, this invention proposes a humanoid robot compliant collaborative transport method and system based on a virtual safety mechanism.

[0004] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution: A compliant collaborative handling method for humanoid robots based on a virtual safety mechanism is executed on a remote operation platform and a humanoid robot that form a master-slave remote control system via a virtual gripper system. The method, executed on the remote operation platform, includes the following steps: Before the transport phase begins, the remote operation platform initiates a multi-view collaborative scanning mode, using multiple depth vision sensors integrated into the humanoid robot body to collect 3D data of the object to be transported from all directions, fuses the collected point cloud data and performs spatial pose calculation, and completes the automatic identification and precise spatial positioning of the object to be transported. Based on the identification and positioning results of the object to be transported, and combined with preset safety and operation rules, the teleoperation platform calculates the optimal spatial contact point for the humanoid robot's dual-arm end effectors to perform the transporting action. The operator on the teleoperation platform dynamically adjusts the position of the optimal spatial contact point through the screen in the cockpit according to the real-time operation situation, generating the humanoid robot's motion trajectory from the initial state to the completion of the transporting task, and constructing a virtual spatial constraint domain for the teleoperating arm. During the execution of the humanoid robot's trajectory, the remote operation platform continuously senses the interaction force feedback information between the humanoid robot and the environment and the object to be transported, and dynamically adjusts the movement posture and force of the humanoid robot's arms in real time based on the interaction force feedback information.

[0005] A further technical solution is that when the operator drives the humanoid robot's arms through the teleoperating arm, the teleoperating platform continuously monitors the spatial deviation between the actual trajectory of the humanoid robot's end effector and the preset path. If the spatial deviation exceeds the preset tolerance threshold, the virtual gripper triggers the adaptive force guidance model, that is, it applies a directional feedback force vector to the operator through the teleoperating arm. The direction of the directional feedback force vector points to the position of the object to be transported, and the intensity increases nonlinearly with the spatial deviation.

[0006] A further technical solution is that the remote operation platform performs high-precision spline interpolation on the discrete target pose transmitted by the humanoid robot to generate a continuous smooth trajectory of 1ms, and plans the velocity and acceleration constraint motion of the humanoid robot's joint space in real time based on the continuous smooth trajectory.

[0007] A further technical solution is that the remote operation platform executes an adaptive admittance force compliant control strategy, specifically including the following steps: The system updates and records the current external force information in real time during the humanoid robot's movement. When the degree of change in external force information exceeds a threshold, the external force information before the change is set as the desired force. Substituting this into the adaptive admittance control, the desired trajectory estimate is calculated. Environmental stiffness estimation The calculation formula is as follows: in, Indicates force error, , These are the initial estimates of environmental stiffness and the desired trajectory, respectively. , For adaptive parameters; This refers to the actual location of the target's end. Time interval; Indicates time; Based on the estimated reference trajectory and environmental stiffness, compliant control is performed, and the calculation formula is as follows: Where M is the mass coefficient, B is the damping coefficient, and K is the stiffness coefficient; and These are the actual velocity and the actual acceleration, i.e., the first derivative and the second derivative, respectively. These are the adaptive control coefficients.

[0008] A humanoid robot compliant cooperative transport method based on a virtual safety mechanism, wherein the humanoid robot performs the following steps: Driven by the remote operation platform, the humanoid robot travels to the target location, collects multi-visual information through its own camera, and sends it to the remote operation platform. The humanoid robot adjusts its posture under the control of the remote operation platform, including adjusting the upper body height, pitch angle and rotation angle. The humanoid robot uses the camera to identify the three-dimensional posture of the object to be transported in real time, automatically generates a virtual fixture constraint domain, and transmits it to the remote operation platform. The robotic arm of the humanoid robot moves objects under the control of the teleoperating arm in the teleoperating platform. The operating trajectory of the teleoperating arm is dynamically adjusted according to the collision risk and posture deviation perceived by the force feedback system. The force feedback system perceives the collision risk and posture deviation by synchronously integrating the environmental contact force and the virtual gripper guiding force. The humanoid robot's dual arms are equipped with high-precision six-dimensional force sensors that collect end-effector contact forces in real time. The raw data of the end-effector contact forces are processed by a gravity dynamic compensation algorithm to eliminate the influence of the tool's own weight at the end of the robotic arm. Then, the real environmental interaction force vector is calculated by a real-time transformation matrix from the sensor coordinate system to the world coordinate system. The environmental interaction force vector is fused with the virtual gripper's guiding force to generate an enhanced force feedback signal, which is then transmitted back to the remote operation platform. A further technical solution is that the enhanced force feedback signal is calculated using the following formula: in, For virtual clamping force, The final position of the target to be reached. This is the current target endpoint position. This is the stiffness coefficient. Threshold limit; in To provide feedback force to the teleoperated arm, This is the amplification factor for external environmental forces. As an external force of the environment, The clamping force weight; Indicates positional error. Represents the natural constant.

[0009] A humanoid robot compliant collaborative handling system based on a virtual safety mechanism, executing the method, is characterized in that: the system includes a teleoperation platform and a humanoid robot connected by communication, the teleoperation platform and the humanoid robot forming a master-slave telecontrol system through a virtual gripper system; the teleoperation platform is equipped with a teleoperation platform, a screen and a teleoperation arm, the teleoperation arm being operated by an operator.

[0010] A further technical solution is that the remote operation platform and the humanoid robot are connected via a 5G private network.

[0011] Beneficial effects: Due to the adoption of the above technical solution, the present invention has the following beneficial effects: The humanoid robot compliant collaborative handling method and system proposed in this invention, based on a virtual safety mechanism, deeply integrates the concept of active safety protection with a compliant control framework. It not only enables the humanoid robot to respond gently to the operation guidance of the remote operation platform and interact with the environment under the operation guidance, but also innovatively introduces a dynamic virtual safety mechanism. Through the virtual safety mechanism, safety boundaries are constructed and adjusted in real time, dynamically constraining robot behavior. It can also intuitively prompt operators of potential risks and pose deviations. Through the fusion of visual and force perception across modalities and high-reliability, low-latency communication, it can achieve compliant anti-collision control and load adaptive stability during the handling process, achieving simultaneous optimization of safety and work efficiency. Attached Figure Description

[0012] Figure 1 This is a flowchart of the humanoid robot compliant collaborative transport method based on a virtual safety mechanism proposed in Embodiment 1 of the present invention.

[0013] Figure 2 This is a schematic diagram of the humanoid robot compliant collaborative transport system based on a virtual safety mechanism proposed in Embodiment 2 of the present invention.

[0014] Figure 3 for Figure 2 3D example of a telemetry platform Figure 1 .

[0015] Figure 4 for Figure 2 3D example of a telemetry platform Figure 2 .

[0016] Figure 5 for Figure 4 A 3D view of a humanoid robot example.

[0017] Figure 6 for Figure 1 A schematic diagram of the dual-arm adaptive admittance force control algorithm used in the diagram.

[0018] In the diagram: 1-base, 2-screen mounting bracket, 3-screen, 4-cabin, 5-seat cushion, 6-seat, 7-left remote control arm, 8-remote control mounting bracket, 9-right remote control arm, 10-seat backrest, 11-throttle, 12-control buttons, 13-control screen, 14-steering wheel; 100-Lifting mechanism, 101-Waist rotation joint, 102-Chest camera, 103-Pitch joint, 104-Side camera, 105-Arm, 106-End camera, 107-Six-dimensional force sensor, 108-End tool, 109-Forward camera. Detailed Implementation

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] Example 1 Figure 1 This is a flowchart of the humanoid robot compliant cooperative transport method based on a virtual safety mechanism proposed in Embodiment 1 of the present invention. Figure 1 As shown, the humanoid robot compliant collaborative transport method based on virtual safety mechanism includes an initialization and preparation phase, as well as a subsequent collaborative transport execution phase between the robot end and the remote operation platform end, i.e., the operator end.

[0021] like Figure 1 As shown in this embodiment, during the initialization and preparation phase, the humanoid robot and the teleoperation platform interact. The humanoid robot is equipped with a processor capable of data processing. The humanoid robot includes a main arm and a slave arm that perform handling actions, and each arm has a corresponding main arm motor and slave arm motor. The slave arm's movements are set to be consistent with the main arm's movements; therefore, the following steps only describe the control of the main arm, while the corresponding operations are actually performed on the slave arm. The teleoperation platform includes a teleoperation platform, a teleoperation main arm, and a teleoperation slave arm. The teleoperation platform communicates with the humanoid robot to achieve information exchange. The teleoperation platform has a screen that displays real-time information about the humanoid robot, the object to be handled, and its environment. The teleoperation main arm and teleoperation slave arm are operated by the operator, simulating the main arm and slave arm of the humanoid robot.

[0022] Specifically, the initialization and preparation phase mainly includes the following steps: The remote control platform sends control commands to the humanoid robot to remotely control and initialize the robot. The control commands include the location of the object to be transported and the navigation path. The humanoid robot then follows the navigation path to find the object to be transported. The humanoid robot is equipped with multiple depth vision sensors. The remote control platform enables multi-view monitoring of the humanoid robot's walking path and surrounding environment. Based on the real-time environmental conditions, the humanoid robot is controlled to avoid obstacles and reach the target location. After the humanoid robot reaches the target location, the remote control platform can manually adjust the height and torso posture of the humanoid robot according to the target requirements. The remote operation platform utilizes multiple depth vision sensors integrated into the humanoid robot to automatically identify and spatially locate the target, generating the optimal contact point and the desired running trajectory; The remote operation platform constructs a virtual fixture and a virtual fixture constraint domain based on the position of the object to be moved and the desired motion trajectory.

[0023] It should be noted that virtual grippers are a software-generated control strategy in teleoperation and robotics. By defining abstract geometric rules or dynamic properties within the robot's task space, they can enhance, guide, or restrict the robot's motion trajectory, thereby achieving higher precision, safety, and efficiency in human-robot collaboration.

[0024] In this embodiment, a virtual gripper is constructed by creating a virtual repulsive field or impermeable boundary to prevent the robot from entering dangerous or restricted areas, such as a virtual protective semi-transparent barrier placed around vital organs to prevent collisions. Finally, the deviation between the desired motion and the actual motion is calculated in real time and converted into force / tactile feedback or direct motion correction using algorithms such as impedance control, which is then transmitted to the operator or robot controller. It should be noted that a virtual gripper refers to a complete technological paradigm generated by software to guide or restrict robot movement. It is not limited to spatial constraints but also includes various strategies such as guidance, assisted alignment, and compliant control.

[0025] In this embodiment, the virtual spatial constraint domain refers to the prohibitive or restrictive geometric elements used to define spatial boundaries in a virtual fixture, such as virtual walls, tunnels, planes, and enclosed volumes. Essentially, it is a specific type of virtual fixture whose core function is spatial isolation and restriction. In this embodiment, the virtual spatial constraint domain is a motion constraint boundary implemented in a teleoperation system based on virtual fixture technology. It defines the teleoperation platform end as the master end and the robot end as the slave end. Geometric rules (such as planes, curves, and volumes) are predefined in the task space or joint space of the slave-end humanoid robot. A preset algorithm can be used to restrict the motion degrees of freedom of the end effector of the slave-end humanoid robot to a specific path or region, thereby enhancing operational accuracy, avoiding singular configurations, and preventing collisions.

[0026] like Figure 1 As shown, in this embodiment, the collaborative transport execution phase mainly includes the following steps: The humanoid robot uses its integrated six-dimensional force sensor to collect the end contact force of its two arms in real time. The humanoid robot processes the collected end-effector contact forces by performing gravity compensation and coordinate transformation, converting them into forces from the real environment. Gravity compensation and coordinate transformation can be performed by the humanoid robot's own processor. The remote operation platform utilizes depth vision sensors integrated into the humanoid robot, such as cameras on the upper body, to transmit real-time multi-view operation scenes and control the movement of the remote operation main arm. The teleoperation platform calculates the end-effector pose of the teleoperated main arm based on positive motion, and then transmits the calculated end-effector pose of the teleoperated main arm. The current pose of the humanoid robot's main arm is compared with the end pose calculated by the teleoperation platform. Based on the comparison results, the movement of the humanoid robot's main arm is adjusted using teleoperation control. The remote operation platform uses an adaptive admittance force compliant control algorithm to calculate the virtual fixture guiding force. The processor configured on the humanoid robot further integrates the real environmental external force obtained from the aforementioned conversion with the virtual gripper guiding force to generate the final enhanced force feedback signal, and sends the generated enhanced force feedback signal to the remote operation platform. The remote control platform calculates the output torque of the main arm motor based on the dynamic model and the enhanced force feedback signal sent by the robot. Based on the calculated output torque, the remote control platform controls the humanoid robot's main arm to perform corresponding actions via a remote control arm.

[0027] Furthermore, in this embodiment, an adaptive admittance force compliance control algorithm is used to calculate the virtual gripper guiding force, with the aim of enabling the humanoid robot's main arm to comply with external forces when they occur.

[0028] This embodiment enables compliant human-robot collaborative handling: by designing a virtual safety mechanism that integrates real-time force feedback, the operator can intuitively perceive environmental contact forces and ideal operational guidance forces, achieving human-robot "empathy" and significantly improving the compliance and coordination of the handling process. Furthermore, it enables precise posture adjustment: combining multi-degree-of-freedom manual adjustment of the robot body with upper-body visual feedback, the robot can be quickly and accurately adjusted to the optimal working posture, and depth perception accurately identifies the object's pose, laying the foundation for successful handling. In addition, this embodiment significantly reduces operational difficulty and burden: by designing a virtual gripper with real-time fusion feedback of guiding forces and environmental forces, the difficulty and cognitive load of precise remote operation are significantly reduced, making complex handling tasks easier to control.

[0029] The humanoid robot proposed in this embodiment can provide operators with obstacle warnings and operational boundary protection without blind spots by using multi-view real-time monitoring of the chassis and upper body depth perception, combined with the constraint domain generated by the virtual fixture, effectively preventing collision risks and achieving all-round safety assurance.

[0030] Example 2 Figure 2 This is a schematic diagram of the humanoid robot compliant collaborative transport system based on a virtual safety mechanism proposed in Embodiment 2 of the present invention. Figure 2 As shown, in this embodiment, the system includes an integrated cockpit-type remote operation platform and a humanoid robot connected via a dedicated 5G communication network, forming a master-slave remote control system. The operator, through the integrated cockpit, leverages the ultra-low latency, high reliability, and high bandwidth links provided by the 5G private network to achieve cross-domain collaborative control of the robot's mobile platform and dual arms, thus realizing the compliant collaborative handling method described in Embodiment 1. The communication architecture provided in this embodiment effectively supports the needs for remote, real-time, multi-degree-of-freedom precision control and ensures smooth, low-latency transmission of large data such as environmental perception video streams. It overcomes the limitations of traditional limited connections or short-range wireless transmissions (such as WiFi) in terms of coverage, stability, and bandwidth. It can provide a safe and efficient solution that combines human decision-making flexibility with machine load capacity in industrial handling scenarios such as hazardous material transfer, heavy assembly, and flexible production lines.

[0031] Specifically, the integrated cockpit in this embodiment includes a main control module and an operation module. The main control module includes a motion controller, a PC industrial computer, a data processing unit, and various acquisition devices. The operation module includes the cockpit body, a screen, various operating mechanisms (buttons, switches, accelerator pedal), a main control robotic arm, etc.

[0032] Figure 3 for Figure 2 3D example of a telemetry platform Figure 1 . Figure 4 for Figure 2 3D example of a telemetry platform Figure 2 .like Figure 3 and Figure 4 As shown, the remote control platform includes a base 1, a screen mounting bracket 2 on the base 1, a screen 3 at the top of the screen mounting bracket 2, a control screen 13, control buttons 12, and a steering wheel 14 at the bottom of the screen mounting bracket 2, and an accelerator 11 at the bottom of the screen mounting bracket 2. A cockpit 4 is located on the base 1 opposite the screen mounting bracket 2. A seat 6 and a seat cushion 5 are located above the cockpit 4. The seat 6 has a seat back 10, and a remote control mounting bracket 8 is mounted on the seat back 10. A remote control left arm 7 and a remote control right arm 9 are located on both sides of the remote control mounting bracket 8.

[0033] Furthermore, Figure 5 for Figure 3 A 3D view of a humanoid robot example. (See image below.) Figure 5As shown, in this embodiment, the humanoid robot has a chassis with wheels at its bottom, and a side camera 104 and a front camera 109 are provided on the outer wall of the chassis. A lifting mechanism 100 for driving the humanoid robot to rise and fall is installed on the top of the chassis. The humanoid robot itself includes a waist rotation joint 101, a pitch joint 103, and two arms 105. Each arm 105 is equipped with an end effector 108, an end effector camera 106, and a six-dimensional force sensor 107. A chest camera 102 is provided at the front end of the humanoid robot.

[0034] In this embodiment, multiple depth vision sensors can be deployed at key nodes of the humanoid robot's chassis to form a multi-channel high-definition visual sensing system. This includes six independent channels: forward, backward, left front / rear corner, and right front / rear corner. The system is activated by the operator. Once activated, the real-time video streams captured by each camera are transmitted via a 5G private network to the remote operation platform within the cockpit. Using multi-channel video synchronous transmission and processing technology, the six-channel image streams are rendered with low latency and intelligently combined and presented on screen 3, generating a 360° surround-view monitoring interface with panoramic perception capabilities. This interface provides the operator with blind-spot-free visual coverage of the external environment, laying the foundation for safe movement and situational awareness.

[0035] Furthermore, the humanoid robot compliant cooperative transport system based on a virtual safety mechanism proposed in this embodiment executes the compliant cooperative transport method proposed in Embodiment 1, which mainly consists of three steps: S1, Chassis Navigation Control: The operator drives the humanoid robot to move by using the throttle 11 and steering wheel 14. Multiple cameras of the humanoid robot push video streams to the screen 3 in the cockpit 4 in real time to assist in omnidirectional obstacle observation and ensure safe navigation to the target location.

[0036] S2. Robot Body Posture Adjustment: After the humanoid robot arrives at the target position, the operator triggers the control button 12 inside the cockpit 4 to manually adjust the upper body height, pitch angle, and rotation angle of the humanoid robot. Combined with the perspective feedback from the humanoid robot's chest camera 102, the humanoid robot's body posture is precisely adjusted to the working adaptation state.

[0037] S3. Force Feedback Teleoperated Handling: After the humanoid robot's posture is adjusted, the chest camera 102 identifies the 3D posture of the object to be handled in real time and automatically generates a virtual gripper constraint domain. When the operator controls the movement of the robotic arm through the teleoperated arm, the force feedback system synchronously integrates the forces of the real environment with the guiding force of the virtual gripper, allowing the operator to directly perceive the collision risk and posture deviation, and dynamically correct the operation trajectory accordingly until it stabilizes and the handling is achieved.

[0038] Furthermore, in step S1 of this embodiment, namely the chassis navigation control step, the operator can realize the navigation and obstacle avoidance control of the humanoid robot through multi-view monitoring.

[0039] Specifically, the operator uses the steering wheel 14 and throttle 11 within the cockpit 4 to establish a precise closed-loop control input for the humanoid robot's direction and speed. While driving the humanoid robot towards the object to be transported, i.e., the predetermined transport coordinate point, the operator continuously monitors the real-time environmental images provided by the multi-view monitoring system on screen 3, paying particular attention to the relative distance changes and potential movement trends of obstacles in each independent viewpoint. Based on this multi-dimensional visual feedback information, the operator dynamically assesses the safety of the current path and instantly fine-tunes and corrects the humanoid robot's trajectory through control inputs (including steering wheel steering, throttle acceleration / deceleration, etc.). This human-machine collaborative decision-making and control mechanism based on real-time environmental perception effectively avoids static and dynamic obstacles on the route, ultimately ensuring that the robot can safely and accurately reach the target transport location.

[0040] Furthermore, in step S2 of this embodiment, during the movement of the humanoid robot, the operator can trigger a preset height adjustment command in the cabin 4 based on the estimated size characteristics of the object to be transported. This command drives the lifting actuator integrated into the robot body to work, thereby achieving height adjustment of the humanoid robot.

[0041] Specifically, the humanoid robot's lifting mechanism 100 responds to commands, causing the upper body of the humanoid robot to move vertically within a preset safe working height range (e.g., 1.5 meters to 2.2 meters). During height adjustment, the operator needs to monitor the visual images transmitted back by the humanoid robot's chest camera 102 in real time. By observing the changes in the relative positional relationship between the object to be moved and the humanoid robot's end effector 108 in the image, the operator can intuitively judge whether the current height of the humanoid robot is suitable for subsequent grasping operations, and continuously adjust it until the optimal working height that meets the requirements of the grasping task is reached, thus achieving spatial adaptation adjustment between the humanoid robot and the work object.

[0042] In step S2 of this embodiment, namely the robot body posture adjustment step, after the initial positioning of the humanoid robot's body height is completed, the operator uses the dedicated control button 12 in the cockpit 4 to further fine-tune the spatial posture of the humanoid robot's torso.

[0043] Specifically, the spatial pose fine-tuning process includes two key degrees of freedom: 1. Pitch angle adjustment of the torso around the horizontal axis; 2. Horizontal rotation correction of the torso around the vertical axis. After the operator issues a pitch or rotation command, the corresponding joint drive system of the humanoid robot begins to move, causing the torso structure to produce the desired posture change. In order to accurately evaluate and confirm the posture adjustment effect, the operator needs to simultaneously switch and call up the real-time operation scene transmitted by multiple different visual cameras on the humanoid robot on the screen 3 of the cockpit 4 according to the operation requirements, such as the main view image transmitted by the chest camera 12 and the real-time operation scene transmitted by the end cameras 106 installed at the ends of the arms. By comprehensively observing these real-time images transmitted from different perspectives, especially the position, orientation and occlusion of the object to be transported in the image, the operator continuously performs posture fine-tuning. The goal of the adjustment is to make the object to be transported stably located in the center recognition area of ​​the main operation camera, i.e., the chest camera 12, and to minimize the obstruction of the field of view, thereby laying the optimal visual perception foundation for subsequent accurate recognition, positioning and grasping operations, and completing the final operation posture calibration closed loop. Height adjustment and pitch and rotation correction can also be made during placement.

[0044] Furthermore, step S3 of this embodiment, namely the force feedback teleoperation handling step, specifically includes the following steps: S3.1 Optimal spatial contact point generation; S3.2, Virtual fixture generation; S3.3, Force Feedback Handling; S3.4 Adaptive admittance compliant control.

[0045] Specifically, in step S3.1, the process for generating the optimal spatial contact point is as follows: Before the transport phase begins, the remote operation platform automatically completes the spatial calibration of the working environment using multiple depth vision sensors integrated into the robot body. That is, the operator starts the multi-view collaborative scanning mode, and these sensors collect three-dimensional data of the object to be transported from all directions. The remote operation platform automatically fuses the collected point cloud data and performs spatial pose calculation to complete the automatic identification and precise spatial positioning of the object to be transported.

[0046] S3.12. Based on the automatic identification and positioning results of the object to be transported, and combined with preset safety and operation rules, the remote operation platform automatically calculates the optimal contact point position for the end effector 108 of the humanoid robot's dual arms to perform the transport action. The preset safety and operation rules refer to the optimal operation path training simulation results provided in advance for specific tasks. For example, in the operation of turning a valve, the optimal turning path planning will be provided, thereby outputting safety and operation rule prompts.

[0047] S3.13. Based on the real-time operation, the operator uses the teleoperated arm on screen 3 of the cockpit 4 to dynamically fine-tune the position of the optimal contact point, ensuring that the humanoid robot can safely and effectively perform the transport. Subsequently, the teleoperation platform plans and generates a safe and smooth motion trajectory from the initial state to the completion of the transport task, i.e., the desired motion trajectory, based on the final determined position reference and fine-tuned posture.

[0048] Specifically, in step S3.2, the process for generating the virtual fixture is as follows: S3.21. Using the calibrated coordinates of the object to be moved as a reference, the remote operation platform end constructs virtual spatial constraint domains for the remote operation left arm 7 and remote operation right arm 9 of the remote operation platform end respectively, and generates virtual fixtures.

[0049] S3.22. During the process of the humanoid robot performing the desired motion trajectory, the remote control platform continuously senses the interaction force feedback information between the humanoid robot and the environment or the object to be transported. Based on the interaction force feedback information, the remote control arm in the cockpit 4 dynamically adjusts the motion posture and force of the humanoid robot's two arms to ensure the smooth execution and stable completion of the transport action, forming a closed-loop correction of posture and force control throughout the process.

[0050] S3.23. When the operator drives the humanoid robot's arms via the teleoperated arm within the cockpit 4, the teleoperation platform continuously monitors the spatial deviation between the actual trajectory of the humanoid robot's end effector and the preset path. If the detected spatial deviation exceeds a preset tolerance threshold, the virtual gripper will trigger an adaptive force guidance model, applying a directional feedback force vector to the operator via the teleoperated arm. This directional feedback force vector always points towards the object to be transported, and its intensity increases non-linearly with the spatial deviation. This improves the spatial convergence efficiency and error prevention capabilities for precision operations in complex environments without restricting the teleoperated arm's degrees of freedom.

[0051] In this step, since the virtual fixture is a virtual detection mechanism, the main content of the mechanism is to make a threshold judgment based on the set trajectory. When a certain threshold is exceeded (such as trajectory error value or torque error value), the virtual fixture takes effect and performs position and force constraints in real time. Therefore, the virtual fixture can trigger the adaptive force guidance model.

[0052] Specifically, in step S3.3, the force feedback transport process is as follows: S3.31 During remote operation from the teleoperation platform, the high-precision six-dimensional force sensors equipped on the humanoid robot's dual arms collect end-effector contact force information in real time. The raw data is processed by a gravity dynamic compensation algorithm to eliminate the influence of the tool's own weight at the end of the robotic arm. Then, through a real-time transformation matrix from the sensor coordinate system to the world coordinate system, the actual environmental interaction force vector is calculated. This vector is fused with the virtual gripper's guiding force to generate an enhanced force feedback signal, which is then transmitted back to the teleoperation platform via the aforementioned high-speed, low-latency communication link.

[0053] The formula for calculating the enhanced force feedback signal is as follows: in, For virtual clamping force, For the target end position, This is the current target endpoint position. This is the stiffness coefficient. This is a threshold limitation. Here, the target refers to the object to be moved.

[0054] in To provide feedback force to the teleoperated arm, This is the amplification factor for external environmental forces. As an external force of the environment, This represents the clamping force weight.

[0055] S3.32. The operator simultaneously observes the multi-view operation scene transmitted back by multiple cameras of the humanoid robot, and combines it with the real-time tactile force perception information received by the telemanipulator to perform multimodal collaborative adjustment of the end effector posture, contact force, and movement speed during the handling process. Through the fusion of visual and force perception across modalities and the guarantee of high-reliability, low-latency communication, compliant anti-collision control and load adaptive stabilization are achieved during the handling process, realizing the simultaneous optimization of safety and operational efficiency.

[0056] Specifically, in step S3.4, the adaptive admittance force compliance control process is as follows: S3.41. During teleoperation, the humanoid robot may accidentally come into contact with the environment. To prevent damage caused by excessive interaction forces from rigid contact, the teleoperation platform integrates an adaptive admittance force control algorithm on top of teleoperation. This compliant control strategy enables the humanoid robot to compliantly follow the direction of unexpected external forces. Once the external force disappears, the system can seamlessly resume accurate tracking of teleoperation commands.

[0057] Figure 6 for Figure 1 A schematic diagram of the dual-arm adaptive admittance force control algorithm used. (See diagram below.) Figure 6 As shown, in this embodiment, the steps of the dual-arm adaptive admittance force control algorithm are as follows: During the motion, the current force information is updated and recorded in real time. When the degree of change in external force information exceeds a threshold, the external force information before the change is set as the desired force. Substituting this into adaptive admittance control, the desired trajectory estimate is calculated. Environmental stiffness estimation The calculation formula is as follows: in , To estimate the initial value, , These are adaptive parameters. Indicates a time interval.

[0058] Based on the estimated reference trajectory and environmental stiffness, compliant control is performed, and the calculation formula is as follows: Where M is the mass coefficient, B is the damping coefficient, and K is the stiffness coefficient, all of which are constant matrices. The adaptive control coefficients have the following update rates: in, To prevent the denominator from being 0 in the formula, This refers to the update rate. This represents the adaptive parameter at the current time t.

[0059] This design effectively addresses situations where the humanoid robot accidentally comes into contact with the environment during teleoperation. It prevents damage caused by excessive interaction forces from rigid contact, and by introducing a compliant control strategy, the humanoid robot can smoothly follow the direction of unexpected external forces. Once the external force disappears, the system seamlessly resumes accurate tracking of teleoperation commands.

[0060] This invention refines the method of Embodiment 1, employing a virtual safety mechanism to ensure compliant execution and stable completion of handling actions. It establishes a closed-loop correction system for posture and force control throughout the entire process, improving the spatial convergence efficiency and error prevention capabilities of humanoid robots in complex environments without restricting operational freedom. The operator simultaneously observes the multi-view operational scene transmitted back by the humanoid robot, combining it with real-time tactile force perception information received by the teleoperated arm. This allows for multimodal collaborative adjustment of the end-effector posture, contact force, and movement speed during handling, enabling simultaneous teleoperation of the master robot while ensuring compliant force control of the slave robot. Real-time trajectory correction based on the actual human teleoperation trajectory ensures human-machine safety during master-slave teleoperation. Furthermore, through the fusion of visual and force perception across modalities and high-reliability, low-latency communication, compliant collision avoidance control and adaptive load maintenance are achieved during handling, simultaneously optimizing safety and operational efficiency.

[0061] Furthermore, in this embodiment, the teleoperation platform and the humanoid robot preferably achieve cross-domain teleoperation communication based on a low-latency 5G private network, establishing a high-speed data link between the teleoperation platform and the humanoid robot through the 5G private network. At the master end, i.e., the cockpit, control commands are transmitted via the Controller Area Network (CAN) bus. At the humanoid robot end, actuator response commands are driven via the Real-Time Ethernet (EC) bus. However, due to the inconsistency in the communication protocols used by the master and slave ends—a mismatch between the 6ms low-speed data stream of the master end and the 1ms high-speed control of the slave end's EtherCAT—this embodiment employs a motion interpolation and trajectory planning collaborative optimization method to address this issue. Specifically, the preset motion trajectory includes multiple discrete target poses. The humanoid robot performs high-precision spline interpolation on the received discrete target poses to generate a 1ms continuous smooth trajectory, and simultaneously plans the velocity and acceleration constraint motion of each joint space in real time based on this continuous smooth trajectory.

[0062] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A compliant collaborative handling method for humanoid robots based on a virtual safety mechanism, wherein the method is executed on a remote operation platform and a humanoid robot that form a master-slave remote control system through a virtual gripper system; characterized in that, The method is executed on the remote operation platform and includes the following steps: Before the transport phase begins, the remote operation platform initiates a multi-view collaborative scanning mode, using multiple depth vision sensors integrated into the humanoid robot body to collect 3D data of the object to be transported from all directions, fuses the collected point cloud data and performs spatial pose calculation, and completes the automatic identification and precise spatial positioning of the object to be transported. Based on the identification and positioning results of the object to be transported, and combined with preset safety and operation rules, the teleoperation platform calculates the optimal spatial contact point for the humanoid robot's dual-arm end effectors to perform the transporting action. The operator on the teleoperation platform dynamically adjusts the position of the optimal spatial contact point through the screen in the cockpit according to the real-time operation situation, generating the humanoid robot's motion trajectory from the initial state to the completion of the transporting task, and constructing a virtual spatial constraint domain for the teleoperating arm. During the execution of the humanoid robot's trajectory, the remote operation platform continuously senses the interaction force feedback information between the humanoid robot and the environment and the object to be transported, and dynamically adjusts the movement posture and force of the humanoid robot's arms in real time based on the interaction force feedback information.

2. The humanoid robot compliant cooperative transport method based on a virtual safety mechanism according to claim 1, characterized in that: When the operator drives the humanoid robot's arms via the teleoperated arm, the teleoperation platform continuously monitors the spatial deviation between the actual trajectory of the humanoid robot's end effector and the preset path. If the spatial deviation exceeds the preset tolerance threshold, the virtual gripper triggers the adaptive force guidance model, which applies a directional feedback force vector to the operator via the teleoperated arm. The direction of the directional feedback force vector points to the position of the object to be transported, and its intensity increases non-linearly with the spatial deviation.

3. The humanoid robot compliant cooperative transport method based on a virtual safety mechanism according to claim 1, characterized in that: The remote operation platform performs high-precision spline interpolation on the discrete target pose transmitted by the humanoid robot to generate a continuous smooth trajectory of 1ms, and plans the velocity and acceleration constraint motion of the humanoid robot's joint space in real time based on the continuous smooth trajectory.

4. A humanoid robot compliant cooperative transport method based on a virtual safety mechanism, characterized in that, The humanoid robot performs the following steps: Driven by the remote operation platform, the humanoid robot travels to the target location, collects multi-visual information through its own camera, and sends it to the remote operation platform. The humanoid robot adjusts its posture under the control of the remote operation platform, including adjusting the upper body height, pitch angle and rotation angle. The humanoid robot uses the camera to identify the three-dimensional posture of the object to be transported in real time, automatically generates a virtual fixture constraint domain, and transmits it to the remote operation platform. The robotic arm of the humanoid robot moves objects under the control of the teleoperating arm in the teleoperating platform. The operating trajectory of the teleoperating arm is dynamically adjusted according to the collision risk and posture deviation perceived by the force feedback system. The force feedback system perceives the collision risk and posture deviation by synchronously integrating the environmental contact force and the virtual gripper guiding force. The humanoid robot's dual arms are equipped with high-precision six-dimensional force sensors that collect end-effector contact forces in real time. The raw end-effector contact force data is processed by a gravity dynamic compensation algorithm to eliminate the influence of the tool's own weight at the end of the robotic arm. Then, the real environmental interaction force vector is calculated by a real-time transformation matrix from the sensor coordinate system to the world coordinate system. The environmental interaction force vector is fused with the virtual gripper's guiding force to generate an enhanced force feedback signal, which is then transmitted back to the remote operation platform.

5. The humanoid robot compliant cooperative transport method based on a virtual safety mechanism according to claim 4, characterized in that: The formula for calculating the enhanced force feedback signal is as follows: in, For virtual clamping force, The final position of the target to be reached. This is the current target endpoint position. This is the stiffness coefficient. Threshold limit; in To provide feedback force to the teleoperated arm, This is the amplification factor for external environmental forces. As an external force of the environment, The clamping force weight; Indicates positional error. Represents the natural constant.

6. A humanoid robot compliant cooperative transport system based on a virtual safety mechanism, performing the method described in any one of claims 1-5, characterized in that: The system includes a teleoperation platform and a humanoid robot connected by communication. The teleoperation platform and the humanoid robot form a master-slave telecontrol system through a virtual gripper system. The teleoperation platform is equipped with a teleoperation platform, a screen, and a teleoperation arm, which is operated by an operator.

7. The humanoid robot compliant cooperative transport system as described in claim 6, characterized in that: The remote operation platform and the humanoid robot are connected via a 5G private network.

Images