Robot real-time remote control method and system based on motion capture

Abstract

The invention relates to the technical field of robot control, and provides a robot real-time remote control method and system based on motion capture, and the method comprises the steps: obtaining the joint motion information of an operator; performing rotary offset correction on the difference between a joint action information acquisition coordinate system and a robot coordinate system, setting a fixed offset for an elbow joint, dynamically generating and updating connecting rod and joint information of a virtual human body, updating a topological relation model between the joints, extracting a target angle of each joint, and calculating a target angle of each joint; all joint angles of the robot are updated step by step through smooth processing, and follow-up control over robot actions is achieved. According to the method, coordinate correction, topological modeling and proportional mapping are carried out on human body joint motion information, and smooth function processing is combined, so that efficient conversion from human body actions to robot actions is realized. According to the method, accurate modeling or a large amount of training data is not needed, low delay, high stability and good deployability are achieved, and the practicability and usability of remote control of the humanoid robot are remarkably improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of robot control. Specifically, it relates to a method and system for real-time remote control of a robot based on motion capture. Background Art

[0002] The statements in this section merely provide background technical information related to the present disclosure and do not necessarily constitute prior art.

[0003] In recent years, humanoid robot technology has developed rapidly, gradually showing its extensive application potential in multiple fields such as home service, medical assistance, education, entertainment interaction, and industrial automation. Due to its human-like body structure, humanoid robots have obvious advantages in intuitive interaction and natural motion imitation, making them ideal carriers for remote teleoperation tasks, especially suitable for scenarios such as disaster rescue, operation in dangerous environments, and complex precision manipulation. Therefore, researching how to achieve efficient, stable, and flexible control of humanoid robots through teleoperation has become one of the important directions in the field of robotics.

[0004] Existing methods for remotely controlling humanoid robots mainly include collecting human motion data based on RGB cameras, VR devices, or motion capture systems, and achieving humanoid control through motion remapping. However, these methods often rely on the accurate construction of the robot dynamics model or require a large amount of high-quality human motion data for reinforcement learning training to obtain stable and natural control strategies. On the one hand, methods based on end-effector and centroid control have problems such as strong model dependence and cumbersome parameter adjustment; on the other hand, although reinforcement learning methods have robustness and human-likeness, the training process is complex, resource-consuming, and not very user-friendly for non-professional users. Generally speaking, existing technical solutions have bottlenecks such as high operation thresholds, high implementation costs, and limited adaptability, making it difficult to meet the needs of beginners to quickly get started or small systems to be quickly deployed. Summary of the Invention

[0005] To solve the above problems, the present invention proposes a method and system for real-time remote control of a robot based on motion capture. By performing coordinate correction, topological modeling, and proportional mapping on human joint motion information, and combining smooth function processing, efficient conversion of human motions into robot motions is achieved. This method does not require accurate modeling or a large amount of training data, has low latency, high stability, and good deployability, significantly improving the practicality and usability of remotely controlling humanoid robots.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] One or more embodiments provide a method for real-time remote control of a robot based on motion capture, including the following steps:

[0008] Obtain the trunk segments and joint motion information of the operator who issues the control information;

[0009] Based on the received joint motion information, determine the relative positions of the joints according to the trunk segments, correct the rotational offset of the difference between the joint motion information acquisition coordinate system and the robot coordinate system, set a fixed offset for the elbow joint, dynamically generate and update the link and joint information of the virtual human body, and obtain an updated topological relationship model between the joints;

[0010] Based on the updated topological relationship model between the joints, extract the target angle of each joint, and gradually update the joint angles of the robot through smoothing processing to achieve the follow-up control of the robot's actions.

[0011] One or more embodiments provide a robot real-time remote control system based on motion capture, including:

[0012] A control action acquisition module, configured to obtain the trunk segments and joint motion information of the operator who issues the control information; wherein the joint motion information includes comprehensive data on the position, speed, and acceleration states of the joints;

[0013] A topological relationship model construction module between joints, configured to determine the relative positions of the joints according to the trunk segments based on the received joint motion information, correct the rotational offset of the difference between the joint motion information acquisition coordinate system and the robot coordinate system, set a fixed offset for the elbow joint, dynamically generate and update the link and joint information of the virtual human body, and combine the trunk segment information to obtain an updated topological relationship model between the joints;

[0014] A robot control module, configured to extract the target angle of each joint based on the updated topological relationship model between the joints, and gradually update the joint angles of the robot through smoothing processing to achieve the follow-up control of the robot's actions.

[0015] Compared with the prior art, the beneficial effects of the present invention are:

[0016] The present invention obtains the actual motion data of the operator by using a motion capture system, and establishes a joint topology model by means of coordinate system rotation offset and fixed offset correction, so as to realize action mapping without precisely constructing a robot dynamics model, reducing the difficulty of system construction and parameter debugging. The use of a rule-based mapping and smoothing function processing method to replace complex learning algorithms enables the control strategy to be directly derived from real-time data, avoiding the lengthy model training and debugging process, and significantly reducing the system development and deployment costs. Through the smoothing function processing, each joint of the robot smoothly transitions from the current state to the target angle state within continuous time. This smoothing processing effectively alleviates the problem of motion discontinuity that may be caused by untimely sensor data update, and realizes high-fidelity following control of human actions.

[0017] The advantages of the present invention and the advantages of additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute a limitation to the present invention.

[0019] Figure 1 is a flow framework diagram of the real-time remote control method of a robot based on motion capture according to Embodiment 1 of the present invention;

[0020] Figure 2 is a human body contour map of the data captured by Xsens in Rviz according to Embodiment 1 of the present invention;

[0021] Figure 3 is a schematic diagram of carrying an object in the MuJoCo simulation environment in a simulation experiment using the control method according to Embodiment 1 of the present invention;

[0022] Figure 4 is a schematic diagram of the shoulder pitch angle tracking of a robot when carrying an object in a simulation experiment using the control method according to Embodiment 1 of the present invention;

[0023] Figure 5 is a schematic diagram of using the control method according to Embodiment 1 of the present invention to carry an object with a physical robot;

[0024] Figure 6 is an actual robot angle tracking effect diagram without adding a smoothing function in a simulation experiment using the control method according to Embodiment 1 of the present invention;

[0025] Figure 7 is an actual robot angle tracking effect diagram with adding a smoothing function in a simulation experiment using the control method according to Embodiment 1 of the present invention; DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed description is exemplary and is intended to provide further illustration of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0028] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. It should be noted that, without conflict, the various embodiments and features in the present invention can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0029] Embodiment 1

[0030] In the technical solutions disclosed in one or more embodiments, as Figures 1 to 7 shown, a real-time remote control method for a robot based on motion capture includes the following steps:

[0031] Step 1: Obtain the trunk segments and joint motion information of the operator for manipulation; wherein the joint motion information includes comprehensive data on the position, speed, and acceleration states of the joints;

[0032] Step 2: Based on the received information, extract the joint motion information, correct the rotational offset of the difference between the joint motion information acquisition coordinate system and the robot coordinate system, set a fixed offset for the elbow joint, and dynamically generate and update the link and joint information of the virtual human body to obtain an updated topological relationship model between the joints;

[0033] Step 3: Based on the updated topological relationship model between the joints, extract the target angle of each joint, and gradually update the joint angles of the robot through smoothing processing to achieve the follow-up control of the robot's actions;

[0034] In the technical solution of this embodiment, the action capture system (such as Xsens Mvn) is used to obtain the operator's torso segment and joint action information, including position, speed, and acceleration data. These data are collected in real time by inertial sensors worn on the operator's key joint parts, ensuring the dynamic responsiveness and accuracy of the states of each joint. Then, the collected human joint data are mapped to the robot joint system. To solve the difference between the Xsens default coordinate system and the coordinate system of the virtual human model in the ROS system, a specific rotational offset is introduced to the arm link for correction, and a fixed offset is set for the elbow joint to eliminate its pose error. Subsequently, the link structure and joint states of the virtual human are dynamically generated and initialized, and the joint topological relationship model of the virtual human is constructed and updated to achieve the abstract expression and reconstruction of the operator's action structure. The target angles of each joint are extracted from the updated topological relationship model, and

[0035] In this implementation, the actual motion data of the operator are obtained by using the action capture system, and the coordinate system rotation offset and fixed offset correction methods are adopted to establish the joint topological model, so as to achieve action mapping without precisely constructing the robot dynamics model, reducing the difficulty of system construction and parameter debugging. The regular mapping and smoothing function processing methods are used to replace the complex learning algorithm, so that the control strategy can be directly deduced from the real-time data, avoiding the long model training and debugging process, and greatly reducing the system development and deployment costs. Through the smoothing function processing, each joint of the robot smoothly transitions from the current state to the target angle state within continuous time. This smoothing processing effectively alleviates the problem of motion discontinuity that may be caused by untimely sensor data update, and realizes the high-fidelity following control of human actions.

[0036] The implementation steps of the methods from step 1 to step 3 above are mainly controlled by the framework as Figure 1 shown, including using the motion capture system to collect human motion data, reprocessing these data in ROS, and finally mapping them to the robot joints; the data processing framework includes three parts:

[0037] The first part is the Xsens Mvn system, which is used to implement step 1. In this part, the operator wears the action capture system suit, and the image of the operator displayed in the Xsens Mvn simulation environment.

[0038] The data collected from Xsens Mvn are processed to extract detailed kinematic and inertial information related to various torso segments and joints, including comprehensive data capturing the position, speed, and acceleration states of multiple torso segments and joints;

[0039] Furthermore, to improve data transmission efficiency during transmission to the receiver computer, only necessary parameters are selected for transmission for subsequent data processing. Specifically, these include position and orientation data (quaternions), linear kinematics, angular kinematics, time encoding, and joint angle information.

[0040] Achievable, inertial sensors such as accelerometers, gyroscopes, and magnetometers are set on the wearable device of the Xsens Mvn system to obtain motion data in three-dimensional space in real time; by fixing the sensors on the joints of the operator, dynamic data of all joints and the torso can be captured and transmitted in real time;

[0041] In this embodiment, the Xsens Mvn system is adopted to integrate gyroscope data and accelerometer data to accurately estimate the position and orientation of the sensors.

[0042] Furthermore, the comprehensive data of the positions, velocities, and acceleration states of the torso segments and joints collected by the Xsens Mvn system are filtered using a Kalman filter to correct kinematic drift and the uncertainty of joint positions, and to achieve an accurate representation of joint positions and angles.

[0043] The second part is the ROS system, which is used to implement Step 2, reprocess these data in ROS, and finally map them to the robot joints. ROS (Robot Operating System) is an open-source middleware framework designed specifically for robot system development, providing a modular architecture with pre-integrated packages for perception, navigation, and motion planning. ROS provides a hardware abstraction layer that allows developers to write software that is not strictly associated with specific hardware components, making code reuse and platform switching easier. It also includes visualization and debugging tools that allow people to describe sensor data, robot states, and the interconnection of nodes in the graph.

[0044] In this embodiment, the data receiving thread and data processing thread of the ROS system are set as separate threads, so as to achieve fast client response and optimize data processing efficiency;

[0045] Optionally, the receiver computer set in the data receiving thread sets a UDP port to receive real-time human motion data through the UDP port; this ensures efficient communication within a restricted network environment.

[0046] Furthermore, in Step 2, the method for updating the topological relationship model between joints includes the following steps:

[0047] Step 21, by performing a rotational offset operation on the joint angles, align the Xsens default coordinate system with the virtual human model coordinate system in the ROS system;

[0048] The difference between the Xsens default coordinate system and the virtual human model coordinate system in the ROS system is solved by introducing specific rotational offsets to the joint angles;

[0049] Among them, the joints for which rotational offsets are introduced include the left arm roll angle joint and the right arm roll angle joint;

[0050] In step 21, rotational offset operations are performed on the joint angles to align the Xsens default coordinate system and the virtual human model coordinate system in the ROS system, including the following steps:

[0051] Step 211: Calculate the rotational offset based on the actual angle range of the roll angle of the actual human arm joints and the action angle range of the roll angle joints of the robotic arm to obtain the rotational offset angle value;

[0052] Step 212: In the virtual human model coordinate system, increase the rotational angle of the left arm roll angle joint by the rotational offset angle value and decrease the rotational angle of the right arm roll angle joint by the rotational offset angle value;

[0053] The data of the human joints collected by the Xsens Mvn system cannot be directly sent to the robot due to the inconsistency between the size of the human body and the size of the robot, which is likely to cause sub-collisions of the robot. Therefore, the joint data sent to the robot is processed by increasing or decreasing a certain angle to make it closer to the human motion posture. The joint flexibility of the actual robot is less than that of the actual human body, especially for the roll angle of the arm joints. When collecting the actions of the actual human body for robotic manipulation actions, collisions are likely to occur. To solve this problem in this embodiment, offset control is performed during the virtual modeling process, improving the stability and accuracy of the robot's following of human actions.

[0054] Step 22: After alignment, for the joint information sent by the Xsens Mvn system received, dynamically generate and initialize the joint connections and joint state information of the human body, and map the initial joint information to the virtual human model coordinate system of the ROS system to construct a topological relationship model between the joints;

[0055] Dynamically generate and initialize the joint connections and joint state information of the human body. Specifically, it includes the following steps;

[0056] Step 221: Obtain the joint angle information and the corresponding human link information;

[0057] As Figure 2 shown, specifically, determine the relative position relationship between the joints according to the trunk segment information of the human body, and then equivalent the joints of the human body to a link structure to obtain the human link information;

[0058] Step 222: Integrate the joint angle information and human body link information into a ROS-specific message type and publish it as a topic.

[0059] Step 223: Read the information of the published topic message, and use the tf coordinate transformation in Rviz to construct a human body model to achieve the dynamic generation and initialization of the human body in Rviz.

[0060] Among them, Rviz is a 3D visualization tool in the ROS (Robot Operating System) system.

[0061] Step 224: Set a fixed offset for the elbow joint to obtain an updated topological relationship model between joints.

[0062] Specifically, an additional adjustment is made to the elbow joint, and a fixed offset is used to align it with the human posture configuration. Specifically, it includes the following steps:

[0063] Step 231: Extract the first double-arm elbow joint posture data in the initial state of human manipulation and the second double-arm elbow joint posture data in the initial state of the robot, and calculate the elbow joint angle difference between the human body and the robot.

[0064] Step 232: According to the elbow joint angle difference, initialize the elbow joint angle in the topological relationship model so that the elbow joint angle in the topological relationship model is the same as the initial angle of the human body.

[0065] For example, in the initial state, the elbow joint angle of the robot is 90 degrees and is in a horizontal state. If the initial state of human manipulation is vertical, the initial angle in the topological relationship model needs to be increased by a set angle to be in a horizontal state.

[0066] As Figure 2 shown, it is the human body contour map mapped in Rviz of the data captured by the Xsens Mvn system in this embodiment. Each line segment is a human body link. In this embodiment, a mapping mechanism is first integrated to align the Xsens Mvn data with the robot system, and then by introducing a specific rotational offset and setting a fixed offset for the elbow joint, a consistent data representation is ensured, which helps with accurate subsequent analysis. This ensures a consistent data representation and helps with accurate subsequent analysis.

[0067] Step 23: Real-time obtain the joint action information in the manipulation information, update the constructed topological relationship model to obtain an updated topological relationship model between joints.

[0068] For a further technical solution, extract the key data from the updated topological relationship model between joints, package it and transmit it to the controlled robot end through topic communication.

[0069] In this embodiment, the ROS topic communication method is adopted for data exchange. This method supports the transmission of custom message types through the ROS network, effectively solving the compatibility problems related to traditional file-based systems.

[0070] In the above embodiment, the computer receiving the data processes the human body data using the Robot Operating System (ROS), removes unnecessary torso information, retains key joint information, and finally maps it to the robot joints. Moreover, the joint data is smoothed to make the movement of the robot smoother and more natural. By using the standardized ROS system architecture and wearable motion capture devices, combined with an intuitive joint control method, it can be quickly deployed and used without the user having professional knowledge in the field of robot control, improving the friendliness and practicality of the system.

[0071] The third part is the robot terminal, which is used to implement the method of extracting the target angle of each joint based on the updated topological relationship model between joints in step 3, and gradually updating the joint angles of the robot through the smoothing function to achieve the follow-up control of the robot's actions. The specific process is as follows;

[0072] ROS system topic communication provides flexibility and ease of use for distributed system integration, but it also introduces certain limitations in large-scale real-time applications. Specifically, during high-capacity data transmission, the system may have difficulty processing all incoming data simultaneously, resulting in potential latency and eventual data loss. The data received by the robot corresponds to the data n time steps ago. When new data arrives, the difference between consecutive values may become too large, leading to significant motor fluctuations, which may damage the motor.

[0073] To solve this problem, the smoothing function defined in the following formula is incorporated into the proposed method, and the joint angles of the robot are gradually updated through the smoothing function. The update formula is:

[0074] q tar = q init +(q e - q init )r

[0075] where q tar is the joint target angle, q init is the joint initial angle, q e is the joint desired angle; r is the control transition process parameter, which takes changing values and increases sequentially from 0 to 1 according to the set change amplitude, making the robot gradually transition from the initial angle to the target angle.

[0076] Specifically, when r is 1, it indicates that the robot has fully transitioned to the target angle; when r is 0, it means the robot remains in its initial state. The implemented time filtering algorithm effectively alleviates the discontinuity of the joint trajectory, significantly reduces sudden torque fluctuations, and ensures stable experimental execution.

[0077] To illustrate the effectiveness of this embodiment, simulation verification was carried out as follows;

[0078] The experimental platform uses the Unitree G1 humanoid robot, which has a modular bipedal configuration with independent upper and lower body kinematic chains. This architecture provides a high degree of kinematic redundancy, enhancing the flexibility of motion and manipulation tasks. The Unitree G1 platform integrates the NVIDIA Jetson Orin NX module, providing sufficient computing resources for real-time motion planning, and has a centralized interface hub with standardized connectors for joint actuators, multimodal sensory inputs, and gigabit Ethernet. This modular architecture supports rapid hardware reconfiguration while maintaining compatibility with industry-standard communication protocols, significantly reducing the barriers to custom robot system development.

[0079] The experimental architecture implemented a distributed computing framework, including two dedicated computers: a sending computer (Windows) for human motion capture based on Xsens Mvn and a receiving computer (Ubuntu Linux) for ROS-driven kinematic processing. These computers communicate through a UDP interface via a synchronized gigabit Ethernet connection, and hardware timestamps ensure time alignment between motion data acquisition and the robot control flow.

[0080] Through comprehensive evaluations in physical and simulated environments, the feasibility of the framework was verified. ros-bridge was implemented to facilitate communication between the robot control system and the motion data processing pipeline, enabling seamless human-robot motion transfer.

[0081] For simulation-based verification, the MuJoCo physics engine was configured with the URDF model provided by Unitree officially, eliminating the need for manual kinematic modeling. To replicate the stable constraints of the real world, a virtual rope mechanism was applied to the robot to compensate for the differences in ground reaction forces inherent in the simulated environment. The verification protocol adopted a standardized task scenario where the robot executes an object manipulation sequence. As Figure 3 shown, the robot successfully grasped and lifted a cuboid box located on the platform, demonstrating coordinated whole-body motion control under the proposed framework.

[0082] During the transportation process in the simulation scenario, the shoulder pitch angle joint information is recorded and printed. This angle is selected because, as the initial joint of the robotic arm, it bears greater forces, making it more suitable for verifying joint tracking performance. This can be seen in Figure 4 where the red dashed line represents the target position, and the blue solid line represents the actual motor position. As shown in Figure 4 , the joint tracking performance exhibits a high degree of accuracy, with the actual motor position closely following the target trajectory. The minimum deviation between the two curves indicates that the tracking control is effective, ensuring smooth and stable motion execution.

[0083] The proposed framework was further verified through physical deployment on the Unitree G1 humanoid platform. A dedicated Ethernet interface hub was deployed to facilitate network connectivity between the robotic system and the motion capture computer. Both nodes were assigned IP addresses within the same subnet to ensure efficient and low-latency UDP-based data transmission. During the experimental trials, an operator wearing an Xsens Mvn suit performed predefined motion primitives outside the robot's operating range. As shown in Figure 5 , this configuration successfully translated the human upper body kinematics into coordinated robotic manipulation sequences, including box grasping, vertical lifting, and basic logistics handling operations.

[0084] Although the ROS system serves as the central communication middleware for robot control, continuous human motion poses inherent challenges in maintaining high-frequency data processing and system stability. Insufficient update rates of control signals can lead to high-amplitude oscillations in the joint actuators, potentially causing mechanical fatigue and permanent damage to the drive components. To address this critical issue, a data persistence protocol was implemented: when new kinematic updates are unavailable, the most recently verified command packet is automatically retransmitted.

[0085] This keep-optimal-value strategy ensures uninterrupted control signal continuity, effectively decoupling the motion capture latency from the real-time actuator stability requirements. When new data is received, significant differences from the previous data can cause large accelerations of the motors, leading to instability of the robot. To mitigate this issue, a smoothing function is applied to the previous and current data, ensuring a smooth transition of the motors to the target position. The parameter r controls the tendency of the robot joint to transition from the current position to the target position, and its value changes from 0 to 1 as the joint data reaches the target position from the current position. The effectiveness of this method was evaluated by testing the roll angle joint of the robot's shoulder. An operator equipped with an Xsens motion capture system swung the shoulder at a constant speed and then recorded and visualized the position data of the shoulder's side inclination joint in Figure 6 and Figure 7 . In Figure 6In it, the joint information is not smoothed. It can be found that a horizontal line segment appears at the target position of the motor, which means that the joint data is not updated in a timely manner. The motor actually behaves as a protrusion, that is, the part highlighted by the green circle in the figure. As Figure 7 shown, the smoothed joint data presents significantly fewer horizontal segments, indicating a reduction in sudden changes in the joint motor output.

[0086] It can be seen that after adding the smoothing function, the data shows better performance. Experimental verification shows that introducing the smoothing function improves the motion performance of the robot to a certain extent.

[0087] This embodiment proposes a framework that eliminates the need for computationally intensive full-body dynamics modeling, while achieving natural human-to-robot motion transfer through kinematic-level mapping. To enhance system stability, a linear interpolation algorithm is integrated into the motion processing, effectively reducing sudden torque conversions and related actuator wear. It has been comprehensively verified through simulation-based analysis and the physical platform of the G1 robot deployment. Comparative experiments show that the motion smoothness and mechanical reliability have been significantly improved.

[0088] Embodiment 2

[0089] Based on Embodiment 1, this embodiment provides a robot real-time remote control system based on motion capture, including:

[0090] A manipulation action acquisition module, configured to acquire the torso segment and joint action information of the manipulator who issues the manipulation information; wherein the joint action information includes comprehensive data on the position, speed, and acceleration states of the joints;

[0091] A topological relationship model construction module between joints, configured to determine the relative positions of the joints based on the received joint action information according to the torso segment, perform rotation offset correction on the difference between the joint action information acquisition coordinate system and the robot coordinate system, set a fixed offset for the elbow joint, and dynamically generate and update the link and joint information of the virtual human body to obtain an updated topological relationship model between the joints;

[0092] A robot control module, configured to extract the target angle of each joint based on the updated topological relationship model between the joints, and gradually update the joint angles of the robot through smoothing processing to achieve follow-up control of the robot's actions.

[0093] It should be noted here that each module in this embodiment corresponds to each step in Embodiment 1, and its specific implementation process is the same, so it will not be repeated here.

[0094] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0095] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. A real-time remote control method for a robot based on motion capture, characterized in that, The method includes the following steps: Obtain the trunk segments and joint movement information of the operator who issues the control information; Based on the received joint movement information, determine the relative positions of the joints according to the trunk segments, correct the rotation offset of the difference between the joint movement information acquisition coordinate system and the robot coordinate system, set a fixed offset for the elbow joint, and dynamically generate and update the link and joint information of the virtual human model to obtain an updated topological relationship model between the joints; Based on the updated topological relationship model between the joints, extract the target angle of each joint, and gradually update the joint angles of the robot through smoothing processing to achieve the follow-up control of the robot's actions.

2. The real-time remote control method for a robot based on motion capture according to claim 1, wherein: Use the Xsens Mvn system to collect the trunk segments and joint movement information of the operator. Inertial sensors such as accelerometers, gyroscopes, and magnetometers are set on the wearable device of the Xsens Mvn system to obtain motion data in the three-dimensional space in real time.

3. The real-time remote control method of the robot based on motion capture according to claim 2, characterized in that: Filter the comprehensive data of the positions, velocities, and acceleration states of the trunk segments and joints collected by the Xsens Mvn system using a Kalman filter to correct the kinematic drift and the uncertainty of the joint positions.

4. The real-time remote control method for a robot based on motion capture according to claim 1, wherein: The method for updating the topological relationship model between the joints includes the following steps: Align the Xsens default coordinate system and the virtual human model coordinate system in the ROS system by performing a rotation offset operation on the joint angles; After alignment, dynamically generate and initialize the joint connections and joint state information of the human body for the received joint information, and map the initial joint information to the virtual human model coordinate system in the ROS system to construct a topological relationship model between the joints; Obtain the joint movement information in the control information in real time, and update the constructed topological relationship model to obtain an updated topological relationship model between the joints.

5. The real-time remote control method for a robot based on motion capture according to claim 4, wherein: The joints introduced with rotation offset include the left arm roll angle joint and the right arm roll angle joint; Performing a rotation offset operation on the joint angles to align the Xsens default coordinate system and the virtual human model coordinate system in the ROS system includes the following steps: Calculate the rotation offset according to the actual angle range of the roll angle of the actual arm joints of the human body and the action angle range of the roll angle joints of the robot arm to obtain the rotation offset angle value; In the virtual human model coordinate system, increase the rotation angle of the left arm roll angle joint by the rotation offset angle value, and decrease the rotation angle of the right arm roll angle joint by the rotation offset angle value.

6. The real-time remote control method of the robot based on motion capture according to claim 4, characterized in that: Dynamically generate and initialize the joint connections and joint state information of the human body. Specifically, it includes the following steps; Obtain the joint angle information and the corresponding human body link information; Integrate the joint angle information and the human body link information into a ROS-specific message type and publish it as a topic; Read the topic message information published, and use the tf coordinate transformation in ROS to construct a human body model to achieve the dynamic generation and initialization of the human body in Rviz; Adjust the elbow joint and use a fixed offset to align it with the human posture configuration, including the following steps: Extract the first double-arm elbow joint attitude data in the initial state of human body manipulation and the second double-arm elbow joint attitude data in the initial state of the robot, and calculate the elbow joint angle difference between the human body and the robot. According to the elbow joint angle difference, initialize the elbow joint angle in the topological relationship model so that the elbow joint angle in the topological relationship model is the same as the initial angle of the human body.

7. The real-time remote control method of the robot based on motion capture according to claim 1, characterized in that: Gradually update the joint angles of the robot through the processing of the smoothing function, and the update formula is: q tar = q init +(q e - q init )r where q tar is the joint target angle, q init is the joint initial angle, q e is the joint desired angle, r is the control transition process parameter, which takes values that increase successively in accordance with a set variation amplitude from 0 to 1, causing the robot to gradually transition from the initial angle to the target angle.

8. A real-time remote control system for a robot based on motion capture, characterized in that, Including: A manipulation action acquisition module configured to acquire the torso segment and joint action information of the manipulator who issues the manipulation information. A topological relationship model construction module between joints, configured to determine the relative positions of each joint according to the torso segment based on the received joint action information, correct the rotation offset of the difference between the joint action information acquisition coordinate system and the robot coordinate system, set a fixed offset for the elbow joint, and dynamically generate and update the link and joint information of the virtual human body to obtain an updated topological relationship model between joints. A robot control module configured to extract the target angle of each joint based on the updated topological relationship model between joints, and gradually update the joint angles of the robot through smoothing processing to achieve the follow-up control of the robot's actions.

9. The real-time remote control system for a robot based on motion capture according to claim 8, characterized in that, In the robot control module, the joint angles of the robot are gradually updated through the processing of the smoothing function, and the update formula is: q tar = q init +(q e - q init )r where q tar is the joint target angle, q init is the joint initial angle, q e is the joint desired angle, r is the control transition process parameter, which varies in value, increasing sequentially from 0 to 1 according to the set change amplitude, so that the robot gradually transitions from the initial angle to the target angle.

10. The real-time remote control system for a robot based on motion capture according to claim 8, characterized in that, In the topological relationship model construction module between joints, the update method of the topological relationship model between joints includes the following steps: Align the Xsens default coordinate system and the virtual human body model coordinate system in the ROS system by performing a rotation offset operation on the joint angle. After alignment, generate and initialize the joint connection and joint state information of the human body with the received joint information, map the initial joint information to the virtual human body model coordinate system of the ROS system, and construct a topological relationship model between joints. Real-time acquire the joint action information in the manipulation information, update the constructed topological relationship model, and obtain an updated topological relationship model between joints.

Images