Robot remote control system and method, storage medium and computer program product

Abstract

The invention provides a robot remote control system. The robot remote control system comprises VR head-mounted display equipment; an IMU inertial sensor; the data processing module is used for carrying out mapping calculation on parameters acquired by the VR head-mounted display device and the IMU inertial sensor to obtain corresponding real-time target position, posture and arm angle parameters of the tail end of the mechanical arm; the transmission processing module is used for sending the obtained real-time target position, posture and arm angle parameters of the tail end of the mechanical arm to a remote robot; and the mechanical arm adjusting module is used for receiving the parameters sent by the data processing module and adjusting a mechanical arm of the robot, so that the mechanical arm and an arm of an operator are synchronized. The invention further discloses a corresponding method, a storage medium and a computer program product. According to the invention, the precision and the degree of freedom of movement of remote control can be improved, and the device can be applied to a narrow operation space.

Description

Technical Field

[0001] The present invention relates to the technical field of robot remote operation, and in particular to a robot remote control system, method, storage medium and computer program product. Background Art

[0002] Robot remote operation technology is mainly used in specific dangerous scenarios to reduce the risk of casualties. Depending on the sensor, the remote operation system can be divided into two categories: vision-based and inertia-based. Although visual remote operation can improve the accuracy of operation, it will sacrifice a certain degree of freedom; while inertia-based remote operation system provides a higher degree of freedom, but the accuracy is relatively reduced.

[0003] At present, inertial motion capture methods are commonly used in the prior art to control robotic arms. This method uses an inertial measurement unit (IMU) worn on the arm to obtain posture information, and then calculates the joint angles of the robotic arm to achieve the purpose of remote control of the robotic arm.

[0004] In addition, there are also some solutions that combine motion capture systems with IMUs in the prior art. In this solution, the motion capture system is responsible for providing the position and posture information of each joint of the human body, while the IMU is used to solve the problem when the marker points are blocked. Therefore, the main information of remote control still depends on the provision of the motion capture system.

[0005] However, the existing technology still has some shortcomings. For remote control solutions that rely solely on inertial sensors, their accuracy is highly dependent on the accuracy of the human body model, and the human body data parameters need to be remeasured for different users, and the operation process is cumbersome. At the same time, although the method of combining the motion capture system with the IMU can reduce spatial limitations, motion capture still needs to be completed in a specific area, and the cost is high. In addition, whether it is a vision-based or inertial-based system, it may be affected by factors such as the environment and equipment performance, resulting in challenges to the stability and reliability of remote operation. Summary of the invention

[0006] The technical problem to be solved by the present invention is to propose a robot remote control system, method, storage medium and computer program product, which can improve the accuracy and freedom of movement of remote control and can be applied to narrow operating spaces.

[0007] As one aspect of the present invention, there is provided a robot remote control system, which at least comprises:

[0008] VR head-mounted display device, used to obtain the position and posture information of the end of the operator's arm in real time;

[0009] IMU inertial sensor, used to obtain the arm angle data of the operator in real time;

[0010] The data processing module is used to map and calculate the position, posture information, and arm angle data of the end of the operator's arm to obtain the corresponding real-time target position, posture, and arm angle parameters of the end of the robotic arm:

[0011] The transmission processing module is used to send the real-time target position, posture and arm angle parameters of the end of the robotic arm calculated by the data processing module to the remote robot;

[0012] The robot arm adjustment module is used to receive the real-time target position, posture and arm angle parameters sent from the data processing module, and adjust the robot's robot arm according to the parameters to synchronize the robot arm with the operator's arm.

[0013] Wherein, the data processing module further includes:

[0014] The position mapping processing unit is used to map the position information of the end of the operator's arm obtained by the VR head display device in real time using the following position mapping formula to obtain the real-time target position parameters of the end of the robotic arm:

[0015] P 主,t1 =P 主,t0 +K·ΔP 主,t1

[0016] P 从,t1 =P 从,t0 +K·ΔP 从,t1

[0017] Among them, P 从,t0 is the position parameter of the end of the robot arm at the initial moment, P 从,t1 is the target position parameter of the end of the robot arm at the current moment, ΔP 从,t1 is the position change of the end of the robot arm at the current moment; P 主,t0 is the position parameter of the handle at the initial moment, P 凡,t1 is the target position parameter of the handle at the current moment, ΔP 主,t1 is the change in the handle position at the current moment, and K is the mapping coefficient;

[0018] The posture mapping processing unit is used to map the posture information of the end of the operator's arm obtained by the VR head display device in real time using the following posture mapping formula to obtain the real-time target posture parameters of the end of the robotic arm:

[0019] ΔR 从,t1 =ΔR 主,t1 ,

[0020] Among them, ΔR 主,t1 is the change in the handle posture at the current moment, ΔR 从,t1 is the target posture parameter of the end of the robotic arm at the current moment;

[0021] The arm angle calculation unit is used to obtain the real-time target arm angle of the end of the robotic arm based on the IMU inertial sensor data by calculating the angle between the operator's standing posture direction (z-axis) and the normal vector of the x-axis direction plane of the two IMU inertial sensors.

[0022] The data processing unit includes a unified sending node, which is used to subscribe to data from a VR head display device and an IMU inertial sensor in a ROS2 environment.

[0023] Wherein, the mechanical arm adjustment module is further used for:

[0024] After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the target motor angle of each joint is solved through the inverse kinematics of the robotic arm, and each joint of the robotic arm is adjusted to the corresponding target motor angle.

[0025] Accordingly, as another aspect of the present invention, a robot remote control method is also provided, which at least comprises:

[0026] Use VR head-mounted display equipment to obtain the position and posture information of the end of the operator's arm in real time;

[0027] Use IMU inertial sensors to obtain the arm angle data of the operator in real time;

[0028] The position, posture information and arm angle data of the operator's arm end are mapped and calculated to obtain the corresponding real-time target position, posture and arm angle parameters of the robot arm end:

[0029] Sending the real-time target position, posture and arm angle parameters of the end of the robotic arm to the remote robot;

[0030] After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the remote robot adjusts the mechanical arm according to the parameters so that the mechanical arm and the operator's arm are synchronized.

[0031] Among them, mapping and calculating the position, posture information and arm angle data of the end of the operator's arm to obtain the corresponding real-time target position, posture and arm angle parameters of the end of the mechanical arm further includes:

[0032] The following position mapping formula is used to map the position information of the end of the operator's arm obtained by the VR head display device in real time to obtain the real-time target position parameters of the end of the robotic arm:

[0033] P 主,t1 =P 主,t0 +K·ΔP 主,t1

[0034] P 从,t1 =P 从,t0 +K·ΔP 从,t1

[0035] Among them, P 从,t0 is the position parameter of the end of the robot arm at the initial moment, P 从,t1 is the target position parameter of the end of the robot arm at the current moment, ΔP 从,t1 is the position change of the end of the robot arm at the current moment; P 主,t0 is the position parameter of the handle at the initial moment, P 凡,t1 is the target position parameter of the handle at the current moment, ΔP 主,t1 is the change in the handle position at the current moment, and K is the mapping coefficient;

[0036] The following posture mapping formula is used to map the posture information of the operator's arm end obtained by the VR head display device in real time to obtain the real-time target posture parameters of the robot arm end:

[0037] ΔR 从,t1 =ΔR 主,t1 ,

[0038] Among them, ΔR 主,t1 is the change in the handle posture at the current moment, ΔR 从,t1 is the target posture parameter of the end of the robotic arm at the current moment;

[0039] Based on the IMU inertial sensor data, the real-time target arm angle of the end of the robotic arm is obtained by calculating the angle between the operator's standing posture direction (z-axis) and the normal vectors of the x-axis planes of the two IMU inertial sensors.

[0040] Among them, further include:

[0041] Use the unified sending node to subscribe to data from VR headsets and IMU inertial sensors in the ROS2 environment.

[0042] After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the remote robot adjusts the mechanical arm according to the parameters, further comprising:

[0043] After receiving the real-time target position, posture and arm angle parameters sent by the data processing module, the robot arm solves the target motor angle of each joint through the robot arm inverse kinematics, and adjusts each joint of the robot arm to the corresponding target motor angle.

[0044] Accordingly, as another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned method are implemented.

[0045] Accordingly, as another aspect of the present invention, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computer device to perform operations corresponding to the aforementioned method.

[0046] Implementing the embodiments of the present invention has the following beneficial effects:

[0047] The present invention provides a robot remote control system, method, storage medium and computer program product. VR head display provides high-precision position information, and IMU provides posture information. The combination of the two can control the arm angle of the robot arm to be consistent with the human body, more in line with the actual arm posture, and achieve high-precision control under small space constraints.

[0048] In the embodiment of the present invention, the addition of IMU makes the posture control of the robot arm more accurate and stable. Even when the VR handle marker is blocked, IMU can still provide reliable posture information, ensuring that the robot arm shape is the same as the human arm shape while ensuring accurate control, and thus achieving the function of avoiding obstacles when remotely controlling the robot arm.

[0049] The arm angle calculation method selected in this scheme is concise, easy to implement and highly stable.

[0050] In summary, this solution achieves remote and precise control of the robotic arm through innovative equipment combination and algorithm processing. Its advantages are high precision, low space restrictions, enhanced posture controllability, and a simple and stable arm angle selection method. These advantages make this solution have broad application prospects in remote control, industrial automation, medical rehabilitation and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, without paying creative labor, other drawings obtained based on these drawings still belong to the scope of the present invention.

[0052] Figure 1 A schematic structural diagram of an embodiment of a robot remote control system provided by the present invention;

[0053] Figure 2 for Figure 1 A schematic diagram of the structure of the data processing module involved in;

[0054] Figure 3 A schematic diagram of the arm angle involved in the present invention;

[0055] Figure 4 A schematic diagram of the main process of an embodiment of a robot remote control method provided by the present invention. DETAILED DESCRIPTION

[0056] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings.

[0057] like Figure 1 FIG. 1 is a schematic diagram showing a structure of an embodiment of a robot remote control system provided by the present invention; and FIG. Figure 2 , Figure 3 As shown, in this embodiment, the system at least includes:

[0058] VR head display device 1 is used to obtain the position and posture information of the end of the operator's arm in real time; in a specific example, in order to ensure that the VR head display device 1 can obtain the position and posture of the VR handle in real time, its position accuracy is required to be controlled within 1 cm and the refresh frequency is not less than 10 Hz;

[0059] IMU inertial sensor 2 is used to obtain the arm angle data of the operator's arm in real time. In a specific example, the IMU inertial sensor 2 is required to have good anti-magnetic properties, be less affected by the magnetic field, and have a refresh rate of not less than 10Hz and be controllable; in a specific example, the attitude value collected by the IMU inertial sensor 2 can be obtained through Bluetooth and transmitted through the UDP transmission protocol;

[0060] The data processing module 3 is used to map and calculate the position, posture information, and arm angle data of the operator's arm end to obtain the corresponding real-time target position, posture, and arm angle parameters of the manipulator end; in a specific example, the data processing unit 3 includes a unified sending node for subscribing to data from the VR head display device and the IMU inertial sensor in the ROS2 environment. When in use, it is necessary to ensure that the VR space coordinate system is parallel to the manipulator coordinate system. The data processing module 3 is used to process the received VR handle and IMU posture information, and output the manipulator end posture and arm angle parameters; when used for the first time, the arm posture should be consistent with the manipulator posture.

[0061] The transmission processing module 4 is used to send the real-time target position, posture and arm angle parameters of the end of the mechanical arm calculated by the data processing module to the remote robot;

[0062] The mechanical arm adjustment module 5 is used to receive the real-time target position, posture and arm angle parameters sent from the data processing module, and adjust the robot's mechanical arm according to the parameters to synchronize the mechanical arm with the operator's arm.

[0063] like Figure 2 As shown, in a specific example, the data processing module 3 further includes:

[0064] The position mapping processing unit 30 is used to map the position information of the end of the operator's arm obtained by the VR head display device in real time using the following position mapping formula to obtain the real-time target position parameters of the end of the robot arm:

[0065] P 主,t1 =P 主,t0 +K·ΔP 主,t1

[0066] P 从,t1 =P 从,t0 +K·ΔP 从,t1

[0067] Among them, P 从,t0 is the position parameter of the end of the robot arm at the initial moment, P 从,t1 is the target position parameter of the end of the robot arm at the current moment, ΔP 从,t1 is the position change of the end of the robot arm at the current moment; P 主,t0 is the position parameter of the handle at the initial moment, P 凡,t1 is the target position parameter of the handle at the current moment, ΔP 主,t1 is the change in the handle position at the current moment, and K is the mapping coefficient, which is used to adjust the ratio between the position increment of the end of the manipulator and the position increment of the VR handle. It can be understood that the present invention sets a master-slave end mapping model, and the position of the end of the manipulator adopts an incremental mapping method, which can achieve accurate correspondence between the VR handle and the end of the manipulator in position, so that the operator can control the movement of the end of the manipulator by moving the VR handle.

[0068] The posture mapping processing unit 31 is used to use the following posture mapping formula to map the posture information of the end of the operator's arm obtained by the VR head display device in real time to obtain the real-time target posture parameters of the end of the robot arm:

[0069] ΔR 从,t1 =ΔR 主,t1 ,

[0070] Among them, ΔR 主,t1 is the change in the handle posture at the current moment, ΔR 从,t1is the target posture parameter of the end of the robotic arm at the current moment; it can be understood that in the master-slave mapping model of the present invention, the posture adopts absolute mapping, which can achieve accurate correspondence between the VR handle and the end of the robotic arm in posture, ensuring that the robotic arm can maintain the correct posture when performing tasks;

[0071] The arm angle calculation unit 32 is used to obtain the real-time target arm angle of the end of the robotic arm based on the IMU inertial sensor data by calculating the angle between the operator's standing posture direction (z-axis) and the normal vectors of the x-axis direction planes of the two IMU inertial sensors.

[0072] like Figure 3 As shown, in the specific example, the arm angle Ψ is set as the angle between the normal vectors of the plane in the x-axis direction of the two IMU inertial sensors on the arm and the z-axis of the human body coordinate system, so the arm angle Ψ can be calculated using the following formula:

[0073]

[0074] in, Represents the normal vector of the plane where the X-axis directions of the two IMUs are located; Represents the Z-axis vector of the human body coordinate system; Represents the dot product of the normal vector and the Z-axis vector; and They represent the normal vector and the modulus of the Z-axis vector respectively. This formula can provide key parameters for the posture control of the robot arm to ensure the consistency of the posture of the robot arm and the human arm.

[0075] In a specific example, the robot arm adjustment module 5 is further used to:

[0076] After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the target motor angle of each joint is solved by the inverse kinematics of the manipulator, and each joint of the manipulator is adjusted to the corresponding target motor angle. It can be understood that the inverse kinematics solution can be solved by using the inverse kinematics equation related to the Cartesian space, and the position and posture obtained in the Cartesian space coordinate system can be converted into the conversion angle of each motor.

[0077] It is understandable that the above formulas play an important role in the process of remote control of the robotic arm. They realize the precise correspondence between the VR handle and the end of the robotic arm in position and posture, as well as the accurate calculation of the arm angle through precise mathematical expressions. The application of these formulas makes the remote control of the robotic arm more intuitive, accurate and stable, providing strong technical support for various application scenarios. At the same time, the simplicity and stability of these formulas are also one of the innovative points of the present invention in equipment selection and arm angle selection.

[0078] It is understandable that the inertial and visual-based robot remote control system provided by the present invention provides high-precision position information through VR head display and posture information through IMU, and the combination of the two realizes high-precision control under small space constraints. For example, in a narrow laboratory environment, operators can wear VR equipment and IMU to perform delicate experimental operations by remotely controlling the robotic arm without directly contacting dangerous or expensive experimental equipment.

[0079] The addition of IMU makes the posture control of the robot arm more precise and stable. Even when the VR handle markers are blocked, IMU can still provide reliable posture information. For example, in complex industrial environments, the robot arm needs to perform complex posture adjustment tasks. With the assistance of IMU, the operator can ensure that the robot arm can still accurately execute instructions in the case of occlusion or interference.

[0080] The arm angle calculation method selected in this solution is concise, easy to implement and highly stable. For example, in the field of medical rehabilitation, the robot arm needs to simulate the movement of the human arm for rehabilitation training. Through the arm angle selection method of this solution, it can ensure that the movement of the robot arm is more natural and smooth, and improve the rehabilitation effect.

[0081] In summary, this solution achieves remote and precise control of the robotic arm through innovative equipment combination and algorithm processing. Its advantages are high precision, low space restrictions, enhanced posture controllability, and a simple and stable arm angle selection method. These advantages make this solution have broad application prospects in remote control, industrial automation, medical rehabilitation and other fields.

[0082] Figure 4 FIG. 1 is a schematic diagram showing the main flow of an embodiment of a robot remote control method provided by the present invention. In this embodiment, the method adopts the following method: Figures 1 to 3 The system shown in the figure is implemented, and the method at least includes the following steps:

[0083] Step S10, using a VR head display device to obtain the position and posture information of the end of the operator's arm in real time;

[0084] Step S11, using an IMU inertial sensor to obtain arm angle data of the operator's arm in real time;

[0085] It can be understood that in a specific example, a unified sending node can be used to subscribe to data from a VR headset and an IMU inertial sensor in a ROS2 environment.

[0086] Step S12, mapping and calculating the position, posture information and arm angle data of the end of the operator's arm to obtain the corresponding real-time target position, posture and arm angle parameters of the end of the mechanical arm:

[0087] In a specific example, the step S12 further includes:

[0088] Step S120, using the following position mapping formula, the position information of the end of the operator's arm obtained by the VR head display device in real time is mapped to obtain the real-time target position parameters of the end of the robot arm:

[0089] P 主,t1 =P 主,t0 +K·ΔP 主,t1

[0090] P 从,t1 =P 从,t0 +K·ΔP 从,t1

[0091] Among them, P 从,t0 is the position parameter of the end of the robot arm at the initial moment, P 从,t1 is the target position parameter of the end of the robot arm at the current moment, ΔP 从,t1 is the position change of the end of the robot arm at the current moment; P 主,t0 is the position parameter of the handle at the initial moment, P 凡,t1 is the target position parameter of the handle at the current moment, ΔP 主,t1 is the change in the handle position at the current moment, and K is the mapping coefficient;

[0092] Step S121, using the following posture mapping formula, the VR head display device obtains the posture information of the operator's arm end in real time to map and process, and obtains the real-time target posture parameters of the robot arm end:

[0093] ΔR 从,t1 =ΔR 主,t1 ,

[0094] Among them, ΔR 主,t1 is the change in the handle posture at the current moment, ΔR 从,t1 is the target posture parameter of the end of the robotic arm at the current moment;

[0095] Step S122, based on the IMU inertial sensor data, by calculating the angle between the operator's standing posture direction (z-axis) and the normal vectors of the x-axis direction planes of the two IMU inertial sensors, the real-time target arm angle of the end of the robotic arm is obtained;

[0096] More specifically, the arm angle Ψ may be calculated using the following formula:

[0097]

[0098] in, Represents the normal vector of the plane where the X-axis directions of the two IMUs are located; Represents the Z-axis vector of the human body coordinate system; Represents the dot product of the normal vector and the Z-axis vector; and They represent the normal vector and the modulus of the Z-axis vector respectively. This formula is concise and stable, and can provide key parameters for the posture control of the robot arm to ensure the consistency of the posture of the robot arm and the human arm.

[0099] Step S13, sending the real-time target position, posture and arm angle parameters of the end of the robotic arm to the remote robot;

[0100] Step S14, after receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the remote robot adjusts the mechanical arm according to the parameters to synchronize the mechanical arm with the operator's arm.

[0101] In a specific example, the step S14 further includes:

[0102] After receiving the real-time target position, posture and arm angle parameters sent by the data processing module, the robot arm solves the target motor angle of each joint through the robot arm inverse kinematics, and adjusts each joint of the robot arm to the corresponding target motor angle.

[0103] For more details, please refer to and combine the above Figures 1 to 3 The description is not repeated here.

[0104] Accordingly, as another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 4 For more details, please refer to and combine with the above Figure 4 The description is not repeated here.

[0105] Accordingly, as another aspect of the present invention, a computer program product is also provided, comprising computer instructions, wherein the computer instructions instruct a computer device to execute the above-mentioned Figure 4 For more details, please refer to and combine the above Figure 4 The description is not repeated here.

[0106] Implementing the embodiments of the present invention has the following beneficial effects:

[0107] The present invention provides a robot remote control system, method, storage medium and computer program product. VR head display provides high-precision position information, and IMU provides posture information. The combination of the two can control the arm angle of the robot arm to be consistent with the human body, more in line with the actual arm posture, and achieve high-precision control under small space constraints.

[0108] In the embodiment of the present invention, the addition of IMU makes the posture control of the robot arm more accurate and stable. Even when the VR handle marker is blocked, IMU can still provide reliable posture information, ensuring that the robot arm shape is the same as the human arm shape while ensuring accurate control, and thus achieving the function of avoiding obstacles when remotely controlling the robot arm.

[0109] The arm angle calculation method selected in this scheme is concise, easy to implement and highly stable.

[0110] In summary, this solution achieves remote and precise control of the robotic arm through innovative equipment combination and algorithm processing. Its advantages are high precision, low space restrictions, enhanced posture controllability, and a simple and stable arm angle selection method. These advantages make this solution have broad application prospects in remote control, industrial automation, medical rehabilitation and other fields.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] The above disclosure is only a preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A robot remote control system, characterized in that: At least: VR head-mounted display device, used to obtain the position and posture information of the end of the operator's arm in real time; IMU inertial sensor, used to obtain the arm angle data of the operator in real time; The data processing module is used to map and calculate the position, posture information, and arm angle data of the end of the operator's arm to obtain the corresponding real-time target position, posture, and arm angle parameters of the end of the robotic arm: The transmission processing module is used to send the real-time target position, posture and arm angle parameters of the end of the robotic arm calculated by the data processing module to the remote robot; The robot arm adjustment module is used to receive the real-time target position, posture and arm angle parameters sent from the data processing module, and adjust the robot's robot arm according to the parameters to synchronize the robot arm with the operator's arm.

2. The system according to claim 1, characterized in that The data processing module further comprises: The position mapping processing unit is used to map the position information of the end of the operator's arm obtained by the VR head display device in real time using the following position mapping formula to obtain the real-time target position parameters of the end of the robotic arm: P 主,t1 =P 主,t0 +K·ΔP 主,t1 P 从,t1 =P 从,t0 +K·ΔP 从,t1 Among them, P 从,t0 is the position parameter of the end of the robot arm at the initial moment, P 从,t1 is the target position parameter of the end of the robot arm at the current moment, ΔP 从,t1 is the position change of the end of the robot arm at the current moment; P 主,t0 is the position parameter of the handle at the initial moment, P 凡,t1 is the target position parameter of the handle at the current moment, ΔP 主,t1 is the change in the handle position at the current moment, and K is the mapping coefficient; The posture mapping processing unit is used to map the posture information of the end of the operator's arm obtained by the VR head display device in real time using the following posture mapping formula to obtain the real-time target posture parameters of the end of the robotic arm: ΔR 从,t1 =ΔR 主,t1 , Among them, ΔR 主,t1 is the change in the handle posture at the current moment, ΔR 从,t1 is the target posture parameter of the end of the robotic arm at the current moment; The arm angle calculation unit is used to obtain the real-time target arm angle of the end of the robotic arm based on the IMU inertial sensor data by calculating the angle between the operator's standing posture direction (z-axis) and the normal vector of the x-axis direction plane of the two IMU inertial sensors.

3. The system according to claim 2, characterized in that The data processing unit includes a unified sending node for subscribing to data from a VR head display device and an IMU inertial sensor in a ROS2 environment.

4. The system according to claim 3, characterized in that The mechanical arm adjustment module is further used for: After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the target motor angle of each joint is solved through the inverse kinematics of the robotic arm, and each joint of the robotic arm is adjusted to the corresponding target motor angle.

5. A robot remote control method, characterized in that: At least the following steps are included: Use VR head-mounted display equipment to obtain the position and posture information of the end of the operator's arm in real time; Use IMU inertial sensors to obtain the arm angle data of the operator in real time; The position, posture information and arm angle data of the operator's arm end are mapped and calculated to obtain the corresponding real-time target position, posture and arm angle parameters of the robot arm end: Sending the real-time target position, posture and arm angle parameters of the end of the robotic arm to the remote robot; After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the remote robot adjusts the mechanical arm according to the parameters so that the mechanical arm and the operator's arm are synchronized.

6. The method according to claim 5, characterized in that Mapping and calculating the position, posture information and arm angle data of the end of the operator's arm to obtain the corresponding real-time target position, posture and arm angle parameters of the end of the mechanical arm, further comprising: The following position mapping formula is used to map the position information of the end of the operator's arm obtained by the VR head display device in real time to obtain the real-time target position parameters of the end of the robotic arm: P 主,t1 =P 主,t0 +K·ΔP 主,t1 P 从,t1 =P 从,t0 +K·ΔP 从,t1 Among them, P 从,t0 is the position parameter of the end of the robot arm at the initial moment, P 从,t1 is the target position parameter of the end of the robot arm at the current moment, ΔP 从,t1 is the position change of the end of the robot arm at the current moment; P 主,t0 is the position parameter of the handle at the initial moment, P 凡,t1 is the target position parameter of the handle at the current moment, ΔP 主,t1 is the change in the handle position at the current moment, and K is the mapping coefficient; The following posture mapping formula is used to map the posture information of the operator's arm end obtained by the VR head display device in real time to obtain the real-time target posture parameters of the robot arm end: ΔR 从,t1 =ΔR 主,t1 Among them, ΔR 主,t1 is the change in the handle posture at the current moment, ΔR 从,t1 is the target posture parameter of the end of the robotic arm at the current moment; Based on the IMU inertial sensor data, the real-time target arm angle of the end of the robotic arm is obtained by calculating the angle between the operator's standing posture direction (z-axis) and the normal vectors of the x-axis planes of the two IMU inertial sensors.

7. The method according to claim 6, characterized in that Further including: Use the unified sending node to subscribe to data from VR headsets and IMU inertial sensors in the ROS2 environment.

8. The method according to claim 7, characterized in that After receiving the real-time target position, posture and arm angle parameters sent from the data processing module, the remote robot adjusts the mechanical arm according to the parameters, further comprising: After receiving the real-time target position, posture and arm angle parameters sent by the data processing module, the robot arm solves the target motor angle of each joint through the robot arm inverse kinematics, and adjusts each joint of the robot arm to the corresponding target motor angle.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 5 to 8 are implemented.

10. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computer device to perform operations corresponding to the method according to any one of claims 5 to 8.

Images