Teleoperation System of Dual-Arm Intelligent Robot Based on MR Remote Control

Abstract

The present invention discloses a teleoperation system for a dual-arm intelligent robot based on MR remote control. The system composition includes: a robot, which installs a dual-arm manipulator and a binocular camera on a walking vehicle body; an MR remote control terminal, which is an MR helmet worn by an operator on the head; an information processing unit: a central computing and processing device that establishes communication connections with the walking vehicle body, the dual-arm manipulator, the binocular camera, and the MR helmet; a virtual reality system composed of the binocular camera, the MR helmet, and the central computing and processing device. The present invention can perform real-time visual long-distance operations on complex and inaccessible environments through the virtual reality system. The dual arms can flexibly turn, with controllable speed and good degrees of freedom, effectively solving the problem that is likely to cause harm to human life during the existing complex operation process.

Description

Technical Field

[0001] The present invention relates to the technical field of robot applications, and more specifically to a teleoperated dual-arm intelligent robot based on an MR remote control terminal system. Background Art

[0002] To ensure personal safety, in some occasions with complex environments and uncertain dangers, it is often not possible for humans to directly enter the dangerous area for relevant operations. In the prior art, a camera is fixedly installed on the robot, and the robot together with the camera is sent into the set area to work. This method greatly limits the field of view of the robot. When the robot vibrates, it is difficult to aim at the target object. Especially in the case of teleoperating the robot, the robot has poor flexibility and it is difficult to achieve fast and efficient operations, which can no longer meet the usage requirements of various related fields in the rapid development of society. Summary of the Invention

[0003] The present invention aims to avoid the deficiencies of the above-mentioned prior art, and provides a teleoperated dual-arm intelligent robot based on an MR remote control terminal system with good degrees of freedom, flexible operation, and human-machine collaborative visual operation. By using virtual reality technology, the robot can be real-time controlled to perform operations in complex environments, solving the problem of endangering human life in existing complex operations.

[0004] The present invention adopts the following technical solutions to solve the technical problems:

[0005] The teleoperation system of the dual-arm intelligent robot based on MR remote control of the present invention includes:

[0006] Robot: A dual-arm manipulator and a binocular camera are installed on a walking vehicle body;

[0007] MR remote control terminal: An MR helmet worn by the operator on the head;

[0008] Information processing unit: A central computing and processing device that establishes communication connections with the walking vehicle body, the dual-arm manipulator, the binocular camera, and the MR helmet. A virtual reality system is composed of the binocular camera, the MR helmet, and the central computing and processing device;

[0009] The central computing and processing device reconstructs a three-dimensional model of the real space to construct an immersive control environment for the operator; after the operator sends a control command to the robot through the server and establishes a communication connection, the robot uses the binocular camera to collect real-time external true display scene data and transmits it back to the server. A virtual scene is created on the Unity platform and deployed to the mixed reality headset through Visual Studio. The operator can visually manipulate the robot in real time to perform relevant tasks.

[0010] The characteristics of the remote operation system of the dual-arm intelligent robot based on MR remote control of the present invention also lie in that: the binocular camera scans the three-dimensional environment of the site, forms a digital twin environment through data fusion calculation with its own state sensor data, displays the complete three-dimensional environment around the robot and the three-dimensional posture of the robot in the digital twin environment, displays a stereoscopic scene through the immersive display function of the MR helmet, and the operator performs interactive operations such as zooming, panning, and rotating through gestures to observe the task site from different perspectives.

[0011] The characteristics of the remote operation system of the dual-arm intelligent robot based on MR remote control of the present invention also lie in that: the MR remote control terminal has a user intention analysis module, a mixed reality capture module, an image preprocessing module, a communication module, a visual algorithm processing module, an output processing module, a mixed reality rendering module, and a mixed reality interaction module; the system exports the reconstructed three-dimensional model in the.obj file format and places it in the Unity platform scene. After the real scene data is processed with background transparency and superimposed with the virtual scene, the system displays the monitoring video of the robot, the three-dimensional virtual scene, the robot status information, the commander communication information, and the control operation interface in the immersive interface; the operator uses virtual gestures to interact with the control interface through the gesture recognition function, inputs instructions, and manipulates the robot to perform related tasks.

[0012] The characteristics of the remote operation system of the dual-arm intelligent robot based on MR remote control of the present invention also lie in that:

[0013] The communication between the communication module in the MR remote control terminal and the server adopts the SocketAsyncEventArgs method and is carried out according to the following steps:

[0014] Step 1. Create a SocketAsyncEventArgs object and a Socket object;

[0015] Step 2. Set the callback method, buffer, and properties of UserToken of the SocketAsyncEventArgs object;

[0016] Step 3. Use the ConnectAsync method of the Socket object with the SocketAsyncEventArgs object as a parameter to create an asynchronous connection to connect to the server;

[0017] Step 4. Use the ReceiveAsync method to receive messages asynchronously;

[0018] Step 5. Use the SendAsync method to send messages asynchronously.

[0019] The characteristics of the remote operation system of the dual-arm intelligent robot based on MR remote control of the present invention also lie in that:

[0020] Before the virtual-real interaction, the mixed reality interaction module in the MR remote control terminal first establishes an accurate mapping relationship between the virtual scene and the real environment in the following manner, converts the virtual space and the real space into a unified coordinate system, realizes scene registration, and ensures the accurate position matching between the virtual scene and the real environment:

[0021] Let P1(x1, z1) and P2(x2, z2) be two points in the virtual space, and the corresponding points in the real space are represented as P′1(x′1, z′1) and P′2(x′2, z′2) respectively; the vectors and are calculated respectively by equations (a) and (b) as:

[0022]

[0023]

[0024] where: x = x2 - x1; z = z2 - z1; x′ = x′2 - x′1; z′ = z′2 - z′1;

[0025] Let any point in the virtual space be represented as V(V.x, V.z), and the corresponding point in the real space is R(R.x, R.z);

[0026] Then, the coordinate relationship between point V(V.x, V.z) and point R(R.x, R.z) is characterized by equations (c) and (d) as:

[0027]

[0028]

[0029] where: θ is the included angle between the vectors and ;

[0030] sinθ, cosθ and ratio are calculated respectively by equations (e), (f) and (g) as:

[0031]

[0032]

[0033]

[0034] The mutual conversion between the virtual coordinate system and the real space coordinate system is realized by formulas (c) and (d). By traversing the information of each node in the 2D structure diagram, the position in the real space coordinate system is found, thereby constructing a virtual model; through the mapping relationship, the scene content is arranged in the MR helmet in real time.

[0035] The characteristics of the two-armed intelligent robot teleoperation system based on MR remote control of the present invention also lie in:

[0036] The mixed reality interaction module in the MR remote control terminal includes three types, namely static overlay, real-time tracking interaction, and real-time modeling interaction;

[0037] The static overlay is to overlay virtual content in a static real scene. The virtual content is the text description information of the target real object or a preset virtual object;

[0038] The real-time tracking interaction is to identify and track real objects by tracking preset features or models, and perform real-time overlay thereon, or perform position-based collision interaction;

[0039] The real-time modeling interaction is to perform real-time modeling on moving real objects to obtain their pose and contour information, and create a point cloud copy according to this information to make the point cloud copy interact precisely with virtual objects.

[0040] The characteristics of the two-armed intelligent robot teleoperation system based on MR remote control of the present invention also lie in: the MR helmet has mixed reality display glasses; the mixed reality display glasses include:

[0041] A real scene acquisition module, which acquires real-time real scene data from the outside world through acquisition;

[0042] A virtual scene generation module, which is used to generate a virtual scene;

[0043] An image fusion module, which is used to perform image fusion on the real scene and the virtual scene by overlay;

[0044] A space positioning module, which acquires real-time position information of the acquisition space through acquisition;

[0045] An image display module, which displays the image combining reality and virtuality.

[0046] The MR helmet displays the surveillance video of the binocular camera, the three-dimensional virtual scene, the robot state information, the operator communication information, and the control operation interface in an immersive interface by performing background transparency processing on the real-time real scene data and overlaying the virtual scene.

[0047] The characteristics of the two-armed intelligent robot teleoperation system based on MR remote control of the present invention also lie in:

[0048] The walking vehicle body adopts a crawler-type walking mechanism, is equipped with a communication module and a central controller, uses the communication module to establish a connection between the crawler-type walking mechanism and the central controller, and the central controller receives control instructions from the MR helmet and the central computing and processing device, and performs corresponding task operations on the walking vehicle body.

[0049] The characteristics of the dual-arm intelligent robot teleoperation system based on MR remote control of the present invention also lie in:

[0050] The dual-arm manipulator includes a hand and a robotic arm; the proximal end of the robotic arm is installed on the walking vehicle body, and the hand is installed at the distal end of the robotic arm;

[0051] The robotic arm has a total of six degrees of freedom: shoulder roll, shoulder pitch, elbow roll, elbow pitch, wrist roll and wrist pitch;

[0052] The hand follows the robotic arm and has six degrees of freedom. The hand is independently provided with a palm, and fingers are provided on the palm, including a thumb and four fingers; both the thumb and the four fingers are provided with finger joints, and drive motors are provided inside the finger joints and at the connection positions between the palm and the fingers, so that the hand can perform a grasping action.

[0053] Compared with the prior art, the beneficial effects of the present invention are reflected in:

[0054] 1. The present invention is provided with an MR helmet, a binocular camera and a central computing and processing device in cooperation, and can realize visual teleoperation of the robot, greatly reducing the difficulty of robot operation affected by distance and the surrounding environment.

[0055] 2. The central computing and processing device of the present invention is responsible for receiving, processing and synchronously storing relevant information from the MR helmet and the robot; the robot can communicate with the MR helmet and the central computing and processing device through a wireless network, and transmit the information collected by various devices to the MR helmet and the central computing and processing device in real time, so as to establish a good feedback channel between the operator and the robot and ensure accurate control.

[0056] 3. The robot in the present invention has good degrees of freedom, can turn flexibly, has controllable speed, and is extremely beneficial for better performing outdoor dangerous operations through visual teleoperation.

[0057] 4. The present invention adopts a crawler-type mobile trolley, which can cope with complex road surface environments, enabling the robot to quickly reach the destination.

[0058] 5. The dual-arm manipulator in the present invention can not only make the robot have good degrees of freedom and a wide range of activities, but also enable the operator and the robot to cooperate through a virtual reality system to achieve the purpose of accurately grasping objects. Brief Description of the Drawings

[0059] Figure 1 This is a schematic diagram of the overall architecture of the system of the present invention;

[0060] Figure 2 This is a functional architecture diagram of the system of the present invention;

[0061] Figure 3 This is a schematic diagram of the structure of the dual-arm intelligent robot in the system of the present invention.

[0062] Reference numerals in the figure: 1 walking vehicle body, 2 dual-arm manipulator, 3 binocular camera, 4 is an MR helmet, 5 central computing and processing device. Detailed Implementation Manner

[0063] Refer to Figure 1 In this embodiment, the teleoperation system of the dual-arm intelligent robot based on MR remote control includes a robot, an MR remote control terminal, and an information processing unit. Among them: The robot is equipped with a dual-arm manipulator 2 and a binocular camera 3 on a walking vehicle body 1; The MR remote control terminal is an MR helmet 4 worn by the operator on the head. The operator controls the forward and backward movement of the robot, the posture of the robotic arm, and the grasping posture of the robotic fingers through the MR helmet; The operator can also use the MR helmet to communicate with the commander in real time through the network and send relevant videos and other pictures to the commander's end; The information processing unit is a central computing and processing device 5 that establishes communication connections with the walking vehicle body 1, the dual-arm manipulator 2, the binocular camera 3, and the MR helmet 4. A virtual reality system is composed of the binocular camera 3, the MR helmet 4, and the central computing and processing device 5; The central computing and processing device 5 constructs an immersive control environment for the operator by performing three-dimensional model reconstruction on the real space; After the operator sends a control command to the robot through the server and establishes a communication connection, the robot uses the binocular camera 3 to collect real-time external real display scene data and transmits it back to the server, creates a virtual scene in the Unity platform, and deploys it to the mixed reality headset through Visual Studio. The operator visually manipulates the robot in real time to perform relevant tasks.

[0064] In this embodiment, the binocular camera 3 scans the on-site three-dimensional environment, forms a digital twin environment through data fusion calculation with its own state sensor data, displays the complete three-dimensional environment around the robot and the three-dimensional posture of the robot in the digital twin environment, and displays a stereoscopic scene through the immersive display function of the MR helmet. The operator performs interactive operations such as zooming, panning, and rotating through gestures to observe the task site from different perspectives.

[0065] Refer to Figure 2, in this embodiment, the MR remote control terminal has a user intention analysis module, a mixed reality capture module, an image preprocessing module, a communication module, a vision algorithm processing module, an output processing module, a mixed reality rendering module, and a mixed reality interaction module; the system exports the reconstructed 3D model in the.obj file format and places it in the Unity platform scene. After the real scene data is processed with background transparency and superimposed on the virtual scene, the system displays the robot's surveillance video, 3D virtual scene, robot status information, commander communication information, and control operation interface in the immersive interface; the operator uses virtual gestures to interact with the control interface through the gesture recognition function, inputs instructions, and manipulates the robot to perform related tasks.

[0066] The software interaction system of the MR remote control terminal mainly includes three functional modules: panoramic virtual scene construction, instruction capture and recognition, and target detection and tracking; the software system is divided into three parts: a data acquisition layer, a data processing layer, and an application layer. Among them, the data acquisition layer provides basic data for subsequent panoramic generation and display by collecting image, sound, and coordinate data; the data processing layer performs virtual image generation, gesture recognition, speech recognition, and target tracking; the application layer is responsible for the virtual-real interaction of the scene.

[0067] Figure 2 The system shown can record the information collected by the robot for analysis and summary after the task, and can also record the operator's operation information for retrospective analysis after the task.

[0068] In this embodiment, the communication between the communication module in the MR remote control terminal and the server is carried out using the SocketAsyncEventArgs method according to the following steps:

[0069] Step 1. Create a SocketAsyncEventArgs object and a Socket object;

[0070] Step 2. Set the callback method, buffer, and UserToken properties of the SocketAsyncEventArgs object;

[0071] Step 3. Use the ConnectAsync method of the Socket object with the SocketAsyncEventArgs object as a parameter to create an asynchronous connection to connect to the server;

[0072] Step 4. Use the ReceiveAsync method to receive messages asynchronously;

[0073] Step 5. Use the SendAsync method to send messages asynchronously.

[0074] SocketAsyncEventArgs is an asynchronous socket parameter that exists in the System.Net.Sockets namespace. It is a set of asynchronous methods designed for high concurrency in the.NET library. It can achieve the reuse of socket objects, saving object reallocation and synchronization-related issues under high concurrency and improving system performance.

[0075] In this embodiment, before the virtual-real interaction of the MR remote control terminal, the hybrid reality interaction module first establishes an accurate mapping relationship between the virtual scene and the real environment in the following manner. It transforms the virtual space and the real space into a unified coordinate system to achieve scene registration and ensure the accurate position matching between the virtual scene and the real environment:

[0076] Let P1(x1, z1) and P2(x2, z2) be two points in the virtual space, and the corresponding points in the real space are represented as P′1(x′1, z′1) and P′2(x′2, z′2) respectively; the vectors and are calculated respectively from equations (a) and (b) as:

[0077]

[0078]

[0079] where: x = x2 - x1; z = z2 - z1; x′ = x′2 - x′1; z′ = z′2 - z′1;

[0080] Let any point in the virtual space be represented as V(V.x, V.z), and the corresponding point in the real space is R(R.x, R.z);

[0081] Then, the coordinate relationship between point V(V.x, V.z) and point R(R.x, R.z) is characterized by equations (c) and (d):

[0082]

[0083]

[0084] where: θ is the included angle between the vectors and ;

[0085] sinθ, cosθ, and ratio are calculated respectively from equations (e), (f), and (g) as:

[0086]

[0087]

[0088]

[0089] The mutual conversion between the virtual coordinate system and the real space coordinate system is realized by formula (c) and formula (d). By traversing the information of each node in the 2D structure diagram, the position in the real space coordinate system is found, thereby constructing a virtual model; through the mapping relationship, the scene content is arranged in the MR helmet in real time.

[0090] In this embodiment, the mixed reality interaction module in the MR remote control terminal includes three types, namely static superposition, real-time tracking interaction, and real-time modeling interaction; among them, static superposition is to superimpose virtual content on a static real scene, and the virtual content is the text description information of the target real object, or a preset virtual object; real-time tracking interaction is to identify and track real objects by tracking preset features or models, and perform real-time superposition on them, or perform position-based collision interaction; real-time modeling interaction is to perform real-time modeling on moving real objects to obtain their pose and contour information, and create a point cloud copy according to this information, so that the point cloud copy interacts finely with virtual objects.

[0091] In this embodiment, the MR helmet 4 has mixed reality display glasses; the mixed reality display glasses include:

[0092] A real scene acquisition module, which acquires real-time real scene data from the outside world through acquisition;

[0093] A virtual scene generation module, which is used to generate a virtual scene;

[0094] An image fusion module, which is used to perform image fusion on the real scene and the virtual scene by superimposition;

[0095] A space positioning module, which acquires real-time position information of the acquisition space through acquisition;

[0096] An image display module, which displays the image combining reality and virtuality.

[0097] The MR helmet 4 displays the surveillance video of the binocular camera 3, the three-dimensional virtual scene, the robot state information, the operator communication information, and the control operation interface in the immersive interface by performing background transparency processing on the real-time real scene data and superimposing the virtual scene.

[0098] See Figure 3, in this embodiment, the walking vehicle body 1 adopts a crawler-type walking mechanism, which can cope with complex road environments, enabling the intelligent robot to quickly and accurately reach the operation site; a communication module and a central controller are configured, and the communication module is used to establish a connection between the crawler-type walking mechanism and the central controller. The central controller receives control instructions from the MR helmet 4 and the central computing and processing device 5, and performs corresponding task operations on the walking vehicle body 1.

[0099] In this embodiment, a two-arm manipulator is provided to endow the robot with more degrees of freedom and a wider range of activities. The two-arm manipulator 2 includes a hand part and a robotic arm to simulate a human hand. The proximal end of the robotic arm is mounted on the walking vehicle body 1, and the hand part is mounted at the distal end of the robotic arm; the robotic arm has a total of six degrees of freedom including shoulder roll, shoulder pitch, elbow roll, elbow pitch, wrist roll and wrist pitch; the hand part follows the robotic arm and has six degrees of freedom. The hand part is independently provided with a palm, and fingers are provided on the palm, including a thumb and four fingers; both the thumb and the four fingers are provided with finger joints, and drive motors are provided inside the finger joints and at the connection positions between the palm and the fingers, enabling the hand part to perform a grasping action and achieving the purpose of accurately grasping an object.

[0100] The present invention realizes real-time visual control of an intelligent robot, and can better present the sense of distance, three-dimensional sense and immersion in a three-dimensional space. The perfect communication module enables a good connection between the operator and the robot.

[0101] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A two-armed intelligent robot teleoperation system based on MR remote control, characterized in that The system includes: A robot: An articulated manipulator (2) and a binocular camera (3) are installed on a mobile vehicle body (1). An MR remote control terminal: An MR helmet (4) worn by an operator on the head. An information processing unit: A central computing and processing device (5) that establishes communication connections with the mobile vehicle body (1), the articulated manipulator (2), the binocular camera (3), and the MR helmet (4). A virtual reality system is composed of the binocular camera (3), the MR helmet (4), and the central computing and processing device (5). The central computing and processing device (5) constructs an immersive control environment for the operator by performing three-dimensional model reconstruction of the real space. After establishing a communication connection by the operator sending a control command to the robot through a server, the robot uses the binocular camera (3) to collect real-time external real display scene data and transmit it back to the server. A virtual scene is created in the Unity platform and deployed to the mixed reality headset through Visual Studio. The operator can visually manipulate the robot in real time to perform related tasks. The MR remote control terminal has a user intention analysis module, a mixed reality capture module, an image preprocessing module, a communication module, a vision algorithm processing module, an output processing module, a mixed reality rendering module, and a mixed reality interaction module. The system exports the reconstructed three-dimensional model in the.obj file format and places it in the Unity platform scene. After the real scene data is processed with background transparency and superimposed on the virtual scene, the system displays the robot's surveillance video, three-dimensional virtual scene, robot status information, commander communication information, and control operation interface in the immersive interface. The operator interacts with the control interface using virtual gestures through the gesture recognition function, inputs instructions, and manipulates the robot to perform related tasks. Before virtual-real interaction, the mixed reality interaction module in the MR remote control terminal first establishes an accurate mapping relationship between the virtual scene and the real environment in the following manner. The virtual space and the real space are converted to a unified coordinate system to achieve scene registration and ensure accurate position matching between the virtual scene and the real environment: Let P1(x1, z1) and P2(x2, z2) be two points in the virtual space, and the corresponding points in the real space are represented as P′1(x′1, z′1) and P′2(x′2, z′2) respectively; the vectors and are calculated by equations (a) and (b) respectively as follows: where: x = x2 - x1; z = z2 - z1; x' = x' ′2 - x'1; z' = z'2 - z'1; Let V(V.x, V.z) represent any point in the virtual space, and the corresponding point in the real space be R(R.x, R.z). Then: The coordinate relationship formulas of points V(V.x, V.z) and R(R.x, R.z) are characterized by formulas (c) and (d). where: θ is the angle between the vector and ; sinθ, cosθ, and ratio are calculated respectively by formulas (e), (f), and (g). The mutual conversion between the virtual coordinate system and the real space coordinate system is achieved by formulas (c) and (d). The position in the real space coordinate system is found by traversing the information of each node in the 2D structure diagram, thereby constructing a virtual model. Through the mapping relationship, the scene content is arranged in the MR helmet in real time.

2. The telesystem of a dual-arm intelligent robot based on MR remote control according to claim 1, characterized in that: The binocular camera (3) scans the three-dimensional environment of the site, calculates and forms a digital twin environment through data fusion with its own state sensor data, displays the complete three-dimensional environment around the robot and the three-dimensional pose of the robot in the digital twin environment, and displays a stereoscopic scene through the immersive display function of the MR helmet. The operator performs interactive operations of zooming, panning, and rotating through gestures to observe the task site from different perspectives.

3. The remote operation system for a dual-arm intelligent robot based on MR remote control according to claim 1, characterized in that: The communication between the communication module in the MR remote control terminal and the server adopts the SocketAsyncEventArgs method and is carried out according to the following steps: Step 1. Create a SocketAsyncEventArgs object and a Socket object; Step 2. Set the callback method, buffer, and properties of UserToken of the SocketAsyncEventArgs object; Step 3. Use the ConnectAsync method of the Socket object with the SocketAsyncEventArgs object as a parameter to create an asynchronous connection to connect to the server; Step 4. Asynchronously receive messages using the ReceiveAsync method; Step 5. Asynchronously send messages using the SendAsync method.

4. The remote operation system for a dual-arm intelligent robot based on MR remote control according to claim 1, characterized in that: The mixed reality interaction module in the MR remote control terminal includes three types, namely static overlay, real-time tracking interaction, and real-time modeling interaction; The static overlay is to overlay virtual content in a static real scene. The virtual content is the text description information of the target real object or a preset virtual object; The real-time tracking interaction is to identify and track real objects by tracking preset features or models, and perform real-time overlay on them, or perform position-based collision interaction; The real-time modeling interaction is to perform real-time modeling on moving real objects to obtain their pose and contour information, create a point cloud copy according to this information, and make the point cloud copy interact finely with virtual objects.

5. The telesystem for a dual-arm intelligent robot based on MR remote control according to claim 1, characterized in that: The MR helmet (4) has a mixed reality display glasses; the mixed reality display glasses include: A real-scene acquisition module, which acquires real-time real-scene data of the outside world through acquisition; A virtual-scene generation module, which is used to generate a virtual scene; An image fusion module, which is used to perform image fusion on the real scene and the virtual scene by overlaying; A space positioning module, which acquires real-time position information of the acquisition space through acquisition; An image display module, which displays the image combining reality and virtuality; The MR helmet (4) displays the surveillance video of the binocular camera (3), the three-dimensional virtual scene, the robot state information, the operator communication information, and the control operation interface in the immersive interface by performing background transparency processing on the real-time real-scene data and overlaying the virtual scene.

6. The telesystem of a dual-arm intelligent robot based on MR remote control according to claim 1, characterized in that: The walking vehicle body (1) adopts a crawler-type walking mechanism, is equipped with a communication module and a central controller, uses the communication module to establish a connection between the crawler-type walking mechanism and the central controller, and the central controller receives control instructions from the MR helmet (4) and the central computing and processing device (5), and performs corresponding task operations on the walking vehicle body (1).

7. The telesystem of a two-armed intelligent robot based on MR remote control according to claim 1, characterized in that: The two-arm manipulator (2) includes a hand part and a robotic arm; the proximal end of the robotic arm is mounted on the walking vehicle body (1), and the hand part is mounted at the distal end of the robotic arm; The robotic arm has a total of six degrees of freedom including shoulder roll, shoulder pitch, elbow roll, elbow pitch, wrist roll and wrist pitch; The hand part follows the robotic arm and has six degrees of freedom. The hand part is independently provided with a palm, and fingers are provided on the palm, including a thumb and four fingers; finger joints are provided on both the thumb and the four fingers, and drive motors are provided inside the finger joints and at the connection positions between the palm and the fingers, so that the hand part can perform grasping actions.

Images