Control method, device and equipment of teleoperation robot and medium

Abstract

The embodiment of the invention provides a control method, device and equipment for a teleoperation robot and a medium, and the method comprises the steps: responding to an alignment instruction between a master robot and a slave robot, and carrying out the alignment of the master robot and the slave robot based on the real-time pose information of the master robot and the slave robot; respectively displaying a virtual master robot image of the master robot and a virtual slave robot image of the slave robot in real time on a preset display interface; generating coarse alignment prompt information on the display interface to prompt to adjust the posture of the master robot until the posture deviation between the virtual master robot image and the virtual slave robot image meets a preset condition; fine alignment prompt information is generated on the display interface to prompt small-range adjustment of the posture of the master robot, and when the posture deviation between the virtual master robot image and the virtual slave robot image is reduced, the slave robot is controlled to selectively follow the posture movement of the master robot, so that the alignment precision between the master robot and the slave robot can be improved; and finally, the accuracy of teleoperation control is improved.

Description

Technical Field

[0001] This disclosure relates to the field of robot control technology, and in particular to a control method, device, equipment and medium for a teleoperated robot. Background Technology

[0002] With the development of science and technology, teleoperation control is widely used in surgical robots, industrial robots and humanoid robots. During teleoperation control, the operator can control the position and posture of the slave robot through the master robot and observe the movement of the slave robot through the image feedback on the display. The posture of the master robot needs to be consistent with that of the slave robot. Therefore, teleoperation control requires the alignment of the postures of the master robot and the slave robot to ensure the accuracy of teleoperation control.

[0003] In related technologies, since the method of aligning the slave robot with the master robot by actively moving itself carries significant risks, the alignment of the postures of the master and slave robots is often achieved by utilizing the master robot's active driving capability to move itself and align with the slave robot's posture. However, this method requires a high level of driving capability from the master robot. In scenarios where the master robot does not have active driving capability or has limited driving capability, it can easily increase the difficulty of alignment, leading to a reduction in the alignment accuracy between the master and slave robots, and ultimately reducing the accuracy of teleoperation control. Summary of the Invention

[0004] The main objective of this disclosure is to provide a control method, apparatus, device, and medium for a teleoperated robot, which can improve the alignment accuracy between the master robot and the slave robot, and ultimately improve the accuracy of teleoperation control.

[0005] To achieve the above objectives, a first aspect of this disclosure provides a control method for a teleoperated robot, comprising: In response to the alignment command between the master robot and the slave robot, based on the real-time pose information of the master robot and the slave robot, the virtual master robot image of the master robot and the virtual slave robot image of the slave robot are displayed in real time on a preset display interface. A coarse alignment prompt message is generated on the display interface to prompt the adjustment of the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets the preset conditions. The display interface generates a fine alignment prompt message to prompt small adjustments to the posture of the master robot. When the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot.

[0006] In some embodiments, controlling the slave robot to selectively follow the posture movement of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases includes: The attitude increment vector of the master robot during the fine alignment attitude adjustment process is acquired in real time, as well as the attitude deviation vector between the virtual master robot image and the virtual slave robot image. When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle, a target attitude increment is generated based on the corresponding mapping vector. The slave robot's posture is adjusted based on the target posture increment to follow the master robot's posture movement.

[0007] In some embodiments, generating a target attitude increment based on the corresponding mapping vector when the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle includes: Determine the left-multiplied Jacobian matrix and the right-multiplied Jacobian matrix corresponding to the attitude deviation vector; The attitude increment vector is mapped by the right-multiplied Jacobian matrix to obtain the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector. When the angle between the mapping vector and the attitude deviation vector is less than or equal to a preset angle, the mapping vector is inversely mapped by multiplying it by the Jacobian matrix to obtain the target attitude increment.

[0008] In some embodiments, after mapping the attitude increment vector to the attitude increment vector using the right-multiplied Jacobian matrix to obtain the mapped vector of the attitude deviation vector tangent space, the control method for the teleoperated robot further includes: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, the vector component of the mapping vector in the direction of the attitude deviation vector is obtained. Remove the vector components from the mapping vector to obtain the processed mapping vector; The target attitude increment is obtained by performing inverse mapping on the processed mapping vector by left multiplying it by the Jacobian matrix.

[0009] In some embodiments, after mapping the attitude increment vector to the attitude increment vector using the right-multiplied Jacobian matrix to obtain the mapped vector of the attitude deviation vector tangent space, the control method for the teleoperated robot further includes: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, the attitude of the slave robot is controlled to remain unchanged.

[0010] In some embodiments, generating coarse alignment prompts on the display interface to prompt adjustment of the master robot's posture until the posture deviation between the virtual master robot image and the virtual slave robot image meets preset conditions includes: A coarse alignment prompt is generated on the display interface to prompt the adjustment of the position and posture of the master robot so that the virtual master robot image and the virtual slave robot image coincide, until the posture deviation between the virtual master robot image and the virtual slave robot image meets the preset conditions.

[0011] In some embodiments, generating fine-alignment prompts on the display interface to prompt minor adjustments to the master robot's posture, and controlling the slave robot to selectively follow the master robot's posture when the posture deviation between the virtual master robot image and the virtual slave robot image decreases, includes: In response to the fine alignment start command, a fine alignment prompt message is generated on the display interface to prompt a small adjustment of the posture of the master robot. When the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot. Alternatively, during the movement of the master robot, when the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot.

[0012] To achieve the above objectives, a second aspect of this disclosure provides a control device for a teleoperated robot, comprising: The virtual display module is used to respond to the alignment command between the master robot and the slave robot, and to display the virtual master robot image of the master robot and the virtual slave robot image of the slave robot in real time on a preset display interface based on the real-time pose information of the master robot and the slave robot. The coarse alignment module is used to generate coarse alignment prompts on the display interface to prompt the adjustment of the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets a preset condition. The fine alignment module is used to generate fine alignment prompts on the display interface to prompt small-range adjustments to the posture of the master robot, and to control the slave robot to selectively follow the posture of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

[0013] To achieve the above objectives, a third aspect of this disclosure provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the control method for the teleoperated robot described in the first aspect of the embodiment.

[0014] To achieve the above objectives, a fourth aspect of this disclosure provides a storage medium, which is a computer-readable storage medium storing a computer program that, when executed by a processor, implements the control method for the teleoperated robot described in the first aspect embodiment.

[0015] This embodiment of the invention, through the execution of a control method for a teleoperated robot, can respond to alignment commands between a master robot and a slave robot, and based on the real-time pose information of the master robot and the slave robot, display a virtual master robot image and a virtual slave robot image on a preset display interface in real time; generate coarse alignment prompts on the display interface to prompt adjustment of the master robot's posture until the posture deviation between the virtual master robot image and the virtual slave robot image meets preset conditions; generate fine alignment prompts on the display interface to prompt minor adjustments to the master robot's posture, and control the slave robot to selectively follow the posture movement of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

[0016] Therefore, in this embodiment, the virtual master robot image of the master robot and the virtual slave robot image of the slave robot are displayed in real time on a preset display interface. When the operator moves the master robot, the virtual master robot image on the display interface also moves in real time. Then, a coarse alignment prompt message is generated on the display interface so that the operator can move the posture of the master robot under the guidance of the coarse alignment prompt message and intuitively observe the posture deviation between the virtual master robot image and the virtual slave robot image. The coarse alignment operation is completed when the posture deviation meets the preset conditions. Then, a fine alignment prompt message can be generated on the display interface to prompt the operator to adjust the posture of the master robot in a small range, so as to avoid the risk of the slave robot moving a lot before the master-slave alignment is completed. The slave robot is controlled to follow the movement of the master robot when the posture deviation between the master robot and the slave robot is reduced, so that the posture deviation between the master robot and the slave robot can be gradually reduced, and the master-slave alignment operation is finally completed. Therefore, in scenarios where the master robot lacks active driving capability or has limited driving capability, this embodiment of the present disclosure reduces the difficulty of aligning the master and slave robots by first coarse alignment and then fine alignment, and selectively following the slave robot during the fine alignment process. By gradually guiding and reducing the posture deviation between the master and slave robots, the alignment accuracy between the master and slave robots can be improved, thereby improving the accuracy of teleoperation control. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of an application environment for the control method of the teleoperated robot provided in this embodiment of the present disclosure. Figure 2 This is a flowchart illustrating the control method for a teleoperated robot provided in an embodiment of this disclosure; Figure 3 yes Figure 2 A flowchart further includes step S103; Figure 4 yes Figure 3 A flowchart further included in step S202; Figure 5 yes Figure 4 A flowchart illustrating the further steps following step S302; Figure 6 This is a schematic diagram of a scenario illustrating the attitude alignment process of a teleoperated robot provided in an embodiment of this disclosure; Figure 7 yes Figure 2 Another process diagram further included in step S103; Figure 8 This is a schematic diagram of the functional modules of the control device for the teleoperated robot provided in this embodiment of the disclosure; Figure 9This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this disclosure. Detailed Implementation

[0018] To enable those skilled in the art to better understand the solutions disclosed herein, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0019] It is understood that in the specific embodiments of this disclosure, which involve retrieving initial time-series data, initial sample time-series data and related data, when the above embodiments of this disclosure are applied to specific products or technologies, permission or consent from the target is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0020] Furthermore, when the embodiments of this disclosure require access to initial timing data, initial sample timing data, and related data, separate permission or consent to the initial timing data, initial sample timing data, and related data will be obtained through pop-up windows or redirection to a confirmation page. After clearly obtaining separate permission or consent to the initial timing data, initial sample timing data, and related data, the necessary initial timing data, initial sample timing data, and related data for enabling the embodiments of this disclosure to operate normally will be obtained.

[0021] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0022] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations: Artificial intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0023] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0024] With the development of science and technology, teleoperation control is widely used in surgical robots, industrial robots and humanoid robots. During teleoperation control, the operator can control the position and posture of the slave robot through the master robot and observe the movement of the slave robot through the image feedback on the display. The posture of the master robot needs to be consistent with that of the slave robot. Therefore, teleoperation control requires the alignment of the postures of the master robot and the slave robot to ensure the accuracy of teleoperation control.

[0025] In related technologies, the method of aligning the slave robot with the master robot through active movement carries significant risks, especially when applied in the field of surgical robots. Before master-slave alignment is completed, significant movement of the slave robot could potentially cause additional harm to the patient. Therefore, in the process of aligning the postures of the master and slave robots, the active driving capability of the master robot is often used to enable it to actively move and align with the posture of the slave robot. However, this method requires a high level of driving capability from the master robot. In scenarios where the master robot lacks active driving capability or has limited driving capability, the difficulty of alignment is easily increased, leading to a reduction in the alignment accuracy between the master and slave robots, and ultimately reducing the accuracy of teleoperation control.

[0026] To address the aforementioned problems, this disclosure provides a control method, apparatus, device, and medium for a teleoperated robot, which improves the alignment accuracy between the master robot and the slave robot, ultimately enhancing the accuracy of teleoperation control.

[0027] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment for the control method of the teleoperated robot provided in this embodiment of the disclosure, including: terminal 11 and server 12.

[0028] For example, in response to an alignment command between the master robot and the slave robot, the server 12 can display a virtual master robot image and a virtual slave robot image on a preset display interface on the terminal 11 in real time, based on the real-time pose information of the master robot and the slave robot; generate coarse alignment prompts on the display interface to prompt adjustment of the master robot's posture until the posture deviation between the virtual master robot image and the virtual slave robot image meets preset conditions; generate fine alignment prompts on the display interface to prompt small-range adjustment of the master robot's posture, and control the slave robot to selectively follow the posture movement of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

[0029] Terminal 11 can be a mobile phone, computer, smart voice interaction device, smart wearable device, smart home appliance, vehicle terminal, etc., but is not limited to these. Terminal 11 can also independently execute the control method of remotely operating the robot. Terminal 11 and server 12 can be directly or indirectly connected through wired or wireless communication, and this embodiment of the disclosure does not impose any limitations.

[0030] Server 12 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Additionally, server 12 can also be a node server in a blockchain network.

[0031] It should be noted that, Figure 1 The schematic diagram of the implementation environment shown is merely an example. The scenarios described in this disclosure are intended to more clearly illustrate the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided in this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new business scenarios, the technical solutions provided in this disclosure are also applicable to similar technical problems.

[0032] Please see Figure 2 , Figure 2This is a flowchart illustrating a control method for a teleoperated robot provided in an embodiment of this disclosure. This control method for a teleoperated robot can be applied to the server in the above embodiment, or can be jointly executed by a terminal and a server. The control method for a teleoperated robot includes steps S101 to S103: Step S101: In response to the alignment command between the master robot and the slave robot, based on the real-time pose information of the master robot and the slave robot, the virtual master robot image of the master robot and the virtual slave robot image of the slave robot are displayed in real time on the preset display interface. Step S102: Generate coarse alignment prompt information on the display interface to prompt the adjustment of the main robot's posture until the posture deviation between the virtual main robot image and the virtual slave robot image meets the preset conditions. Step S103: Generate fine alignment prompt information on the display interface to prompt small-range adjustments to the posture of the master robot, and control the slave robot to selectively follow the posture movement of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

[0033] Regarding step S101 above, the control device for the teleoperated robot in this embodiment can be a control system for the teleoperated robot, also referred to as a teleoperation system. In response to the alignment command from the master robot and the slave robot, the slave robot acquires the positions of each joint and combines them with forward kinematics to calculate the absolute pose of its end effector. Then, based on the pose relationship between the camera and the slave robot, it maps this pose to the camera coordinate system. The master robot acquires the pose of its end effector relative to the master robot's base coordinate system in real time, and, combined with the pre-calibrated pose relationship between the display and the master robot's base coordinate system, transforms the master robot's end effector pose to the display coordinate system. Subsequently, on the preset display interface of the master robot, it renders a virtual slave robot end effector tool (its pose is synchronized in real time with the actual end effector pose of the slave robot in the camera coordinate system, restoring the geometry of the slave robot's end effector) and a virtual master robot end effector tool (its pose is bound to the master robot's end effector pose in the display coordinate system; the virtual master tool moves synchronously with the master robot's movement and has the same geometry as the virtual slave tool), realizing the real-time display of two types of virtual images. Wherein: The teleoperation system includes a master robot and a slave robot. The master robot, located at the main end, is the interactive device directly controlled by the operator, conveying the operator's control intentions. The master robot can have various structures to adapt to different application scenarios, such as a master end effector or robotic arm, and can utilize magnetic positioning navigation, optical navigation, or a passive teleoperation handle. Therefore, in subsequent embodiments of this disclosure, the master robot can be simply referred to as the master end effector, and this disclosure does not impose specific limitations on it. The slave robot is the robotic device at the execution end of the teleoperation system that performs the actual tasks and postures and is guided and adjusted by the master end effector. The slave robot can also have various structures to adapt to different application scenarios. Therefore, the teleoperation system in this disclosure can be applied to human-computer interaction tasks in the fields of surgical robots and industrial robots, and this disclosure does not impose specific limitations on it.

[0034] The alignment command is the start signal that triggers the alignment of the master and slave robots. It can be initiated by the operator (manually triggered) or automatically by the system. For example, when a scenario such as switching from the slave robot, joint limits, or unusual poses is detected that causes the master and slave postures to become inaccurate, the alignment command will be automatically triggered. Alternatively, the alignment command will be triggered when the remote operating system is turned on or when the system is powered on for the first time.

[0035] The real-time pose information of the master robot and the slave robot describes the position and orientation information of the end effector of the master robot and the end effector of the slave robot in a specific coordinate system. The pose information can be dynamically updated, that is, it can be dynamic data.

[0036] The preset display interface is a visual feedback device set on the master end, that is, the display interface on the monitor. This monitor can be a 2D monitor or a 3D monitor. It is the core interactive carrier for the operator to observe the posture deviation between the master and slave robots and receive alignment prompts and other information.

[0037] The virtual master robot image is a master virtual tool rendered on the display interface and bound to the master robot's end effector posture. Its geometry is consistent with that of the slave robot's end effector. The purpose is to allow the operator to observe the movement of the virtual master robot image by controlling the master robot, thereby establishing an intuitive operational association. The virtual slave robot image is a slave virtual tool rendered on the display interface and synchronized with the slave robot's actual end effector posture. Its pose directly reflects the slave robot's real state and serves as the target reference for the operator to adjust the master robot's posture.

[0038] Furthermore, the teleoperation system also includes a camera on the slave robot to acquire images and other information. During teleoperation alignment, a mapping relationship between the virtual master robot image and the virtual slave robot image needs to be established using a corresponding coordinate system. Specifically, on the slave robot, the positions of each joint can be acquired, and the position of the slave robot's end effector can be calculated using forward kinematics. and posture Multiple robotic arms and actuators are available, and no specific restrictions are imposed here. Next, the camera's position relative to the robot's orientation is considered. By mapping the position and orientation of the robot's end effector to the camera coordinate system, the position of the robot's end effector in the camera coordinate system can be obtained. and posture Finally, the obtained information is transmitted to the master unit for display, where the position of the robot's end effector in the camera coordinate system is displayed. and posture The position and orientation of the virtual robot end effector are used to display the corresponding virtual robot image in the display interface based on the camera coordinate system.

[0039] Similarly, on the master end, regardless of the type of master robot, the position of the master robot's end effector relative to the master end base coordinate system can be obtained in real time. and posture Next, by pre-calibrating, the orientation of the display relative to the master coordinate system can also be obtained. This allows us to obtain the position of the end effector of the main robot in the coordinate system of the main display. and posture At this point, the position of the virtual master robot's end effector in the display interface is defined. and posture Since the virtual master robot's end effector's posture always remains consistent with the master robot's end effector's posture and changes with the master robot's end effector's posture, it can be obtained that... Ultimately, the virtual master robot image of the master robot and the virtual slave robot image of the slave robot are displayed in real time.

[0040] Regarding step S102 above, this embodiment of the present disclosure can generate coarse alignment prompts on the display interface based on the previously constructed vision, guiding the operator to adjust the posture of the main robot through movements such as moving and swinging the wrist, thereby controlling the pose of the virtual main robot's end effector. Simultaneously, the system calculates the posture deviation between the virtual main robot's end effector and the virtual slave robot's end effector in the display coordinate system in real time. When this deviation meets preset conditions, the system prompts the operator to complete the coarse alignment. Wherein: Coarse alignment prompts are interactive signals used to guide operators in making initial posture adjustments. These prompts can take the form of text, graphics, or audio-visual cues. Their core function is to clarify the operational goal (such as aligning two types of virtual images) and reduce the operator's trial-and-error costs. There are several ways to trigger coarse alignment prompts. For example, after responding to the alignment command and displaying the virtual master robot image and the virtual slave robot image, the posture deviation between the two images can be acquired in real time, and coarse alignment prompts can be continuously generated when the posture deviation does not meet preset conditions. Alternatively, a single coarse alignment prompt can be issued after responding to the alignment command and displaying the virtual master robot image and the virtual slave robot image. Another approach is to provide coarse and fine alignment options after responding to the alignment command and displaying the virtual master robot image and the virtual slave robot image, and issue a coarse alignment prompt after the operator selects the coarse alignment option.

[0041] The pose deviation between the virtual master robot image and the virtual slave robot image is a quantitative indicator used to describe the difference in end-effector pose between the virtual master robot and the virtual slave robot, and also corresponds to the pose difference between the master robot and the slave robot. Furthermore, this pose deviation can be obtained through the pose deviation vector feedback between the virtual master robot image and the virtual slave robot image.

[0042] The preset conditions are used to determine whether coarse alignment is complete. For example, in this embodiment, a corresponding deviation threshold can be set for the posture deviation vector, and the preset condition can be set such that the posture deviation vector between the master and slave robots is less than the deviation threshold, or the preset condition can be set such that the posture deviation vector between the master and slave robots is less than or equal to the deviation threshold, thereby determining whether coarse alignment is complete. Furthermore, the required deviation threshold can be set according to the scenario requirements. The deviation threshold essentially transforms the visually apparent near-identical postures into a calculable numerical standard, avoiding subjective judgment errors.

[0043] For example, after starting the master-slave robot alignment function, the display interface shows a virtual slave robot image. This virtual slave robot image is a virtual image with a certain degree of transparency; this transparency makes it more intuitive and convenient for the operator to perform coarse alignment. Then, the interface displays coarse alignment prompts, instructing the operator to adjust the posture of the virtual master robot's end effector by swinging the master robot's end effector. At this time, the posture deviation between the virtual master robot image and the virtual slave robot image in the display coordinate system can be calculated in real time. After exponential mapping, it is transformed into a three-dimensional vector. When the posture deviation meets the preset condition, that is, when the magnitude of the posture deviation vector is less than or equal to the set deviation threshold, the operator is prompted to complete the coarse alignment; otherwise, the operator is prompted to continue adjusting the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets the preset condition.

[0044] Regarding step S103 above, this embodiment of the disclosure can generate fine alignment prompts through the display interface after entering the fine alignment stage, guiding the operator to make small-range swings to adjust the posture of the master robot, while keeping the end effector stationary. Since the posture increment of the master robot can be divided into two parts—one part reducing the posture deviation between the master and slave robots, and the other part increasing the posture deviation—this embodiment of the disclosure only controls the slave robot to selectively follow the posture movement of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases. Wherein: Fine alignment prompts are interactive signals that guide the operator to make small-scale adjustments to the posture. Similar to coarse alignment prompts, fine alignment prompts can take the form of text, graphics, or audio-visual cues. Their core function is to constrain the range of operations, preventing risks that may arise during slave-side follow-up control due to large adjustments by the operator. There are several ways to trigger fine alignment prompts. For example, a fine alignment prompt can be issued immediately after coarse alignment; alternatively, a fine alignment option can be provided after coarse alignment, and the prompt can be issued after the operator selects the option; or, after coarse alignment, the remote operating system can be allowed to enter the application phase, and the fine alignment prompt can be generated during the application process based on the operation initiated by the master.

[0045] The small-scale adjustment of the master robot's posture refers to the constraint on the operator's actions during the fine alignment stage. It should be noted that, in this embodiment, during the fine alignment stage, the operator needs to be guided to adjust the master robot's posture and control the slave robot to selectively follow parts of the master robot's movements. Specifically, in this embodiment, the operator needs to actively adjust the master robot's posture during the fine alignment stage, but there are requirements on the master robot's range of motion. Specifically, the magnitude of the master robot's posture increment must be less than a certain threshold, thereby allowing the master robot to adjust its posture within a small range. The purpose is to avoid the master robot making large movements and ensure that the slave robot's following motion is always a small-amplitude, highly safe, selective following of the master robot's posture movement.

[0046] Selective following control is the core control logic of a robot. It controls the slave robot to follow the master robot's movement only when the posture deviation between the master and slave robots is reduced. During the following control process, the slave robot refuses to follow movements that would increase the master-slave deviation. So, its essence is to screen the movement of the slave robot to prevent the slave robot from blindly following the master robot's movement before the alignment operation is completed. It also allows the slave robot to selectively follow the master robot's movement when it can reduce the posture deviation between the two robots.

[0047] In summary, this embodiment of the present disclosure, through the control method of the teleoperated robot in steps S101 to S103, displays the virtual master robot image of the master robot and the virtual slave robot image of the slave robot in real time on a preset display interface. When the operator moves the master robot, the virtual master robot image on the display interface also moves in real time. Then, by generating coarse alignment prompt information on the display interface, the operator can move the posture of the master robot under the guidance of the coarse alignment prompt information and intuitively observe the posture deviation between the virtual master robot image and the virtual slave robot image until the posture deviation meets the preset conditions and the coarse alignment operation is completed. Then, fine alignment prompt information can be generated on the display interface to prompt the operator to adjust the posture of the master robot in a small range, so as to avoid the risk of the slave robot moving a large distance before the master-slave alignment is completed, and control the slave robot to follow the movement of the master robot when the posture deviation between the master robot and the slave robot is reduced, so as to gradually reduce the posture deviation between the master and slave robots and finally complete the master-slave alignment operation. Therefore, in scenarios where the master robot lacks active driving capability or has limited driving capability, this embodiment of the present disclosure reduces the difficulty of aligning the master and slave robots by first coarse alignment and then fine alignment, and selectively following the slave robot during the fine alignment process. By gradually guiding and reducing the posture deviation between the master and slave robots, the alignment accuracy between the master and slave robots can be improved, thereby improving the accuracy of teleoperation control.

[0048] The following is a detailed description of the further contents included in steps S101 to S103 in the embodiments of this disclosure.

[0049] Please see Figure 3 , Figure 3 yes Figure 2 The flowchart further includes step S103. In some embodiments, when the pose deviation between the virtual master robot image and the virtual slave robot image decreases, the process of controlling the slave robot to selectively follow the pose movement of the master robot may further include steps S201 to S203: Step S201: In real time, acquire the attitude increment vector of the master robot during the fine alignment attitude adjustment process, as well as the attitude deviation vector between the virtual master robot image and the virtual slave robot image. Step S202: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle, the target attitude increment is generated based on the corresponding mapping vector. Step S203: Adjust the pose of the slave robot based on the target pose increment to follow the pose movement of the master robot.

[0050] In the above steps, during the selective following control of the slave robot in this embodiment of the present disclosure, the attitude increment vector of the master robot during the fine alignment attitude adjustment process, and the attitude deviation vector between the virtual master robot image and the virtual slave robot image can be acquired in real time. Wherein: The master robot attitude increment vector refers to the vector corresponding to the attitude change of the master robot within a single control cycle (e.g., from time t to t+Δt) during the fine alignment phase. The master robot attitude increment vector can be denoted as... ,in This represents the pose vector of the main robot at time t. This represents the master robot's attitude vector at time t+Δt. The physical meaning of the master robot's attitude increment vector is to quantify the magnitude and direction of minute changes in the master robot's attitude, and it is a key basis for judging whether the master robot's actions can reduce master-slave bias.

[0051] The pose deviation vector between the virtual master robot image and the virtual slave robot image refers to the vector corresponding to the pose difference between the ends of the virtual master robot image and the virtual slave robot image in the master display interface. Further, the pose deviation between the virtual master robot image and the virtual slave robot image can be denoted as... The corresponding vector, i.e., the attitude deviation vector, is denoted as... Because the virtual master robot's image pose is completely synchronized with the actual master robot's pose, that is... The attitude deviation vector is essentially the same as the vector corresponding to the attitude deviation between the master robot and the actual slave robot end effector. Therefore, the obtained attitude deviation vector is the core benchmark for selecting the follow-up action of the slave robot.

[0052] It should be noted that the posture adjustment of the master robot during the fine alignment stage is a process of small-range, dynamic swaying by the operator. If there is a delay in vector acquisition, it will cause the follow-up action of the slave robot to become disconnected from the actual action of the master robot, which may result in misfollowing or missed following. Therefore, real-time is a prerequisite for ensuring the accuracy and safety of selective following control.

[0053] Next, when the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle, a target attitude increment is generated based on the corresponding mapping vector. This allows for the selection of effective master robot actions that reduce master-slave deviation by judging the vector direction, while eliminating invalid actions that increase deviation. Wherein: The tangent space of the attitude deviation vector is the Lie algebra space corresponding to the attitude deviation vector when the attitude deviation vector is represented by a rotation matrix. Vectors in this space can represent angular velocities (i.e., the rate and direction of attitude change), and it serves as a bridge connecting the rotation matrix (attitude) and the vector (increment). The mapping vector refers to the vector that transforms the master robot's attitude increment vector from the master hand's attitude space to the tangent space of the attitude deviation vector. The preset angle is a pre-set angle in this embodiment, and its core function is to serve as a quantitative screening threshold for effective master hand movements.

[0054] It should be noted that when the angle between the mapping vector of the master robot's posture increment vector in the deviation tangent space and the posture deviation vector is less than or equal to a preset angle, it means that the direction of the master hand's movement is consistent with the direction of reducing master-slave deviation, which is a valid movement and can be further converted into the slave robot's target posture increment. Conversely, the movement will increase the deviation and needs further processing or elimination. Finally, a valid target increment, i.e., the target posture increment, can be generated based on the corresponding mapping vector. Therefore, this embodiment of the present disclosure accurately distinguishes the master hand's movements that reduce / increase deviation by using the preset angle threshold, while ensuring that the slave robot only follows valid movements and does not generate autonomous movements to ensure safety. At the same time, it promotes the continuous convergence of master-slave posture deviation in the fine alignment stage over multiple control cycles, ultimately achieving high-precision alignment.

[0055] Subsequently, embodiments of this disclosure can adjust the posture of the slave robot based on the target posture increment to follow the posture movement of the master robot, thereby converting the selected effective target increment into a posture control quantity that the slave robot can execute, while ensuring the safety and high precision of the adjustment process.

[0056] It should be noted that the mapping vector generated by the above steps is a posture deviation tangent space vector, which cannot be directly used for joint control of the robot. It is necessary to convert the mapping vector under the corresponding conditions into the actual posture increment of the robot to obtain the target posture increment. In this way, the abstract vector increment can be mapped to the posture change of the robot in physical space, thereby ensuring that the robot can understand and execute the target posture increment.

[0057] Furthermore, the robot's posture adjustment needs to be completed within a single control cycle, i.e., synchronized with the real-time acquisition in step S201. Specifically, in this embodiment, the current posture of the slave robot can be calculated in real time, the target posture increment is superimposed on the current posture to obtain the target posture, and then the joint controller of the slave robot drives the joint movement to achieve the target posture. This process is repeated in each control cycle of the fine alignment phase. As the master robot continues to swing within a small range, the effective increment accumulates continuously, and the slave robot's posture gradually approaches the master robot's posture, eventually achieving a smaller and smaller master-slave posture deviation. After multiple control cycles, the master and slave postures become consistent, completing the fine alignment goal. At the same time, since the slave robot's posture adjustment is always based on the effective actions of the master robot, the movement of the slave robot is always within the operator's control, meeting the safety requirement of not generating other or additional autonomous movements.

[0058] Please see Figure 4 , Figure 4 yes Figure 3 The flowchart further includes step S202. In some embodiments, when the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle, the process of generating the target attitude increment based on the corresponding mapping vector may further include steps S301 to S303: Step S301: Determine the left-multiplied Jacobian matrix and the right-multiplied Jacobian matrix corresponding to the attitude deviation vector; Step S302: The attitude increment vector is mapped by right multiplication by the Jacobian matrix to obtain the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector. Step S303: When the angle between the mapping vector and the attitude deviation vector is less than or equal to a preset angle, the mapping vector is inversely mapped by left multiplying it by the Jacobian matrix to obtain the target attitude increment.

[0059] In the above steps, the embodiments of this disclosure can determine the left-multiplied Jacobian matrix and the right-multiplied Jacobian matrix corresponding to the attitude deviation vector, so as to provide a reasonable calculation basis for the subsequent spatial transformation of the master hand increment and the generation of the robot increment. Wherein: Left-multiplying the Jacobian matrix (denoted as ) ) and right-multiplied Jacobian matrix (denoted as ) Both are based on the attitude deviation matrix corresponding to the attitude deviation vector. The calculated key matrix serves as a bridge between Lie groups and Lie algebras; the two Jacobian matrices are essentially attitude deviation matrices. The resulting computational load. Furthermore, embodiments of this disclosure can calculate the posture deviation matrix from the robot's actual end-effector posture and the master hand posture. Then, according to The corresponding rotation axis and rotation angle, combined with the standard calculation formula of the left and right multiplication Jacobian matrix of the SO(3) Lie group, are substituted into the parameters to calculate the left and right multiplication Jacobian matrices that match the attitude deviation vector.

[0060] Next, in this embodiment of the present disclosure, the attitude increment vector can be mapped by right multiplying by the Jacobian matrix to obtain the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector. This is the core step of spatial transformation for judging the effectiveness of the action in the fine alignment stage, so as to map the main hand increment to the deviation tangent space.

[0061] It should be noted that the core purpose of the mapping process is to transform the master robot's attitude increment vector from the master hand's attitude space to the tangent space of the attitude deviation vector, thereby obtaining a mapping vector that can be directly compared with the attitude deviation vector. Furthermore, in obtaining the mapping vector, this embodiment of the disclosure also needs to determine the inverse of the right-multiplied Jacobian matrix based on the right-multiplied Jacobian matrix, through the inverse operation. Only then can the transformation from the master hand pose space vector to the tangent space vector be achieved, and the resulting mapping vector can be denoted as: And because It describes the changes in the dominant hand's posture, while This describes the difference in master-slave posture, where the two have different spatial references. They need to be mapped to the same space (deviation tangent space) in order to determine whether the deviation can be reduced.

[0062] Subsequently, when the angle between the mapping vector and the attitude deviation vector is less than or equal to the preset angle, the mapping vector is inversely mapped by multiplying it by the Jacobian matrix on the left to obtain the target attitude increment. This is the execution conversion link for the effective action landing in the fine alignment stage, which can convert the effective mapping vector in the tangent space into the attitude control quantity that the robot can physically execute.

[0063] The inverse mapping process transforms valid mapping vectors in the tangent space into executable increments from the robot's end-effector pose space. This is because the mapping vectors... The abstract Lie algebra so(3) space vector only describes the direction and magnitude of angular velocity and cannot directly drive the movement of robot joints. Therefore, it is necessary to transform the tangent space to the robot posture space by left multiplying the Jacobian matrix to obtain the target posture increment, which directly corresponds to the posture change that needs to be adjusted from the robot end. It can be further transformed into joint angle increments from the robot's inverse kinematics to drive the robot's movement.

[0064] Furthermore, the preset angle can be set to 90 degrees. When the angle between the mapping vector and the attitude deviation vector is less than or equal to 90 degrees, and Same direction means that the equivalent effect of the dominant hand's movement is to reduce the master-slave posture deviation; conversely, if the angle is greater than 90 degrees, and Conversely, the actions of the main robot will increase the posture deviation.

[0065] It's important to note that without the left and right Jacobian matrices, the pose increments of the master robot and the slave robot will exist in different spatial dimensions, making effective conversion impossible. The master robot's increment belongs to its own pose change space, while the slave robot's increment belongs to the slave robot's end effector's pose control space. The pose deviation tangent space is the intermediate unified space between the two. Only by mapping from the master robot to the tangent space through the right Jacobian matrix can the direction be compared with the pose deviation vector. Only by performing the inverse mapping from the tangent space to the slave robot through the left Jacobian matrix can the effective motion be converted into an executable control quantity for the slave robot.

[0066] Please see Figure 5 , Figure 5 yes Figure 4 The flowchart following step S302 is shown in the following embodiments. In some embodiments, after mapping the attitude increment vector to the attitude deviation vector tangent space by right multiplication with the Jacobian matrix to obtain the mapping vector of the attitude increment vector, steps S401 to S403 may be further included: Step S401: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, obtain the vector component of the mapping vector in the direction of the attitude deviation vector. Step S402: Remove the vector components from the mapping vector to obtain the processed mapping vector; Step S403: The processed mapping vector is inversely mapped by left multiplying it by the Jacobian matrix to obtain the target attitude increment.

[0067] In the above steps, when the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than the preset angle, the mapping vector needs to be adjusted in order to generate a target attitude increment that will not increase the attitude deviation vector. Ultimately, this satisfies the goal of controlling the slave robot to selectively follow the attitude movement of the master robot when the attitude deviation between the virtual master robot image and the virtual slave robot image is reduced.

[0068] Here, the vector component refers to the projection component of the mapped vector in the direction of the attitude deviation vector. When the angle between the mapped vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, the direction of this projection component is completely opposite to that of the attitude deviation vector, indicating that the master robot's actions will increase the master-slave bias during this motion cycle. Therefore, by obtaining this inverse component, a clear target is provided for subsequent accurate elimination, avoiding the accidental deletion of effective components due to directly eliminating the entire mapped vector, and ensuring that the slave robot can still retain a usable safe action basis.

[0069] Next, in this embodiment of the present disclosure, the vector components in the mapping vector can be removed to obtain the processed mapping vector. The processed mapping vector only contains the component that is perpendicular to the attitude deviation vector, and the angle between it and the vector is always 90 degrees. This component neither increases the master-slave deviation nor actively reduces the deviation, and is a safe neutral component.

[0070] Finally, embodiments of this disclosure can perform inverse mapping on the processed mapping vector by left multiplying it by the Jacobian matrix to obtain the target pose increment, thereby converting the safe mapping vector after removing the inverse component into a pose control quantity that can be physically executed by the robot.

[0071] It should be noted that the processed mapping vector also belongs to the tangent space of the attitude deviation vector, that is, the vector of the so(3) Lie algebra space. It only describes the direction and magnitude of the angular velocity and cannot directly drive the movement of the robot joint. Therefore, it is necessary to complete the inverse mapping transformation from the tangent space to the robot attitude space by left multiplying the Jacobian matrix, and transform the abstract vector into an attitude increment that can be executed from the robot end.

[0072] The final target pose increment corresponds only to a small pose change perpendicular to the master-slave pose deviation direction. It will not increase the master-slave deviation. Since the master robot needs to adjust its pose within a small range during the fine alignment stage, the magnitude of this increment is small and belongs to the pose adjustment amount of a high safety level. Therefore, the target pose increment acts on the pose adjustment of the slave robot, ensuring that the slave robot can still perform pose adjustment based on the safety component after processing. This promotes the continuous convergence of the master-slave pose deviation during the fine alignment stage, while ensuring the safety and efficiency of the alignment process.

[0073] In some embodiments, after mapping the attitude increment vector to the attitude deviation vector tangent space by right multiplication with the Jacobian matrix to obtain the mapped vector of the attitude increment vector, the process may further include: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, the robot's attitude remains unchanged.

[0074] In the above steps, when the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, this embodiment of the present disclosure can not only remove the vector component in the mapping vector that is opposite to the direction of the vector where the attitude deviation vector is located, and generate the target attitude increment for controlling the movement of the robot based on the processed mapping vector, but also directly control the attitude of the robot to remain unchanged.

[0075] It should be noted that keeping the slave robot's posture unchanged is a direct response to invalid master hand movements. Its underlying logic is entirely consistent with the safety requirement that the slave robot does not generate autonomous movement and that its movement remains under the operator's control. This further ensures that the slave robot will not cause collisions or amplify deviations due to following invalid movements. Compared to the processing method in the above embodiments that removes the reverse component while retaining the vertical component, keeping the posture unchanged directly avoids the slight posture fluctuations that the vertical component may cause. Although the vertical component does not increase the deviation, it still produces posture changes. Therefore, it can be applied to high-risk scenarios requiring zero posture fluctuations, such as surgical robots (e.g., when near sensitive tissues) and precision industrial assembly (e.g., near fragile workpieces). Therefore, when the system has low requirements for alignment efficiency but extremely high requirements for safety, the method of keeping the slave robot's posture unchanged can be preferred, thus allowing the embodiments of this disclosure to cover the invalid movement processing needs in different application scenarios.

[0076] In some embodiments, the process of generating a coarse alignment prompt message on the display interface to prompt adjustment of the master robot's posture until the posture deviation between the virtual master robot image and the virtual slave robot image meets a preset condition may further include: A coarse alignment prompt is generated on the display interface to prompt the adjustment of the position and posture of the master robot so that the virtual master robot image and the virtual slave robot image coincide, until the posture deviation between the virtual master robot image and the virtual slave robot image meets the preset conditions.

[0077] Furthermore, in the coarse alignment process of this embodiment, the position of the main robot can be adjusted. The coarse alignment prompt information can further include prompts for the operator to adjust the position of the main robot, combined with prompts for adjusting the posture of the main robot. Ultimately, it can prompt the operator to adjust the position and posture of the main robot so that the virtual main robot image and the virtual slave robot image overlap. Overlapping of the virtual main robot image and the virtual slave robot image means that, in the display interface, the position coordinates and posture angles of the virtual main robot image are completely consistent with those of the virtual slave robot image. This is a direct criterion for judgment in the coarse alignment stage. Compared to posture matching alone, complete overlap can minimize the initial pose deviation, allowing subsequent fine alignment to achieve high-precision alignment with only minor adjustments.

[0078] For example, please see Figure 6 , Figure 6 This is a schematic diagram of a scenario illustrating the posture alignment process of a teleoperated robot provided in this embodiment. In this example, the master robot is equipped with a master end effector, and the slave robot is equipped with an end effector. The shape of the master end effector is consistent with the shape of the slave robot's end effector. The display interface shows a virtual image of the slave robot (the actual end effector in the figure) and a virtual image of the master robot (the virtual end effector in the figure) of the master end effector. When the operator moves the position and posture of the master end effector, the virtual master robot image (the virtual end effector in the figure) in the display interface will also adjust its position and posture accordingly. During the coarse alignment process, the operator can observe whether the virtual master robot image (the virtual end effector in the figure) and the virtual slave robot image (the actual end effector in the figure) overlap by adjusting the position and posture of the master end effector. When they overlap, the coarse alignment operation is completed.

[0079] It should be noted that, by initially overlapping the positions and aligning the postures of the master robot and the slave robot in this embodiment, the operator can visually guide the alignment of the positions and postures of the virtual master robot end effector and the virtual slave robot end effector under the guidance of the display interface. Finally, the posture deviation is quantified by meeting preset conditions, ensuring that the deviation after coarse alignment is controllable. This avoids the problem of low efficiency or failure in fine alignment due to excessive initial posture deviation, thus improving the effectiveness of coarse alignment and laying a solid foundation for subsequent fine alignment.

[0080] Furthermore, when the operator is prompted to move the position and posture of the master robot during the coarse alignment process, in the subsequent fine alignment process, this embodiment of the present disclosure needs to keep the position of the slave robot unchanged. Then, when the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot. This avoids the safety risks caused by the movement of the slave robot's position during the fine alignment stage, and also avoids the risk of increasing the posture deviation between the master and slave robots due to the movement of the slave robot's position.

[0081] It should be noted that the embodiments of this disclosure prioritize aligning the postures of the master and slave robots. To further improve the accuracy of posture alignment, the alignment of the positions between the master and slave robots is comprehensively considered in the coarse alignment stage, while the position of the slave robot remains unchanged in the fine alignment stage. This is because consistent robot posture is key to ensuring the intuitiveness of teleoperation. In the surgical, industrial, and other scenarios, posture deviation has a much greater impact on operational accuracy than position deviation. Posture alignment is a prerequisite for ensuring operational precision. Master-slave posture deviation will directly lead to incorrect operation direction, while position deviation has a larger margin of error and can be corrected in subsequent operations. Therefore, the embodiments of this disclosure are suitable for scenarios without a driven master hand. The operator can easily manually adjust the posture of the master hand, and the position adjustment only needs to be initially completed in the coarse alignment stage or corrected in the subsequent stage.

[0082] Please see Figure 7 , Figure 7 yes Figure 2 Another flowchart further includes step S103. In some embodiments, during the process of generating fine alignment prompts on the display interface to prompt small-range adjustments to the master robot's posture, and controlling the slave robot to selectively follow the master robot's posture movement as the posture deviation between the virtual master robot image and the virtual slave robot image decreases, steps S501 to S502 may be further included: Step S501: In response to the fine alignment start command, a fine alignment prompt message is generated on the display interface to prompt a small adjustment of the master robot's posture. When the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot. Step S502, or, during the movement of the master robot, when the posture deviation between the virtual master robot image and the virtual slave robot image decreases, control the slave robot to selectively follow the posture movement of the master robot.

[0083] As can be seen from the above steps, there are two methods for performing fine alignment in this embodiment of the disclosure. Among them: The first fine alignment method is to generate fine alignment prompt information on the display interface in response to the fine alignment start command, so as to prompt the master robot to make small adjustments to its posture, and control the slave robot to selectively follow the posture movement of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

[0084] In this embodiment, the fine alignment start command is used as the trigger signal to clearly start the fine alignment stage. Then, the fine alignment prompt information on the display interface is used to constrain the operator's movement range (small-range adjustment) to ensure that the master robot's movement is controllable. Then, the slave robot is controlled to selectively follow only when the master-slave posture deviation decreases, ensuring that the slave robot does not generate autonomous movement.

[0085] In this embodiment, the fine alignment start command can be triggered in two ways: first, it can be automatically generated by the system after coarse alignment is completed; second, it can be manually triggered by the operator, such as by clicking the "Start Fine Alignment" button on the display interface, which is suitable for scenarios where confirmation is required after coarse alignment. The core function of this command is to activate the prompts and control logic related to fine alignment, ensuring that the fine alignment stage starts in an orderly manner.

[0086] Therefore, this embodiment ensures the independence and controllability of the fine alignment stage through instruction triggering and explicit guidance, reduces the difficulty of operation through visual cues, and ensures safety by relying on selective following, making it particularly suitable for scenarios with high requirements for standardized operation procedures.

[0087] The second method of fine alignment involves dynamically incorporating fine alignment during the movement of the main robot. In this embodiment, after coarse alignment is completed, the operator can control the slave robot to perform master-slave operations by moving the main robot. During the movement of the main robot, as the posture deviation between the virtual main robot image and the virtual slave robot image decreases, this embodiment then controls the slave robot to selectively follow the posture movement of the main robot. This allows for synchronous convergence of posture deviations during the normal movement of the main robot, improving operational efficiency.

[0088] The master robot motion process refers to the dynamic state in which the master robot is actively adjusted by the operator, including two typical scenarios: First, after coarse alignment is completed, the operator starts the master robot motion directly without initiating an independent fine alignment process, such as a surgical robot starting surgical operations after coarse alignment, or an industrial robot starting assembly operations after coarse alignment; Second, during remote operation, when the master and slave postures deviate due to situations such as the robot reaching joint limits or switching slave robotic arms, the operator adjusts the master robot's motion. The core feature of this motion process is that there is no independent fine alignment process, and the fine alignment logic is dynamically integrated with the master robot's regular operations.

[0089] It should be noted that in this embodiment, a separate fine alignment start command is not required. As long as the master robot is in motion, such as when it directly enters teleoperation after coarse alignment, or when a posture deviation occurs during teleoperation, the system monitors the master-slave posture deviation changes in real time. When the master robot's movement causes the virtual master-slave image posture deviation to decrease, the slave robot's selective following is automatically triggered to ensure that the deviation is always in a convergent trend, which neither affects normal teleoperation nor fails to complete fine alignment simultaneously.

[0090] Therefore, this embodiment, through dynamic integration design, omits the independent fine alignment step, improves teleoperation efficiency, and at the same time maintains the selective following logic of the slave robot to ensure safety and alignment accuracy, further expanding the scope of application of the embodiments of this disclosure.

[0091] Please see Figure 8This disclosure also provides a control device for a teleoperated robot, which can implement the above-described control method for a teleoperated robot. The control device for the teleoperated robot includes: The virtual display module 801 is used to respond to the alignment command between the master robot and the slave robot, and to display the virtual master robot image of the master robot and the virtual slave robot image of the slave robot in real time on a preset display interface based on the real-time pose information of the master robot and the slave robot. The coarse alignment module 802 is used to generate coarse alignment prompts on the display interface to prompt the adjustment of the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets the preset conditions. The fine alignment module 803 is used to generate fine alignment prompts on the display interface to prompt small-range adjustments to the posture of the master robot, and to control the slave robot to selectively follow the posture of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

[0092] In summary, the control device for the teleoperated robot, through the control method of the teleoperated robot described in the above embodiments, displays a virtual master robot image and a virtual slave robot image on a preset display interface in real time. When the operator moves the master robot, the virtual master robot image on the display interface also moves in real time. Then, by generating coarse alignment prompts on the display interface, the operator can move the master robot's posture according to the instructions of the coarse alignment prompts and intuitively observe the posture deviation between the virtual master robot image and the virtual slave robot image until the posture deviation meets the preset conditions and the coarse alignment operation is completed. Then, fine alignment prompts can be generated on the display interface to prompt the operator to make small adjustments to the posture of the master robot, avoiding the risk of large movements of the slave robot before the master-slave alignment is completed, and controlling the slave robot to follow the master robot's movement when the posture deviation between the master robot and the slave robot is reduced, the posture deviation between the master and slave robots can be gradually reduced, and the master-slave alignment operation can be completed. Therefore, in scenarios where the master robot lacks active driving capability or has limited driving capability, this embodiment of the present disclosure reduces the difficulty of aligning the master and slave robots by first coarse alignment and then fine alignment, and selectively following the slave robot during the fine alignment process. By gradually guiding and reducing the posture deviation between the master and slave robots, the alignment accuracy between the master and slave robots can be improved, thereby improving the accuracy of teleoperation control.

[0093] The specific implementation of the control device for the teleoperated robot is basically the same as the specific embodiment of the control method for the teleoperated robot described above, and will not be repeated here. Subject to meeting the requirements of the embodiments of this disclosure, the control device for the teleoperated robot may also be equipped with other functional modules to implement the control method for the teleoperated robot in the above embodiments.

[0094] This disclosure also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the control method for the teleoperated robot described above. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0095] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the control method of the teleoperated robot according to the embodiments of this disclosure. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0096] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the control method for the teleoperated robot described above.

[0097] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0098] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0099] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0100] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] Those skilled in the art will understand that all or some of the steps, apparatuses, or functional modules / units in the methods disclosed above can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0102] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0103] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0104] In the several embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between systems or units, and may be electrical, mechanical, or other forms.

[0105] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0106] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] The preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present disclosure. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of the present disclosure shall be within the scope of the claims of the present disclosure.

Claims

1. A control method for a teleoperated robot, characterized in that, include: In response to the alignment command between the master robot and the slave robot, based on the real-time pose information of the master robot and the slave robot, the virtual master robot image of the master robot and the virtual slave robot image of the slave robot are displayed in real time on a preset display interface. A coarse alignment prompt message is generated on the display interface to prompt the adjustment of the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets the preset conditions. The display interface generates a fine alignment prompt message to prompt small adjustments to the posture of the master robot. When the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot.

2. The control method for a teleoperated robot according to claim 1, characterized in that, When the pose deviation between the virtual master robot image and the virtual slave robot image decreases, controlling the slave robot to selectively follow the pose movement of the master robot includes: The attitude increment vector of the master robot during the fine alignment attitude adjustment process is acquired in real time, as well as the attitude deviation vector between the virtual master robot image and the virtual slave robot image. When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle, a target attitude increment is generated based on the corresponding mapping vector. The slave robot's posture is adjusted based on the target posture increment to follow the master robot's posture movement.

3. The control method for a teleoperated robot according to claim 2, characterized in that, When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is less than or equal to a preset angle, generating a target attitude increment based on the corresponding mapping vector includes: Determine the left-multiplied Jacobian matrix and the right-multiplied Jacobian matrix corresponding to the attitude deviation vector; The attitude increment vector is mapped by the right-multiplied Jacobian matrix to obtain the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector. When the angle between the mapping vector and the attitude deviation vector is less than or equal to a preset angle, the mapping vector is inversely mapped by multiplying it by the Jacobian matrix to obtain the target attitude increment.

4. The control method for a teleoperated robot according to claim 3, characterized in that, After mapping the attitude increment vector to the attitude deviation vector tangent space using the right-multiplied Jacobian matrix to obtain the mapped vector of the attitude increment vector, the control method for the teleoperated robot further includes: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, the vector component of the mapping vector in the direction of the attitude deviation vector is obtained. Remove the vector components from the mapping vector to obtain the processed mapping vector; The target attitude increment is obtained by performing inverse mapping on the processed mapping vector by left multiplying it by the Jacobian matrix.

5. The control method for a teleoperated robot according to claim 3, characterized in that, After mapping the attitude increment vector to the attitude deviation vector tangent space using the right-multiplied Jacobian matrix to obtain the mapped vector of the attitude increment vector, the control method for the teleoperated robot further includes: When the angle between the mapping vector of the attitude increment vector in the tangent space of the attitude deviation vector and the attitude deviation vector is greater than a preset angle, the attitude of the slave robot is controlled to remain unchanged.

6. The control method for a teleoperated robot according to claim 1, characterized in that, The step of generating coarse alignment prompts on the display interface to prompt adjustments to the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets preset conditions includes: A coarse alignment prompt is generated on the display interface to prompt the adjustment of the position and posture of the master robot so that the virtual master robot image and the virtual slave robot image coincide, until the posture deviation between the virtual master robot image and the virtual slave robot image meets the preset conditions.

7. The control method for a teleoperated robot according to claim 1, characterized in that, The step of generating fine alignment prompts on the display interface to prompt minor adjustments to the posture of the master robot, and controlling the slave robot to selectively follow the posture of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases, includes: In response to the fine alignment start command, a fine alignment prompt message is generated on the display interface to prompt a small adjustment of the posture of the master robot. When the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot. Alternatively, during the movement of the master robot, when the posture deviation between the virtual master robot image and the virtual slave robot image decreases, the slave robot is controlled to selectively follow the posture movement of the master robot.

8. A control device for a teleoperated robot, characterized in that, include: A virtual display module is used to respond to an alignment command between a master robot and a slave robot, and to display a virtual master robot image of the master robot and a virtual slave robot image of the slave robot in real time on a preset display interface based on the real-time pose information of the master robot and the slave robot. The coarse alignment module is used to generate coarse alignment prompts on the display interface to prompt the adjustment of the posture of the main robot until the posture deviation between the virtual main robot image and the virtual slave robot image meets a preset condition. The fine alignment module is used to generate fine alignment prompts on the display interface to prompt small-range adjustments to the posture of the master robot, and to control the slave robot to selectively follow the posture of the master robot when the posture deviation between the virtual master robot image and the virtual slave robot image decreases.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the control method of the teleoperated robot according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the control method for the teleoperated robot according to any one of claims 1 to 7.

Images