Teleoperation method, computing device and readable storage medium

Abstract

The invention discloses a teleoperation method, computing equipment and a readable storage medium, and the teleoperation method comprises the steps that a preset mode is entered in response to the fact that a preset triggering condition is met, and the preset triggering condition represents a scene or condition which causes a robot to have an abnormal action risk in the teleoperation process; and performing action speed-limiting control on the robot according to the input control attitude information and a preset teleoperation strategy. According to the technical scheme, when a scene or condition possibly causing abnormal actions of the robot in the teleoperation process is recognized, the specific mode corresponding to the slow action mechanism is started, so that the speed of the actions of the robot mapped by the input control posture information is limited in the specific mode, and the speed of the robot is increased. Therefore, when the control posture of the operator changes greatly or the control posture of the operator is greatly different from the posture of the robot, the actual action speed of the robot can be limited within a reasonable range, and the risk of abnormal action of the robot is avoided or reduced.

Description

Remote operation methods, computing devices and readable storage media Technical Field

[0001] This application relates to the field of robotics, and in particular to a teleoperation method, computing device, and computer-readable storage medium. Background Technology

[0002] In existing robot teleoperation methods, the operator (also known as a telemanipulator) uses motion capture suits, VR glasses, or exoskeletons to map the operator's control posture to the remote robot in order to achieve synchronous motion control. For example, the operator's arm or upper limb control posture is mapped to the remote robot's arm to achieve synchronous motion control of the robot's arm.

[0003] However, existing robot teleoperation methods can lead to abnormal robot movements (such as sudden jumps, unexpected large movements, or violent actions) when performing synchronized motion control in situations where the robot's posture differs significantly from the operator's. Therefore, addressing these shortcomings is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a teleoperation method, computing device, and computer-readable storage medium that can avoid or reduce the risk of abnormal robot movements during teleoperation by using a slow-motion mechanism, thereby improving the safety and stability of teleoperation and enhancing the user experience.

[0005] To achieve the above objectives: In a first aspect, embodiments of this application provide a teleoperation method, including the steps of: entering a preset mode in response to a preset trigger condition being met, wherein the preset trigger condition represents a scenario or condition that causes the robot to have a risk of abnormal movement during the teleoperation process; and controlling the robot's movement speed limit based on the input control posture information and the preset teleoperation strategy.

[0006] Optionally, the preset triggering conditions include at least one of the following: determining that it is in the startup initialization phase; determining that it is in the pause recovery phase; determining that the control posture input is abnormal. Optionally, the preset teleoperation strategy includes: determining the deceleration information corresponding to all machine joints of the robot uniformly, or determining the deceleration information corresponding to multiple machine joints of the robot separately; adjusting the deceleration of the angular velocity and / or angular acceleration of each machine joint in the control command mapped to the control posture information according to its corresponding deceleration information; and / or performing smooth interpolation processing on the control command mapped to the control posture information within the teleoperation cycle; and performing motion speed limiting control on the robot according to the control command after deceleration adjustment and / or smooth interpolation processing.

[0007] Optionally, the method for determining the deceleration information includes one of the following: determining the deceleration information uniformly corresponding to all machine joints of the robot or determining the deceleration information corresponding to multiple machine joints of the robot respectively based on fixed value configuration information; using an AI prediction model to calculate the deceleration information uniformly corresponding to all machine joints of the robot or calculating the deceleration information corresponding to multiple machine joints of the robot respectively based on the control posture information and the robot posture information; determining the deceleration information uniformly corresponding to all machine joints of the robot or determining the deceleration information corresponding to multiple machine joints of the robot respectively based on preset rate curve information.

[0008] Optionally, after the step of controlling the robot's motion speed limit based on the input control posture information and the preset teleoperation strategy, the process includes: exiting the preset mode in response to meeting the preset exit conditions; and performing teleoperation on the robot based on the control commands mapped by the input control posture information.

[0009] Optionally, the preset exit conditions include at least one of the following: the continuous timing corresponding to the preset mode is greater than or equal to the preset duration; the attitude difference determined based on the current control attitude information and the current robot attitude information is less than or equal to the difference threshold.

[0010] Optionally, after the step of entering the preset mode in response to the preset triggering condition being met, the process includes: starting the timer; and re-executing the step of starting the timer when the continuous timer is less than the preset duration and the control posture input is determined to be abnormal based on the current control posture information.

[0011] Optionally, the remote operation method of this application further includes at least one of the following: performing activation prompt control for a preset mode; performing work log recording.

[0012] Secondly, embodiments of this application provide a computing device, including: a processor and a memory storing a computer program, wherein when the processor runs the computer program, the steps of the teleoperation method as described in any of the preceding claims are implemented.

[0013] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the teleoperation method as described in any of the preceding claims.

[0014] This application provides a teleoperation method, computing device, and computer-readable storage medium. The teleoperation method includes the following steps: entering a preset mode in response to a preset trigger condition, wherein the preset trigger condition represents a scenario or condition that may cause abnormal robot movements during teleoperation; and controlling the robot's movement speed according to the input control posture information and a preset teleoperation strategy. Through the technical solution of this application, when a scenario or situation that may cause abnormal robot movements during teleoperation is identified, a specific mode corresponding to a slow-motion mechanism can be activated. In this specific mode, the robot's movements mapped by the input control posture information are speed-limited (i.e., a slow-motion mechanism). This ensures that even when the operator's control posture changes significantly or the difference between the operator's control posture and the robot's posture is large, the robot's actual movement speed is limited to a reasonable range, avoiding or reducing the risk of abnormal robot movements. Thus, the technical solution of this application can avoid or reduce the risk of abnormal robot movements during teleoperation in a specific mode through a slow-motion mechanism, thereby improving the safety and stability of teleoperation and enhancing the user experience. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0016] Figure 1 is a flowchart illustrating the teleoperation method provided in an embodiment of this application.

[0017] Figure 2 is a schematic diagram of the framework of the teleoperation system of this application example.

[0018] Figure 3 is a flowchart of the console in the remote operating system of this application example.

[0019] Figure 4 is a schematic diagram of the structure of the computing device provided in an embodiment of this application.

[0020] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0022] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0023] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0024] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0025] It should be noted that step designations such as S11 and S12 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S12 first and then S11, etc., but these should all be within the protection scope of this application.

[0026] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0027] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0028] Referring to Figure 1, a teleoperation method is provided in an embodiment of this application (hereinafter referred to as "this embodiment"). This method can be executed by a computing device provided in this embodiment, which can be implemented in software and / or hardware.

[0029] This embodiment provides a teleoperation method, including the following steps: S11: In response to meeting a preset trigger condition, enter a preset mode, wherein the preset trigger condition represents a scenario or condition that causes the robot to have a risk of abnormal movement during the teleoperation process.

[0030] In one embodiment, the preset triggering conditions include, but are not limited to, at least one of the following: determining that it is in the initialization phase; determining that it is in the pause recovery phase; determining that the control posture input is abnormal.

[0031] Understandably, during the initialization phase, there may be differences between the operator's control posture and the robot's posture. Directly performing synchronous motion control carries the risk of abnormal robot movements (such as the robot suddenly jumping). Therefore, it is advisable to enter a preset mode corresponding to the slow-motion mechanism.

[0032] In one embodiment, the circumstances under which the initialization phase is determined include, but are not limited to: detecting the establishment of a connection with the hardware device and initialization; detecting the completion of calibration work.

[0033] Among them, hardware devices such as robots and motion capture devices are included.

[0034] Among them, motion capture devices can represent any device that captures the operator's control posture and outputs control posture information, such as motion capture suits, VR glasses, exoskeletons, etc.

[0035] This includes detecting the completion of calibration work, such as detecting the completion of posture calibration of the motion capture suit, and detecting the completion of visual calibration of the VR glasses.

[0036] Understandably, during the pause and recovery phase, there may be differences between the operator's control posture and the robot's posture, or the robot may have a response delay. Directly performing synchronous motion control may lead to abnormal robot movements (such as the robot making large movements that do not meet expectations). Therefore, it is possible to enter a preset mode corresponding to the slow-motion mechanism.

[0037] In one embodiment, the circumstances under which a pause / recovery phase is determined include, but are not limited to: detecting that the control command preceding the current recovery command is a pause command; or detecting that a fault signal existed before the current recovery.

[0038] Understandably, determining abnormal control posture input can indicate a situation where operator error or communication failure causes a sudden change in the same joint angle between two detected control postures. When control posture input is abnormal, directly performing synchronous motion control carries the risk of robot malfunction (e.g., the robot performing abrupt movements). Therefore, a preset mode corresponding to a slow-motion mechanism can be entered.

[0039] In one embodiment, the circumstances under which the control posture input is determined to be abnormal include, but are not limited to: when the angle change of any joint exceeds an angle threshold for at least two consecutive frames based on the current control posture information and the previous control posture information, the control posture input can be determined to be abnormal; when the rate of change of the operator's control posture exceeds a preset threshold, the control posture input can be determined to be abnormal.

[0040] S12: Control the robot's movement speed according to the input control posture information and preset teleoperation strategy.

[0041] In one embodiment, a preset teleoperation strategy represents the specific rules or process by which control commands mapped from the control posture information are processed, and the processed control commands are able to control the robot's movements to slowly fit the control pose corresponding to the control posture information. Specifically, controlling the robot's movements to slowly fit the control pose corresponding to the control posture information indicates that the speed at which the robot fits the operator's control posture is slower than the speed at which the operator's control pose changes.

[0042] In one embodiment, the preset teleoperation strategy may include: determining deceleration information uniformly corresponding to all machine joints of the robot, or determining deceleration information corresponding to multiple machine joints of the robot respectively; adjusting the angular velocity and / or angular acceleration of each machine joint in the control command mapped to the control posture information according to its corresponding deceleration information; and / or performing smooth interpolation processing on the control command mapped to the control posture information within the teleoperation cycle; and performing motion speed limiting control on the robot according to the control command after deceleration adjustment and / or smooth interpolation processing.

[0043] It is understood that the speed reduction information can characterize the specific parameters or data set used to adjust the angular velocity and / or angular acceleration of the machine joint corresponding to the control command. Optionally, the speed reduction information may include angular velocity values, angular acceleration values, speed reduction ratios, or speed sequences constructed from multiple velocity values.

[0044] In one embodiment, control commands determined based on the uniform deceleration information corresponding to all machine joints can enable the robot to perform actions according to a uniform deceleration ratio for all machine joints corresponding to the control posture information.

[0045] In one embodiment, control commands based on the deceleration information corresponding to each of the robot's multiple joints can enable all the robot's joints corresponding to the control posture information to perform actions according to the deceleration information (such as speed value, acceleration value, or deceleration ratio) corresponding to each joint.

[0046] In one embodiment, the method for determining the deceleration information includes, but is not limited to, one of the following: determining the deceleration information uniformly corresponding to all machine joints of the robot or determining the deceleration information separately corresponding to multiple machine joints of the robot based on fixed value configuration information; using an AI prediction model to calculate the deceleration information uniformly corresponding to all machine joints of the robot or calculating the deceleration information separately corresponding to multiple machine joints of the robot based on the control posture information and the robot posture information; determining the deceleration information uniformly corresponding to all machine joints of the robot or determining the deceleration information separately corresponding to multiple machine joints of the robot based on preset rate curve information.

[0047] Understandably, the fixed-value configuration information includes a fixed-value reduction ratio uniformly corresponding to all robot joints; or, the fixed-value configuration information includes identification information for multiple robot joints and the fixed-value reduction ratio associated with the identification information of each robot joint (i.e., each robot joint is associated with an independent fixed value as its reduction ratio). These fixed values ​​can be pre-configured through custom operations or configured at the factory using the teleoperation method of this embodiment.

[0048] In one embodiment, the method of determining the deceleration information through fixed value configuration information has the advantages of high stability, ease of implementation and convenient maintenance.

[0049] In one embodiment, the processing principle of the AI ​​prediction model is to determine the posture difference information (i.e., the overall posture difference between the operator's current control posture and the robot's current posture and / or the posture difference of each joint) based on the control posture information and the robot posture information. Then, based on the posture difference information, the deceleration information corresponding to all robot joints is output uniformly or the deceleration information corresponding to multiple robot joints is calculated separately.

[0050] In one embodiment, the control posture information includes, but is not limited to, at least one of the following: joint angle, position, velocity, acceleration, etc., corresponding to each joint. The robot posture information includes, but is not limited to, at least one of the following: joint angle, position, angular velocity, angular acceleration, etc., corresponding to each robot joint (identification information of each robot joint).

[0051] In one embodiment, robot posture information can be obtained through the robot's own sensors (such as encoders, IMUs, etc.).

[0052] In one embodiment, the AI ​​prediction model can be obtained by training a model framework using a pre-built dataset. Each training data point in the dataset includes operator posture information, robot posture information, and uniform deceleration information or deceleration information corresponding to each robot joint. The model framework includes, but is not limited to, independent neural network models or hybrid models comprising multiple models, including, but not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and long short-term memory networks (LSTMs).

[0053] In one embodiment, the deceleration information corresponding to each machine joint in the training data can be labeled by experts according to the speed limit weight of different machine joints (such as wrist > shoulder) so as to achieve layered protection when controlling the speed limit of movements in the future.

[0054] In one embodiment, the technical solution of this embodiment determines the deceleration information through an AI prediction model, which can dynamically adjust the deceleration information according to real-time attitude differences, and has strong adaptability.

[0055] In one embodiment, the deceleration information is determined based on a preset rate curve. The determined deceleration information includes a velocity sequence constructed from multiple velocity values. Control commands processed based on this deceleration information can enable the robot to perform progressively speed-limited actions (e.g., actions gradually increasing in speed or decreasing in speed) or floating speed-limited actions (e.g., actions with non-linear changes), ensuring smoother and safer robot movements during teleoperation. The speed of the speed-limited actions is always lower than or slower than the speed of normal synchronous motion control.

[0056] In one embodiment, the rate curve information includes a data sequence describing the change of velocity over time, used to control the velocity variation of the robot's joints. The data sequence can be linear, nonlinear (e.g., exponential, logarithmic, polynomial, etc.), or a series of discrete velocity values.

[0057] In one embodiment, the rate curve information may include multiple data sequences and identification information of the machine joints associated with each data sequence, thereby determining the velocity sequence corresponding to each machine joint corresponding to the control posture information based on this rate curve information; all or some of the data sequences in the multiple data sequences of the rate curve information are different from each other. In another embodiment, the rate curve information may include a single data sequence, thereby allowing the same data sequence to be used uniformly to determine the velocity sequence corresponding to each machine joint corresponding to the control posture information.

[0058] In one embodiment, the multiple data sequences in the rate curve information can be pre-configured with speed limiting weights assigned to different machine joints (e.g., wrist > shoulder) to achieve layered protection when performing motion speed limiting control later.

[0059] In one embodiment, the data sequence can ensure smooth speed changes, thereby avoiding abrupt changes in robot actions caused by the speed sequence determined based on the data sequence after processing control commands, thus reducing the impact and instability of robot actions.

[0060] For example, the computing device used in the teleoperation method provided in this embodiment can be a state machine, so that the state machine can control the robot to perform speed-limited actions according to the rate curve information.

[0061] For example, in robot teleoperation, the operator's control posture information (such as the movements of the arm, hand, or leg) is mapped to the robot's joints to generate initial control commands for synchronous motion control of the robot. However, when there is a significant difference between the operator's control posture and the robot's posture, or when the robot experiences response delays (such as during startup initialization or abnormal control posture input), directly controlling the robot's synchronous motion with the initial control commands can lead to overly violent and unstable robot movements. To avoid the aforementioned problems, a preset teleoperation strategy may include: adjusting the angular velocity and / or angular acceleration of each joint in the control commands (such as the initial control commands) based on their corresponding deceleration information; and performing motion speed limiting control on the robot based on the deceleration-adjusted control commands. If a uniform reduction amount information is used, the control command after the reduction adjustment can reduce the angular velocity of all machine joints by the same reduction ratio, and / or reduce the angular acceleration of all machine joints by the same reduction ratio. For example, if a uniform reduction ratio (20%~50%, 50% is used as an example) is used to adjust all machine joints corresponding to the control posture information: the content represented by the control posture information includes the operator's arm angular velocity of 30 degrees per second, and the corresponding angular velocity of the robot arm can be 15 degrees per second. If multiple deceleration information corresponding to each machine joint is used, the control command after deceleration adjustment can individually reduce the deceleration information corresponding to the angular velocity of each machine joint corresponding to the control posture information, and / or individually reduce the deceleration information corresponding to the angular acceleration of each machine joint corresponding to the control posture information. For example, if the angular velocity of joint 1 is 30 degrees per second, the deceleration ratio is 50%; if the angular velocity of joint 2 is 25 degrees per second, the deceleration ratio is 40%. In the case of adjusting all machine joints corresponding to the control posture information: the angular velocity of machine joint 1 corresponding to joint 1 is 15 degrees per second, and the angular velocity of machine joint 2 corresponding to joint 2 is 15 degrees per second. Thus, the technical solution of this embodiment, by adjusting the deceleration of the angular velocity and / or angular acceleration in the control command mapping the control posture information, and performing motion speed limiting control on the robot according to the adjusted control command, can significantly improve the motion stability and safety of the robot during teleoperation. This control strategy not only reduces robot jerks and violent movements caused by significant differences between the operator's and robot's postures or abnormal input of the operating posture, but also optimizes the user experience and improves operational efficiency, making it particularly suitable for high-precision and dynamic operation tasks.

[0062] For example, in robot teleoperation, the operator's control posture information (such as the movements of the arm, hand, or leg) is mapped onto the robot's joints to generate initial control commands for synchronous motion control of the robot. However, when there is a significant difference between the operator's control posture and the robot's posture, or when the robot experiences response delays (such as during startup initialization, pause recovery, or abnormal control posture input), directly controlling the robot's synchronous motion with the initial control commands can lead to overly violent and unstable robot movements. To avoid the aforementioned problems, a preset teleoperation strategy may include: first, adjusting the angular velocity and / or angular acceleration of each joint in the control commands mapped from the control posture information according to their corresponding deceleration information; then, performing smooth interpolation on the decelerated control commands within the teleoperation cycle; and finally, using the smooth interpolated control commands to limit the robot's motion speed. The specific scenarios of this preset teleoperation strategy include: (1) using uniform corresponding deceleration information, the control command after deceleration adjustment is then smoothed by interpolation to obtain the final control command and to perform motion speed limit control on the robot; (2) using deceleration information corresponding to multiple machine joints respectively, the control command after deceleration adjustment is then smoothed by interpolation to obtain the final control command and to perform motion speed limit control on the robot.

[0063] Smoothing interpolation refers to inserting smooth transition values ​​between control commands to reduce abrupt changes in control commands and ensure the continuity and stability of robot movements. Optionally, smoothing interpolation can be linear interpolation, nonlinear interpolation, etc. For example, if the control command for deceleration is an angular velocity of 20 degrees per second for a specific robot joint, and an abnormal control posture input is detected, linear interpolation is performed to generate a velocity sequence [5, 10, 20] and the control command is updated accordingly. Based on the updated control command, the robot's movement speed is limited, allowing the robot to gradually adjust its movement speed according to the velocity sequence, smoothly transitioning from 5 degrees per second to 30 degrees per second, ensuring stable movement. The technical solution of this embodiment first reduces the angular velocity and / or angular acceleration in the control command that maps the control posture information, then performs smooth interpolation on the reduced control command, and finally performs motion speed limiting control on the robot based on the smooth interpolation control command. This not only reduces robot jumps and violent movements caused by any triggering conditions (such as large differences between the operator's control posture and the robot's posture, robot response delay, or abnormal control posture input), thus improving the robot's motion stability and safety during teleoperation, but also optimizes the user experience, improves operational efficiency, and is suitable for high-precision and dynamic environment operation tasks.

[0064] For example, in robot teleoperation, the operator's control posture input may be abnormal, such as the operator suddenly making a fast or unreasonable movement. These abnormal control posture inputs may cause the robot to perform unexpected actions or even cause safety problems. To avoid the aforementioned problems, the preset teleoperation strategy may include: when the control posture input is determined to be abnormal, performing smooth interpolation processing on the control command mapped to the control posture information within the teleoperation cycle; and performing motion speed limiting control on the robot based on the smoothed interpolation control command. For example, if the control command mapped to the control posture information is an angular velocity of 30 degrees per second for a specific machine joint, when the control posture input is determined to be abnormal, linear interpolation processing is performed to generate a velocity sequence [10, 20, 30] and the control command is updated accordingly. Based on the updated control command, the robot's motion speed is limited, allowing the robot to gradually adjust its motion speed according to the velocity sequence, smoothly transitioning from 10 degrees per second to 30 degrees per second, ensuring smooth motion. Thus, when the technical solution of this embodiment determines that the control instruction mapping the control posture information is abnormal, it performs smooth interpolation processing on the control instruction and performs motion speed limiting control on the robot according to the processed control instruction. This can reduce the abrupt changes between the current control instruction and the previous control instruction, ensure the smoothness and continuity of the robot's movements, and significantly improve the safety and reliability of the operation. In addition, the technical solution of this embodiment also optimizes the user experience, improves the operation efficiency, and is suitable for operation tasks in high-precision and dynamic environments.

[0065] In one embodiment, after the step of controlling the robot's motion speed limit based on the input control posture information and the preset teleoperation strategy, the method includes: exiting the preset mode in response to meeting the preset exit conditions; and performing teleoperation on the robot based on the control commands mapped by the input control posture information.

[0066] Understandably, the preset exit condition can characterize the scenario or condition used to determine when to exit the preset mode corresponding to the slow-motion mechanism and return to the normal teleoperation mode.

[0067] Understandably, by teleoperating the robot based on the control commands (such as the initial control commands) mapped from the input maneuvering posture information, synchronous motion control of the robot can be achieved.

[0068] Thus, the technical solution of this embodiment, by defining reasonable preset exit conditions and exiting the preset mode when the conditions are met, and restoring to the normal remote operation mode, can ensure that the robot's actions during remote operation are safer, more efficient, and in line with the operator's intentions. This not only improves the safety and efficiency of the operation but also optimizes the user experience.

[0069] In one embodiment, the preset exit condition includes, but is not limited to, at least one of the following: the continuous timing corresponding to the preset mode is greater than or equal to the preset duration; the attitude difference determined based on the current control attitude information and the current robot attitude information is less than or equal to the difference threshold.

[0070] Understandably, the preset duration can be configured based on actual needs or experimental test results.

[0071] For example, if the continuous timing corresponding to the preset mode is greater than or equal to 5 seconds, the preset mode will be exited. Subsequently, when new input control posture information is received, the robot will be remotely operated according to the control instructions mapped by the input control posture information.

[0072] For example, if the difference in posture determined by the current control posture information and the current robot posture information is less than or equal to 2 degrees, the preset mode can be exited. Subsequently, when new input control posture information is received, the robot can be remotely operated according to the control commands mapped by the input control posture information.

[0073] For example, if the continuous timing corresponding to the preset mode is less than 5 seconds, and the difference in posture determined by the current control posture information and the current robot posture information is less than or equal to 2 degrees, then the preset mode can be exited in advance.

[0074] In one embodiment, after the step of entering a preset mode in response to meeting a preset trigger condition, the process includes: starting a timer; and re-executing the step of starting the timer when the continuous timer is less than a preset duration and the control posture input is determined to be abnormal based on the current control posture information.

[0075] Thus, in the preset mode, the technical solution of this embodiment can continuously monitor the operator's posture input and analyze in real time whether the posture data is abnormal (such as excessively high rate of change of posture, posture amplitude exceeding the normal range, discontinuous posture data, etc.). When it is determined that the control posture input is abnormal, the duration of the preset mode can be extended to ensure that the robot is always in a safe and stable control state.

[0076] In one embodiment, the teleoperation method of this application further includes, but is not limited to, at least one of the following: performing activation prompt control for a preset mode; performing work log recording.

[0077] Understandably, executing the activation prompt control for the preset mode can indicate that when the robot enters the preset mode, it provides the operator with clear prompt information to inform the operator that the robot has entered a special control mode.

[0078] In one embodiment, the activation prompt control includes, but is not limited to, at least one of the following: controlling a visual display device to display a prompt message indicating that a preset mode has been activated; controlling an audio device to play a prompt message indicating that a preset mode has been activated.

[0079] In one embodiment, the visual display device includes, but is not limited to, VR / AR devices, displays, indicator lights, etc. For example, when an operator uses a VR or AR device, a prompt message may pop up on the device's display interface, such as "Entered preset mode" or "Entered slow motion mode." Alternatively, a prompt message (such as text prompts, icons, or flashing warning lights) may be displayed on the state machine's control panel or display screen.

[0080] Understandably, performing work log recording can characterize the automatic recording of key information during robot teleoperation to form log content. Key information includes operator posture input (manipulation posture information), robot posture feedback (robot posture information), control commands, abnormal events, etc. The recorded key information can be used for subsequent system review, fault analysis, and performance optimization.

[0081] For example, the log content includes manipulation posture information, robot posture information, control commands, abnormal events, and timestamps. Among them, the manipulation posture information includes the operator's joints and their corresponding joint angles, angular velocities, angular accelerations, etc.

[0082] The teleoperation method provided in this embodiment includes the following steps: S11: In response to meeting a preset trigger condition, enter a preset mode, wherein the preset trigger condition represents a scenario or condition that may cause abnormal robot movement during teleoperation; S12: Perform speed limiting control on the robot's movement based on the input control posture information and the preset teleoperation strategy. Through the technical solution of this embodiment, when a scenario or situation that may cause abnormal robot movement during teleoperation is identified, a specific mode corresponding to a slow-motion mechanism can be activated. In this specific mode, the robot's movement mapped by the input control posture information is speed-limited (i.e., a slow-motion mechanism). This ensures that even when the operator's control posture changes significantly or the difference between the operator's control posture and the robot's posture is large, the robot's actual movement speed is limited to a reasonable range, avoiding or reducing the risk of abnormal robot movement. Thus, the technical solution of this embodiment can avoid or reduce the risk of abnormal robot movement during teleoperation in a specific mode through a slow-motion mechanism, thereby improving the safety and stability of teleoperation and enhancing the user experience.

[0083] Based on the same inventive concept as the foregoing embodiments, the foregoing embodiments will be illustrated by a specific example below. In this example, a remote operating system will be used as an example.

[0084] In some teleoperation implementations, the operator (also known as a telemanipulator) uses motion capture equipment to map the operator's arm or upper limb control posture to the remote robot arm, achieving synchronized motion control of the robot's arm. However, in the following scenarios, some teleoperation implementations are prone to causing abnormal robot movements, posing safety hazards: 1. Startup initialization phase: The robot and operator have different initial postures, and direct synchronized motion control can cause the robot to jump abruptly; 2. Pause and resume phase: When the operator pauses and then resumes control, differences in robot and operator postures or robot response delays can cause unexpected large movements; 3. Abnormal input phase (i.e., determining abnormal control posture input): Operator misoperation or communication abnormalities can cause a sudden change in the angle of a joint, resulting in the robot performing violent movements.

[0085] Referring to Figure 2, to address the aforementioned shortcomings of some implemented teleoperation methods, this example provides a teleoperation system including a console B102, a robot B103, and a motion capture device B101. When the console B102 is operational, it can remotely operate the robot B103 using the teleoperation method described in the above embodiment. Teleoperation represents the process by which an operator remotely acquires their motion signals (i.e., the aforementioned control posture information) through the motion capture device B101, converts them into control commands via a mapping algorithm, and transmits them to the robot B103, thereby performing real-time posture control (i.e., synchronous motion control) on the robot B103.

[0086] Referring to Figure 3, the workflow of console B102 in the teleoperation system of this example includes: 1.1 Performing status detection; 1.2 Determining whether the preset trigger conditions are met based on the status detection results; if yes, proceeding to step 1.3; if no, performing teleoperation in normal teleoperation mode; 1.3 Entering slow motion mode (i.e., the aforementioned preset mode) and setting the timer = 0 to start timing; 1.4 Performing motion speed limiting control; 1.5 Determining whether the posture is consistent with the current control posture information and the current robot posture information; if yes, proceeding to step 1.6; if no, proceeding to step 1.7; 1.6 Exiting slow motion mode to gradually restore to normal teleoperation mode for synchronous motion control; 1.7 Determining whether the continuous timing is greater than or equal to 5 seconds; if yes, executing step 1.6; if no, executing step 1.8; 1.8 Determining whether there is an abnormal control posture input based on the current control posture information; if yes, returning to set the timer = 0 to start timing; if no, continuing to execute step 1.4.

[0087] The state detection result represents any of the following situations, and is then determined to meet the preset triggering conditions: 1. Start-up initialization phase; 2. Pause-recovery phase; 3. Abnormal input phase.

[0088] Among them, the situation of judging abnormal input stage is, for example, based on the current control posture information and the previous control posture information, it is identified that the angle change of any joint is greater than or equal to 30 degrees for at least two consecutive frames.

[0089] The preset teleoperation strategy for motion speed limiting control includes: adjusting the angular velocity and / or angular acceleration of each machine joint in the control command mapped to the control posture information according to a reduction ratio of 20% to 50% of the speed of the corresponding joint of the operator; and / or performing smooth interpolation processing on the control command mapped to the control posture information (such as the initial control command or the control command after speed reduction adjustment) within the teleoperation cycle; performing motion speed limiting control on robot B103 according to the control command after speed reduction adjustment and / or smooth interpolation processing; and executing activation prompt control for preset modes to provide visual or UI prompts to notify the operator to enter slow motion mode.

[0090] Understandably, the mechanism of step 1.6 exiting slow motion mode to gradually restore to normal teleoperation mode for synchronous motion control can be summarized as follows: (1) the slow motion mode continuously counts for more than 5 seconds; (2) if the difference in posture determined by the current control posture information and the current robot B103 posture information is less than or equal to 2 degrees (i.e. the posture is judged to be consistent), then step 1.6 can be executed in advance before the continuous count reaches 5 seconds.

[0091] Understandably, step 1.8 determines whether there is an abnormal control posture input based on the current control posture information; if so, it returns to set the timer to 0 to start the timing; if not, it continues to execute step 1.4. In this way, the technical solution of this example can restart the timing when the state detection result re-acquired in slow motion mode represents a sudden input (i.e., the abnormal input stage), so as to extend the start time of slow motion mode and achieve safety protection.

[0092] In the remote operating system of this embodiment, console B102 can also (b) provide a log recording and exception prompt interface to facilitate system review.

[0093] Optionally, in addition to using a fixed deceleration ratio (one of the aforementioned deceleration ratios between 20% and 50%) to adjust control commands, the remote operating system console B102 in this example can also utilize an AI prediction model to calculate the deceleration ratio uniformly corresponding to all joints of the robot B103 based on the control posture information and the robot B103 posture information, or calculate the deceleration ratio corresponding to multiple joints of the robot B103 respectively. This enables adaptive adjustment of the deceleration ratio based on posture differences, thereby adjusting the control commands. Furthermore, the console B102 can also serve as a safety state machine incorporating slow-motion mode triggering and exit strategies. The console B102 can determine the speed sequence corresponding to each joint based on the speed curve information and then adjust the control commands according to the speed sequence, thereby enabling the joints of the robot B103 to move according to the speed curve (for details, please refer to the relevant content in the aforementioned embodiments, which will not be repeated here).

[0094] Among them, the AI ​​prediction model determines the reduction ratio corresponding to each of the multiple robot joints of robot B103 and the speed sequence corresponding to each robot joint based on the rate curve information. It can independently configure the reduction ratio or speed sequence for each robot joint according to the speed limit weight corresponding to different robot joints (such as the speed limit weight of the wrist is greater than that of the shoulder) to achieve layered protection.

[0095] The technical solution provided in this example can automatically enter slow-motion mode when the teleoperation system determines that the state detection result meets the preset trigger conditions (i.e., it meets key risk moments such as start-up, pause-and-resume, and angle change). This allows for proportional scaling of the angular velocity of the robot B103's joints, limiting the maximum angular acceleration and achieving smooth start-up and safe transition. When consistent posture is detected or the continuous timing of slow-motion mode exceeds a threshold, the system automatically exits slow-motion mode. The entire control logic of the teleoperation system in this example can be summarized in three stages: 1. Slow-motion trigger judgment stage; 2. Slow-motion execution stage; 3. Exit condition detection stage. Thus, the teleoperation system of this example can automatically activate slow-motion mode in three scenarios: startup, pause and resumption, and sudden joint angle changes. By scaling the angular velocity and angular acceleration of the robot B103's joints, motion smoothing is achieved. Through posture error detection and continuous timing, the slow-motion mode is automatically exited according to a consistent exit logic. Therefore, the aforementioned technical solution of this example can avoid safety accidents such as violent initial movements and collisions of the robot B103, and can provide a buffer period for the operator's reaction. As a result, the teleoperation system of this example can dynamically respond to different operating scenarios, improving safety, stability, and user experience.

[0096] Based on the same inventive concept as the foregoing embodiments, this application provides a computing device, as shown in FIG4. The device includes a processor 310 and a memory 311 storing a computer program. The processor 310 shown in FIG4 does not refer to a single processor 310, but only to the positional relationship of the processor 310 relative to other devices. In practical applications, there can be one or more processors 310. Similarly, the memory 311 shown in FIG4 has the same meaning, that is, it only refers to the positional relationship of the memory 311 relative to other devices. In practical applications, there can be one or more memory 311. When the processor 310 runs the computer program, the teleoperation method applied to the above-mentioned device is implemented.

[0097] The device may also include at least one network interface 312. The various components of the device are coupled together via a bus system 313. It is understood that the bus system 313 is used to enable communication between these components. In addition to a data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 313 in Figure 4.

[0098] The memory 311 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 311 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0099] The memory 311 in this embodiment is used to store various types of data to support the operation of the device. Examples of this data include any computer programs used to operate on the device, such as operating systems and applications. The operating system includes various system programs, such as framework layers, core library layers, and driver layers, used to implement various basic business functions and handle hardware-based tasks. Applications can include various applications, such as media players and browsers, used to implement various application services. Here, the program implementing the method of this embodiment can be included in the application.

[0100] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program stored in the computer-readable storage medium is executed by a processor, it implements the above-described teleoperation method. The specific steps implemented when the computer program is executed by the processor are described in the embodiment shown in Figure 1, and will not be repeated here.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A teleoperation method, characterized in that, The steps include: entering a preset mode in response to a preset trigger condition being met, wherein the preset trigger condition represents a scenario or condition that could lead to abnormal robot movements during teleoperation; and controlling the robot's movement speed limit based on the input control posture information and the preset teleoperation strategy.

2. The teleoperation method according to claim 1, characterized in that, The preset triggering conditions include at least one of the following: determining that it is in the initialization phase; determining that it is in the pause and recovery phase; determining that the control posture input is abnormal.

3. The teleoperation method according to claim 1, characterized in that, The preset teleoperation strategy includes: determining deceleration information uniformly corresponding to all machine joints of the robot, or determining deceleration information corresponding to multiple machine joints of the robot respectively; adjusting the deceleration of the angular velocity and / or angular acceleration of each machine joint in the control command of the mapped control posture information according to its corresponding deceleration information; and / or performing smooth interpolation processing on the control command of the mapped control posture information within the teleoperation cycle; and performing motion speed limiting control on the robot according to the deceleration adjustment and / or the smooth interpolation processed control command.

4. The teleoperation method according to claim 3, characterized in that, The method for determining the deceleration information includes one of the following: determining the deceleration information uniformly corresponding to all machine joints of the robot or determining the deceleration information corresponding to multiple machine joints of the robot respectively based on fixed value configuration information; calculating the deceleration information uniformly corresponding to all machine joints of the robot or calculating the deceleration information corresponding to multiple machine joints of the robot respectively based on the control posture information and robot posture information using an AI prediction model; or determining the deceleration information uniformly corresponding to all machine joints of the robot or determining the deceleration information corresponding to multiple machine joints of the robot respectively based on preset rate curve information.

5. The teleoperation method according to any one of claims 1 to 4, characterized in that, After performing motion speed limiting control on the robot based on the input control posture information and the preset teleoperation strategy, the process includes: exiting the preset mode in response to meeting the preset exit conditions; and performing teleoperation on the robot based on the control commands mapped by the input control posture information.

6. The teleoperation method according to claim 5, characterized in that, The preset exit condition includes at least one of the following: the continuous timer corresponding to the preset mode is greater than or equal to the preset duration; The attitude difference determined based on the current control attitude information and the current robot attitude information is less than or equal to the difference threshold.

7. The teleoperation method according to claim 6, characterized in that, After the step of entering a preset mode in response to meeting the preset trigger conditions, the process includes: starting a timer; and re-executing the step of starting the timer when the continuous timer is less than the preset duration and the control posture input is determined to be abnormal based on the current control posture information.

8. The teleoperation method according to claim 1, characterized in that, It also includes at least one of the following: performing activation prompt control for the preset mode; performing work log recording.

9. A computing device, characterized in that, include: A processor and a memory storing a computer program, wherein, when the processor executes the computer program, the steps of the teleoperation method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the teleoperation method according to any one of claims 1 to 8.

Images