Robotic teleoperation control method and system, device, medium, product

Abstract

The embodiment of the application provides a robot teleoperation control method and system, equipment, medium and product, the method comprises the following steps: obtaining link state information of a teleoperation link; in the case that the link state information represents that the teleoperation link meets a safety intervention condition, determining a safety intervention strategy corresponding to the teleoperation link based on the link state information; and performing action control on the robot based on the safety intervention strategy. The embodiment of the application improves the problem that link abnormality may cause action discontinuity, control data expiration or robot motion instability in the robot teleoperation process, thereby improving the continuity, stability and safety of robot teleoperation control.

Description

Technical Field

[0001] This application relates to the field of robot teleoperation technology, and in particular to robot teleoperation control methods, systems, devices, media, and products. Background Technology

[0002] With the development of robotics technology, robots are increasingly being applied in scenarios such as industrial manufacturing, emergency rescue, medical rehabilitation, inspection and maintenance, home services, and operations in complex environments. In some scenarios, relying solely on autonomous planning and control of robots may be insufficient to meet the comprehensive requirements of real-time performance, flexibility, and safety. Robot teleoperation can convert teleoperation data generated by the operator into control data that the robot can execute; however, the continuity, stability, and safety of current robot teleoperation control are still insufficient to meet the needs of practical applications.

[0003] Based on this, embodiments of this application provide robot teleoperation control methods, systems, devices, media, and products to improve related technologies. Summary of the Invention

[0004] The purpose of this application is to provide a robot teleoperation control method, system, device, medium, and product to improve the continuity, stability, and safety of robot teleoperation control.

[0005] In a first aspect, embodiments of this application provide a robot teleoperation control method, the method comprising: acquiring link status information of a teleoperation link, wherein the teleoperation link is a processing link from receiving teleoperation data to outputting robot control data; determining a safety intervention strategy corresponding to the teleoperation link based on the link status information when the link status information indicates that the teleoperation link meets safety intervention conditions; and performing motion control on the robot based on the safety intervention strategy.

[0006] Secondly, embodiments of this application provide a robot teleoperation control system, the system comprising: a status acquisition module, configured to acquire link status information of a teleoperation link, wherein the teleoperation link is a processing link from receiving teleoperation data to outputting robot control data; a status analysis module, configured to determine a safety intervention strategy corresponding to the teleoperation link based on the link status information, provided that the link status information indicates that the teleoperation link meets safety intervention conditions; and an action control module, configured to perform action control on the robot based on the safety intervention strategy.

[0007] Thirdly, embodiments of this application provide a computer device, the computer device including a processor and a memory, the memory storing a computer program, the computer program being executed by the processor to implement the steps of any of the above methods.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0009] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the steps of any of the above methods.

[0010] This application provides a robot teleoperation control method, system, device, medium, and product. Addressing the issues of discontinuous actions, outdated control data, or robot motion instability caused by link anomalies during robot teleoperation, it employs a mechanism that determines safety intervention strategies based on link status information and controls robot actions according to these strategies, thereby improving the continuity, stability, and safety of robot teleoperation control. Attached Figure Description

[0011] The embodiments of this application are further described below with reference to the accompanying drawings and specific implementation details.

[0012] Figure 1 This is a flowchart illustrating a robot teleoperation control method provided in an embodiment of this application.

[0013] Figure 2 This is a structural block diagram of a robot teleoperation control system provided in an embodiment of this application.

[0014] Figure 3 This is a structural block diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this application.

[0016] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0017] With the development of robotics technology, robots are increasingly being applied in scenarios such as industrial manufacturing, emergency rescue, medical rehabilitation, inspection and maintenance, home services, and operations in complex environments. In some scenarios, robots need to perform delicate operations, move across areas, engage in human-robot collaborative interaction, or operate in hazardous environments. Relying solely on autonomous planning and control may be insufficient to meet the comprehensive requirements of real-time performance, flexibility, and safety. Therefore, robot teleoperation technology has attracted widespread attention.

[0018] Robot teleoperation can collect the operator's movements, postures, positions, hand operations, foot movements, or other control intentions through the operator's terminal, and convert the teleoperation data generated by the terminal into control data that the robot can execute. For robots with many degrees of freedom, such as humanoid robots, dual-arm robots, and mobile manipulation robots, teleoperation data often needs to go through multiple processing stages in the teleoperation link before it can finally be applied to the robot's joints, base, or end effector.

[0019] In actual teleoperation processes, the teleoperation link typically needs to meet high real-time requirements. For example, the operator can continuously generate teleoperation data at a fixed frequency, and the robot control side also needs to continuously output robot control data according to the control cycle. If any processing link in the teleoperation link malfunctions, it may affect the continuity and stability of the robot's movements.

[0020] See Figure 1 , Figure 1 This is a flowchart illustrating a robot teleoperation control method provided in an embodiment of this application.

[0021] To address the issues of discontinuous actions, outdated control data, or robot motion instability caused by link anomalies during robot teleoperation, this application provides a robot teleoperation control method, which includes steps S101 to S103.

[0022] Step S101: Obtain the link status information of the teleoperation link, wherein the teleoperation link is the processing link from receiving teleoperation data to outputting robot control data.

[0023] Step S102: If the link status information indicates that the teleoperation link meets the security intervention conditions, determine the security intervention strategy corresponding to the teleoperation link based on the link status information.

[0024] Step S103: Based on the safety intervention strategy, perform motion control on the robot.

[0025] Robot teleoperation control methods can refer to methods that control robot actions based on teleoperation data, and, when the teleoperation link meets the safety intervention conditions, intervene in the robot action control process according to the corresponding safety intervention strategy.

[0026] In some embodiments, teleoperation data can be generated by an operator terminal for the robot. The operator terminal can be used to generate teleoperation data. The operator terminal may include a motion capture device, a wearable posture acquisition device, a handle, a master robotic arm, a virtual reality interaction device, or a host control device.

[0027] Teleoperation data refers to data generated by the operator in response to the robot, used to express remote operation intentions. Teleoperation data can serve as input to the teleoperation link, used to subsequently generate robot control data or execute motion control.

[0028] A teleoperation link can refer to the processing link between receiving teleoperation data and outputting robot control data.

[0029] Link status information can be used to characterize the operational status of a teleoperation link. This information can be used to determine whether a teleoperation link meets the conditions for secure intervention and serves as the basis for determining secure intervention strategies.

[0030] Safety intervention conditions refer to the conditions that trigger the robot's teleoperation control method to perform safety interventions. Safety intervention conditions are used to identify abnormal states in the teleoperation link that may affect the stability and safety of the robot's movements.

[0031] Safety intervention strategies can be action control strategies corresponding to link status information. These strategies can be applied to the robot's control data generation process or its control data output process, enabling the robot to execute action control corresponding to the safety intervention strategy when the teleoperation link malfunctions.

[0032] Robot control data can refer to the data used to control the robot's actions.

[0033] Motion control refers to controlling a robot to perform teleoperated actions or enter a safe holding posture based on a safety intervention strategy corresponding to the teleoperation link. Motion control emphasizes that the safety intervention strategy ultimately affects the robot's motion. The teleoperated action corresponding to the safety intervention strategy can be the teleoperated action performed by the robot under the action of the safety intervention strategy. This teleoperated action can reflect the degree of safety intervention, and is not limited to completely stopping the robot's movement.

[0034] Safe attitude maintenance refers to a controlled, stable state that a robot enters when a teleoperation link anomaly is severe or prolonged. Safe attitude maintenance is used to reduce the risk of robot instability caused by abnormal teleoperation data or outdated robot control data.

[0035] In this embodiment, the robot teleoperation control method acquires the link status information of the teleoperation link and determines whether the teleoperation link meets the safety intervention conditions based on the link status information. When the teleoperation link meets the safety intervention conditions, the robot teleoperation control method determines the corresponding safety intervention strategy based on the link status information. Finally, based on the safety intervention strategy, the robot performs motion control, causing the robot to execute the teleoperation action corresponding to the safety intervention strategy or enter a safe holding posture.

[0036] This embodiment acquires the link status information of the teleoperation link to identify abnormal states in different processing stages, such as teleoperation data reception, motion processing, and control data output. By determining a safety intervention strategy when the link status information indicates that the teleoperation link meets the safety intervention conditions, and applying the safety intervention strategy to the robot motion control process, the impact of teleoperation data interruption, motion solution timeout, motion state jump, or control data expiration on robot motion stability can be reduced. This embodiment addresses the problem that link anomalies during robot teleoperation may lead to discontinuous motion, expired control data, or robot motion instability. It adopts a mechanism that determines a safety intervention strategy based on link status information and controls the robot's motion according to the safety intervention strategy, thereby improving the continuity, stability, and safety of robot teleoperation control.

[0037] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each control cycle lasting 10 milliseconds. The operator continuously generates teleoperation data from the first to the 80th control cycle. During this period, if the acquired link status information indicates that the teleoperation link does not meet the safety intervention conditions, the robot executes the corresponding teleoperation action based on the teleoperation data.

[0038] In the 81st control cycle, the link status information indicates a short-term interruption of teleoperation data, with an interruption duration of 10 milliseconds. Based on the robot teleoperation control method, a corresponding safety intervention strategy is determined to control the robot to maintain the previous effective teleoperation action, such as maintaining the arm posture and robot base posture corresponding to the previous control cycle.

[0039] In the 95th control cycle, the link status information indicates that the teleoperation data interruption has lasted for 150 milliseconds. Based on the robot teleoperation control method, a corresponding safety intervention strategy is determined, and the robot is controlled to enter a safe holding posture, such as maintaining its current position and limiting its speed output. This example demonstrates that the robot teleoperation control method can determine different safety intervention strategies based on link status information and execute different types of motion control on the robot.

[0040] In some embodiments, the teleoperation link may include at least one of the following processing stages: teleoperation data receiving stage, data conversion stage, motion solving stage, motion state protection stage, control data publishing stage, and marker state management stage.

[0041] Different processing stages correspond to different types of security risks. For example, the teleoperation data receiving stage can be associated with data interruption risk, the motion solving stage can be associated with solution timeout risk, the motion state protection stage can be associated with state transition risk, the control data publishing stage can be associated with the risk of continuing to output expired data, and the tagging state management stage can be associated with the risk of teleoperation tagging being unavailable.

[0042] The teleoperation data receiving stage refers to the processing stage that receives teleoperation data generated by the operator and records the input status of the teleoperation data. This stage can record whether a teleoperation data frame has arrived, the input time of the teleoperation data frame, the most recent valid teleoperation data frame, and the continuity of teleoperation data frames. If no new teleoperation data is received within the target control cycle, the stage status information corresponding to the teleoperation data receiving stage can indicate an abnormal teleoperation input.

[0043] The data conversion stage refers to the processing stage that converts teleoperation data generated by the operator into target motion information adapted to the robot's structure and motion capabilities. Since the operator and the robot may differ in size, joint structure, number of degrees of freedom, and range of motion, the data conversion stage can map the hand pose, foot pose, torso posture, or position information of teleoperation markers from the operator into the robot's end-effector target pose, robot base target pose, robot joint target constraints, or robot motion target information.

[0044] The motion solution stage refers to the processing stage that calculates the robot's motion state or control objectives based on the target motion information obtained from the data conversion stage. The motion solution stage can include inverse kinematics solving, whole-body motion solving, multi-constraint optimization solving, or joint target solving. The status of the motion solution stage can reflect whether the solution is complete, whether the solution timed out, or whether the solution has not converged, providing a basis for determining whether safety intervention strategies such as result reuse are necessary.

[0045] Motion state protection refers to the process of safely processing the motion state variables corresponding to motion solution results, teleoperation data, or robot control data. Motion state protection does not emphasize generating motion state variables from scratch, but rather emphasizes processing the already obtained motion state variables through filtering, transition detection, speed limiting, speed freezing, or attitude protection.

[0046] The control data publishing stage refers to the processing stage where robot control data is output to the robot. Based on the robot control data acquisition time and set validity period, this stage determines whether to continue outputting robot control data, whether to update the output time of the robot control data, and whether to restrict the continued output of expired robot control data. This stage is used to reduce the risk of the robot executing expired control data for extended periods.

[0047] The marker status management stage refers to the processing stage that manages the availability status of teleoperation markers in teleoperation data. Teleoperation markers may include markers used to represent the operator's pelvis, torso, hands, feet, elbows, knees, or other body parts. The marker status management stage can determine whether teleoperation markers are available, whether the number of available markers meets the motion control requirements, whether key markers (e.g., the first marker) are missing, and provide a marker status basis for subsequent data transformation and motion solving.

[0048] In some embodiments, the link status information may include link status information corresponding to at least one processing link, and the link status information may include status type information and status duration information; wherein, the status type information may include at least one of teleoperation input status, data conversion status, motion solving status, motion status protection status, control data publishing status, and marked availability status; the status duration information includes at least one of the following: the duration for which the corresponding status type information satisfies the safety intervention conditions, the number of continuous control cycles, and the number of consecutive occurrences.

[0049] Status type information is used to characterize the status category of a processing step. For example, teleoperation input status can indicate whether teleoperation data has been interrupted, motion solution status can indicate whether motion solution has timed out, and control data release status can indicate whether teleoperation data has exceeded the set validity period.

[0050] State persistence information can be used to characterize the duration under which corresponding state type information meets the conditions for safety intervention. For example, a 50-millisecond interruption of teleoperation data, two consecutive timeouts in motion solution, or three consecutive control cycles of motion state variable jumps can all be considered as state persistence information. State persistence information can be used to distinguish between intermittent and persistent anomalies, thereby improving the accuracy of safety intervention strategy determination.

[0051] Teleoperation input status is a type of status information that can be used to determine the input status of the teleoperation data receiving stage. Teleoperation input status can include states such as new teleoperation data received, no new teleoperation data received, discontinuous teleoperation data frames, and invalid teleoperation data.

[0052] Data conversion status is a type of status information that can be used to determine the processing status of the data conversion process. Data conversion status can include states such as data conversion complete, data conversion processing time exceeding the set processing time, data conversion result unavailable, and insufficient data conversion input.

[0053] Motion solution status is a type of status information that can be used to determine the processing status of the motion solution process. Motion solution status can include states such as motion solution completed, motion solution timeout, motion solution not converged, and motion solution result unavailable.

[0054] Motion state protection status is a type of status information that can be used to determine the safe processing status of motion state protection mechanisms. Motion state protection status can include states such as no change in motion state quantity, a change in motion state quantity, a velocity change exceeding a threshold, and meeting velocity freeze conditions.

[0055] Control data release status is a type of status information that can be used to determine the release conditions or risks in the control data release process. Control data release status can include states such as robot control data is publishable, robot control data requires update time, teleoperation data has exceeded a set validity period, and robot control data output has stopped. Control data release status can be used to control whether robot control data continues to be output to the robot.

[0056] The availability status of a tag is a type of status information that can be used to determine whether a teleoperation tag is available for teleoperation control. Tag availability status can include all teleoperation tags available, some teleoperation tags unavailable, critical teleoperation tags (e.g., the first tag) unavailable, and auxiliary teleoperation tags (e.g., the second tag) unavailable.

[0057] Duration can represent the length of time elapsed from the occurrence of an anomaly to the current moment, such as 100 milliseconds. The number of continuous control cycles can represent the number of control cycles that an anomaly continuously covers, such as 10 consecutive control cycles. The number of consecutive occurrences can represent the number of times an anomaly is detected consecutively, such as 3 consecutive motion solution timeouts. These three parameters can be used individually or in combination to adapt to different control frequencies and different anomaly detection methods.

[0058] This embodiment divides the teleoperation link into multiple processing stages, enabling the identification of input anomalies, data conversion anomalies, motion solving anomalies, motion state protection anomalies, control data publishing anomalies, and availability marking anomalies during the teleoperation process at each stage. By including the stage status information corresponding to at least one processing stage in the link status information, a correspondence can be established between anomaly states and specific processing stages, reducing the risk of inaccurate safety intervention strategy selection due to confusion between different anomaly sources. By including state type information and state duration information in the stage status information, both anomaly type and anomaly duration can be determined simultaneously, facilitating the differentiation between short-term and persistent anomalies. By setting the state duration information to at least one of duration, number of continuous control cycles, and number of consecutive occurrences, it can adapt to different control frequencies and different anomaly detection methods. This embodiment addresses the problem that different anomalies may occur in multiple processing stages during robot teleoperation, and the varying degrees of anomaly duration lead to different safety handling requirements. It adopts a mechanism based on processing stages, state type information, and state duration information to describe the link status, thereby providing a more accurate state basis for subsequent layered safety intervention.

[0059] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each control cycle lasting 10 milliseconds. If the teleoperation data receiving stage does not receive a new teleoperation data frame between the 21st and 23rd control cycles, the corresponding status type information for that stage is "No new teleoperation data received," and the status duration is either 30 milliseconds, 3 control cycles, or 3 consecutive occurrences. If the motion solving stage's processing time in the 40th control cycle is 18 milliseconds, exceeding the set processing time by 10 milliseconds, the corresponding status type information for that stage is "Motion solving timeout," and the status duration is either 1 control cycle or 1 consecutive occurrence. If the control data publishing stage detects that the most recent valid teleoperation data is more than 300 milliseconds away from the current time, the corresponding status type information for that stage is "Teleoperation data exceeds the set validity period," and the status duration is 300 milliseconds. These link status information can provide a basis for subsequently determining safety intervention strategies.

[0060] In some embodiments, the security intervention strategy may include at least one of input retention processing, result reuse processing, motion state protection processing, release control processing, and constraint adjustment processing.

[0061] Therefore, different handling methods can be adopted for anomalies in different links of the teleoperation link. Input retention processing can reduce control interruptions caused by short-term teleoperation data interruptions. Result reuse processing can reduce the risk of discontinuous actions caused by data conversion or motion solution timeouts. Motion state protection processing can limit the impact of abrupt changes in motion state variables or sudden velocity changes on the stability of robot actions. Control release processing can reduce the safety risks caused by the continued output of expired robot control data. Constraint adjustment processing can maintain usable motion solution capabilities when some teleoperation markers are unavailable. This embodiment addresses the issue that anomalies in different processing links during robot teleoperation have different impact ranges and safety handling requirements, and adopts a mechanism of multiple safety intervention strategies to improve the continuity, stability, and safety of robot teleoperation control.

[0062] The following section provides a detailed explanation of several security intervention strategies.

[0063] In some embodiments, the teleoperation data may include multiple teleoperation data frames generated in a time sequence, and the link status information indicating that the teleoperation link meets the security intervention conditions may include: the link status information corresponding to the teleoperation data receiving link indicating that no teleoperation data frames were received within the target control period, and the number of input hold counts did not reach the hold limit.

[0064] The security intervention strategy may include input retention processing.

[0065] The motion control of the robot based on the safety intervention strategy may include: performing input hold processing based on the previous valid teleoperation data frame, generating target robot control data corresponding to the current control processing, and performing motion control of the robot based on the target robot control data.

[0066] A teleoperation data frame refers to a set of teleoperation data generated by the operator within a sampling time or a control cycle. Multiple teleoperation data frames arranged chronologically can reflect the operator's continuous control intentions towards the robot. By organizing the teleoperation data into multiple time-series generated teleoperation data frames, it is possible to determine whether the teleoperation data is continuously input based on the robot's teleoperation control method.

[0067] The target control cycle refers to the time period during which the robot teleoperation control method performs one control process. The target control cycle can be set according to the robot's control frequency; for example, if the control frequency is 100 Hz, a target control cycle can be 10 milliseconds. Within the target control cycle, the robot teleoperation control method can determine whether a new teleoperation data frame has been received and, based on the determination result, decide whether to perform input hold processing.

[0068] In this embodiment, the stage status information corresponding to the teleoperation data receiving stage can characterize whether a new teleoperation data frame has been received within the target control cycle, and whether the number of hold operations has reached the hold limit. Through this stage status information, short-term data interruptions in the teleoperation data receiving stage can be converted into a state basis that can be recognized by security intervention strategies.

[0069] In this embodiment, safety intervention conditions can be used to identify short-term teleoperation data interruption scenarios, so that the robot teleoperation control method does not have to immediately enter a safe holding posture when a single teleoperation data frame has not arrived.

[0070] The input hold count refers to the number of times the robot teleoperation control method has performed input hold processing using the previous valid teleoperation data frame within a control cycle in which no new teleoperation data frames have been received consecutively. The input hold count can increase with the consecutive absence of new teleoperation data frames and can be reset upon the receipt of a new valid teleoperation data frame. The input hold count is used to measure the duration of teleoperation data interruption.

[0071] The maximum number of hold operations refers to the maximum number of times an input hold operation is allowed to be executed consecutively. This maximum number can be set based on the control frequency, robot safety requirements, and the permissible interruption time for the teleoperation data. For example, with a control frequency of 100 Hz and a maximum permissible input hold time of 100 milliseconds, the maximum number of hold operations can be set to 10. By setting a maximum number of hold operations, the repeated use of the previous valid teleoperation data frame can be limited, reducing the risk of the robot executing expired teleoperation intentions.

[0072] Input hold processing refers to a method of continuing control processing based on the last valid teleoperation data frame when no new teleoperation data frame is received within the target control cycle and the number of input hold attempts has not reached the maximum hold limit. Input hold processing can be used to address situations such as communication jitter, short-term frame loss, and short-term blockage at the operator end. Input hold processing does not predict teleoperation data; instead, it uses the most recent still valid teleoperation data frame to maintain control continuity.

[0073] The previous valid teleoperation data frame refers to the most recent teleoperation data frame deemed valid and usable for generating robot control data before the target control cycle. The previous valid teleoperation data frame can be updated upon receiving a new valid teleoperation data frame, or it can be reused for input hold processing if no new teleoperation data frame is received for a short period. The previous valid teleoperation data frame provides the data basis for generating target robot control data during short data interruptions.

[0074] Target robot control data can refer to robot control data generated during this control processing and used to control the robot's actions.

[0075] In this embodiment, motion control can manifest as the robot maintaining the previous valid posture, retaining the previous valid action state, or continuing to execute teleoperation actions affected by input hold processing based on the target robot control data corresponding to the previous valid teleoperation data frame. Through motion control, the results of input hold processing can be applied to the robot's motion process.

[0076] In this embodiment, by setting the teleoperation data as multiple teleoperation data frames generated in a time sequence, the continuity of teleoperation data input can be identified on a control cycle basis. By triggering input hold processing when no new teleoperation data frame is received within the target control cycle and the number of input hold counts has not reached the hold limit, short-term teleoperation data interruptions can be distinguished from long-term teleoperation data failures. By generating the target robot control data corresponding to the current control processing based on the previous valid teleoperation data frame, the continuity of robot motion control can be maintained during short-term frame loss. By setting a hold limit, the duration of input hold processing can be limited, reducing the risk of the robot performing actions based on expired teleoperation data for extended periods. This embodiment addresses the problem that short-term teleoperation data interruptions during robot teleoperation can easily lead to motion control interruptions or discontinuous robot actions. It adopts a mechanism of performing input hold processing based on the previous valid teleoperation data frame when the number of input hold counts has not reached the hold limit, thereby improving the continuity and stability of robot teleoperation control under short-term data interruptions.

[0077] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each target control cycle lasting 10 milliseconds, and a maximum number of cycles set to 10. The operator continuously generates teleoperation data frames from the 1st to the 50th target control cycle, and the teleoperation data frame of the 50th target control cycle is recorded as the previous valid teleoperation data frame.

[0078] In the 51st target control cycle, the teleoperation data receiving stage did not receive any new teleoperation data frames. At this point, the input hold count was 1, falling short of the maximum hold count of 10. The robot teleoperation control method determines a safety intervention strategy including input hold processing and generates the target robot control data corresponding to the current control processing based on the previous valid teleoperation data frame corresponding to the 50th target control cycle. For example, if the target joint angle of the robot's right arm is 30 degrees and the target position of the robot base is the current position in the 50th target control cycle, the robot teleoperation control method can continue to perform motion control on the robot based on this target data in the 51st target control cycle.

[0079] If no new teleoperation data frame is received between the 52nd and 60th target control cycles, the number of input hold counts increases sequentially to 10, and reaches the upper limit of 10 hold counts in the 60th target control cycle. In this case, the robot teleoperation control method will no longer continue to perform input hold processing based on the previous valid teleoperation data frame corresponding to the 50th target control cycle. Instead, it can restrict the generation or output of robot control data, such as stopping the generation of new follow control data based on the previous valid teleoperation data frame corresponding to the 50th target control cycle, or controlling the robot to enter a safe hold posture.

[0080] In some embodiments, the link status information indicating that the teleoperation link meets the safety intervention conditions may include: for any processing stage in the data conversion stage and / or the motion solving stage, the link status information corresponding to the processing stage indicates that the processing time exceeds the corresponding set processing time and the number of consecutive reuses has not reached the reuse limit.

[0081] The security intervention strategy may include result reuse processing.

[0082] The action control of the robot based on the safety intervention strategy may include: performing result reuse processing based on the processing result of the processing link corresponding to the previous control cycle to generate target robot control data corresponding to the current control processing, and performing action control of the robot based on the target robot control data.

[0083] Processing time refers to the time consumed from the start of processing to obtaining the result. For data conversion, processing time can be the time from receiving teleoperation data to obtaining target motion information; for motion solving, processing time can be the time from receiving target motion information to obtaining motion solution results. Processing time can be used to determine whether a processing step exceeds the real-time control budget time.

[0084] The set processing time refers to the maximum allowable processing time preset for a processing stage. The set processing time can be determined based on the robot's control frequency, the length of each control cycle, the processor's computing power, and the real-time requirements of motion control. Different processing stages can correspond to the same or different set processing times. As an example, the set processing time for the data conversion stage can be less than the set processing time for the motion solving stage.

[0085] The number of consecutive reuses refers to the number of times the processing results from the previous control cycle are reused. The number of consecutive reuses measures the duration of processing timeouts. For example, if the motion solving process exceeds the set processing time in two consecutive target control cycles, and the motion solving results from the previous control cycle are reused in both cycles, then the number of consecutive reuses can be recorded as 2.

[0086] The reuse limit refers to the maximum number of times the result processing is allowed to be reused consecutively. The reuse limit can be set based on the control frequency, robot motion continuity requirements, and safety requirements. For example, at a control frequency of 100 Hz, the reuse limit can be set to 2 times, corresponding to a continuous reuse window of approximately 20 milliseconds. By setting the reuse limit, the robot's reliance on expired process results for motion control can be limited.

[0087] In this embodiment, result reuse processing can be used in scenarios such as short-term processing timeouts, temporary computing resource shortages, or increased time consumption for complex posture solutions. Result reuse processing does not wait for the current timeout processing to complete again, but instead uses the processing results already effectively obtained in the previous control cycle to generate the target robot control data corresponding to the current control processing.

[0088] The processing result corresponding to the previous control cycle can refer to the processing result obtained by the data conversion or motion solving stage in the adjacent control cycle before the target control cycle and usable for motion control. For the data conversion stage, the processing result corresponding to the previous control cycle may include the target motion information of the previous control cycle; for the motion solving stage, the processing result corresponding to the previous control cycle may include joint angles, base poses, or other motion solving results from the previous control cycle. The processing result corresponding to the previous control cycle provides the data basis for result reuse processing.

[0089] In this embodiment, motion control can manifest as the robot continuing to execute the corresponding action based on the processing result of the previous control cycle, or maintaining the action state corresponding to the previous control cycle during the short processing timeout period.

[0090] In this embodiment, by monitoring the processing time of the data conversion and / or motion solving stages, situations where the generation of target motion information or motion solving exceeds the real-time control budget can be identified in a timely manner. By using the condition that the processing time exceeds the set processing time and the number of consecutive reuses does not reach the reuse limit, a distinction can be made between short-term processing timeouts and continuous processing anomalies. By reusing the results based on the processing results of the previous control cycle, the continuity of the target robot control data can be maintained when new processing results are not obtained in time in the current control cycle. By setting a reuse limit, the number of consecutive result reuses can be limited, reducing the risk of the robot relying on expired processing results to perform actions for extended periods. This embodiment addresses the problem that data conversion or motion solving during robot teleoperation may exceed the real-time control budget due to complex postures and changes in computational load, leading to discontinuous robot motion control. It adopts a mechanism of reusing the processing results of the previous control cycle when the number of consecutive reuses does not reach the reuse limit, thereby improving the continuity and stability of robot teleoperation control under short-term processing timeouts.

[0091] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each target control cycle lasting 10 milliseconds. The data conversion stage has a set processing time of 3 milliseconds, and the motion solution stage has a set processing time of 6 milliseconds. Both have a maximum reuse limit of 2 times.

[0092] In the 100th target control cycle, the motion solving process obtains the motion solution result within 5 milliseconds. This result includes the robot's right shoulder joint angle of 20 degrees, right elbow joint angle of 45 degrees, and robot base posture angle of 0 degrees, and serves as the processing result for the 100th target control cycle. In the 101st target control cycle, due to the complexity of the target posture, the motion solving process takes 8 milliseconds to process, exceeding the set processing time of 6 milliseconds, and the number of consecutive reuses is 0, falling short of the reuse limit by 2 times. The robot teleoperation control method determines a safety intervention strategy that includes result reuse processing and generates target robot control data for the 101st target control cycle based on the motion solution result for the 100th target control cycle, enabling the robot to continue motion control according to the right shoulder joint angle of 20 degrees, right elbow joint angle of 45 degrees, and robot base posture angle of 0 degrees.

[0093] If the motion solution step in the 103rd target control cycle still exceeds the set processing time, the number of consecutive reuses increases to 2, and the maximum number of reuses is reached, then the robot teleoperation control method will no longer continue to reuse the results based on the processing results of the previous control cycle. Instead, it can restrict the generation or output of robot control data, such as stopping the generation of target robot control data that follows the changes in the current teleoperation data, or controlling the robot to enter a safe holding posture based on the most recently generated target robot control data.

[0094] In some embodiments, the link status information indicating that the teleoperation link meets the safety intervention conditions may include: the link status information corresponding to the motion state protection link indicating that the position residual exceeds a set residual.

[0095] The process of determining the position residual may include filtering state information and position observation information. The process of determining the position residual may include: determining the filtering state of the robot's motion state filter in the previous control cycle based on the filtering state information; predicting the predicted position quantity corresponding to the target control cycle based on the filtering state of the motion state filter in the previous control cycle; determining the position correlation quantity corresponding to the target control cycle based on the position observation information; and determining the position residual based on the position correlation quantity corresponding to the target control cycle and the predicted position quantity.

[0096] The safety intervention strategy may include motion state protection processing, which includes position jump protection processing.

[0097] The motion control of the robot based on the safety intervention strategy may include: performing position jump protection processing based on the position-related quantities corresponding to the target control cycle, generating target robot control data corresponding to the current control processing, and performing motion control of the robot based on the target robot control data.

[0098] The position-related quantities may include joint angles and / or robot base positions; the set residuals may include a first set residual corresponding to the joint angles, and / or a second set residual corresponding to the robot base positions.

[0099] Position residuals refer to the difference between the position-related quantity corresponding to the target control period and the predicted position quantity. The larger the position residual, the more significant the deviation between the position-related quantity corresponding to the target control period and the predicted motion trend.

[0100] Setting a residual can refer to a threshold used to determine whether the position residual exceeds the allowable deviation range. The residual can be set according to the type of position-related quantity; for example, a first set residual corresponds to the joint angle, and a second set residual corresponds to the robot base position. By setting different set residuals for different types of position-related quantities, joint angle jump detection and robot base position jump detection can have different sensitivities.

[0101] A motion state filter can be defined as a filter used to predict the motion state of a target control cycle based on historical motion states. The motion state filter can record the filtering state of the previous control cycle and predict the corresponding position value for the target control cycle based on that filtering state.

[0102] The filtered state of the previous control cycle can refer to the state estimation result obtained by the motion state filter in the most recent control cycle before the target control cycle. Robot teleoperation control methods can predict the predicted position for the target control cycle based on the filtered state of the previous control cycle.

[0103] The predicted position quantity refers to the position estimate of the target control cycle predicted based on the filtered state of the motion state filter in the previous control cycle. The predicted position quantity represents a reasonable prediction result of the position-related quantity of the target control cycle under continuously changing motion states. By comparing the position-related quantity corresponding to the target control cycle with the predicted position quantity, it can be determined whether there are abnormal jumps in the position-related quantity.

[0104] Position-related quantities can refer to quantities used to characterize the robot's position, posture, or joint angle state. In this embodiment, position-related quantities may include joint angles and / or robot base position. Position-related quantities can serve as the observation basis for calculating position residuals.

[0105] Joint angles refer to the angular positions of robot joints during the target control cycle. Since joint angles directly affect robot limb movements, excessive changes in joint angles between adjacent control cycles can lead to abrupt changes in robot movements. Therefore, a first preset residual can be set for the position residual corresponding to the joint angle.

[0106] The robot base position refers to the location of the robot body or robot base in space. The robot base position can be used to describe the overall translational state of the robot or the reference position of the robot's torso. If the robot base position deviates significantly from the predicted position, it may indicate a jump in teleoperation data or motion processing results. Therefore, a second preset residual can be set for the position residual corresponding to the robot base position.

[0107] In this embodiment, the safety intervention condition can be used to identify situations where the position-related quantity deviates abnormally from the predicted position quantity, enabling the robot teleoperation control method to perform protective processing when the motion state quantity undergoes a significant jump.

[0108] Position jump protection processing refers to the method of safely processing the position-related quantities corresponding to the target control cycle and generating the target robot control data corresponding to this control processing when the position residual exceeds the set residual. For example, it first determines whether the position-related quantities have an abnormal jump relative to the predicted motion trend; if a jump occurs, instead of directly generating the velocity output according to the jump, the velocity state of the motion state filter is set to zero, so that subsequent robot control data does not inherit the unstable motion state caused by the abnormal jump.

[0109] In this embodiment, by predicting the predicted position value corresponding to the target control cycle based on the filtering state of the motion state filter in the previous control cycle, a reference basis for determining whether the position-related quantity changes continuously can be obtained. By determining the position residual based on the position-related quantity corresponding to the target control cycle and the predicted position value, the deviation of the position-related quantity corresponding to the target control cycle from the predicted motion trend can be quantified. By determining that the teleoperation link meets the safety intervention conditions when the position residual exceeds the set residual, the risk of jumps in joint angles or robot base positions can be identified in a timely manner. By distinguishing the set residual into a first set residual corresponding to the joint angle and a second set residual corresponding to the robot base position, the adaptability of jump detection for different types of position-related quantities can be improved. By performing position jump protection processing based on the position-related quantity corresponding to the target control cycle and generating target robot control data, the risk of position-related quantity jumps being directly transmitted to the robot motion control process can be reduced. This embodiment addresses the problem that position-related quantities may change abruptly during robot teleoperation due to teleoperation data jumps or motion processing anomalies, leading to discontinuous robot movements or motion shocks. It adopts a mechanism that determines the position residual based on the predicted position quantity and position-related quantity, and performs position jump protection processing when the position residual exceeds the set residual, thereby improving the smoothness, stability and safety of robot motion control.

[0110] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each target control cycle lasting 10 milliseconds. For the robot's right elbow joint angle, the first set residual is set to 0.5 radians; for the robot's base position, the second set residual is set to 0.3 meters.

[0111] In the 200th control cycle, the motion state filter corresponding to the robot's right elbow joint obtains a filtered state including a position state of 1.00 radians and a velocity state of 0.20 radians per second. Based on this filtered state, the predicted position for the 201st control cycle is 1.002 radians. If the right elbow joint angle for the 201st control cycle is 1.05 radians, the position residual is 0.048 radians, which does not exceed the first preset residual of 0.5 radians. Therefore, the robot teleoperation control method can assume that the right elbow joint angle has not undergone a significant change and continue motion control according to the method of not triggering the position change protection.

[0112] If the right elbow joint angle corresponding to the 202nd control cycle suddenly changes to 1.80 radians, while the predicted position based on the filtered state of the previous control cycle is 1.06 radians, the position residual is 0.74 radians, exceeding the first set residual of 0.5 radians. In this case, the robot teleoperation control method determines that the teleoperation link meets the safety intervention conditions and uses position jump protection processing to generate the target robot control data corresponding to this control processing. For example, the robot teleoperation control method can perform protection processing based on the right elbow joint angle corresponding to the 202nd control cycle, so that the target robot control data does not directly inherit the violent movement trend corresponding to this sudden change, thereby controlling the robot's right elbow joint to perform actions in a protected manner.

[0113] In some embodiments, the motion state protection process may include a velocity output protection process, which may include at least one of a base angular velocity estimation process, a velocity increment clamping process, and a velocity freeze process. Specifically, the base angular velocity estimation process can be used to determine the angular velocity of the robot base in the body coordinate system based on the robot base attitude changes corresponding to adjacent control cycles; the velocity increment clamping process can be used to clamp the velocity-related quantity when the change in the velocity output relative to the previous control cycle exceeds a velocity increment threshold; and the velocity freeze process can be used to set the velocity-related quantity used to generate the target robot control data to zero when a velocity freeze condition is met.

[0114] Base angular velocity estimation refers to a process that estimates the robot base angular velocity based on the robot base posture corresponding to adjacent control cycles. This process first determines the rotational increment of the robot base posture between adjacent control cycles, then converts the rotational increment into a rotation vector, and filters multiple components of the rotation vector to obtain the angular velocity of the robot base in the body coordinate system. This method makes the calculated robot base angular velocity results more continuous and smooth.

[0115] Robot base pose refers to the orientation of the robot's main body, waist, torso, or floating base in space. Robot base pose can be expressed using rotation matrices, quaternions, rotation vectors, or other pose representations. The robot base pose corresponding to adjacent control cycles can be used to calculate the pose change of the robot base within a control cycle, thus providing the input basis for base angular velocity estimation.

[0116] The rotation increment refers to the relative rotational change between the robot base postures of adjacent control cycles. It characterizes the amount of rotation that occurs when the robot base changes posture from the previous control cycle to the target control cycle posture. The rotation increment can be calculated from the robot base posture of the previous control cycle and the robot base posture of the target control cycle, and can be further converted into a rotation vector.

[0117] A rotation vector can be a vector form representing the rotation increment. The direction of the rotation vector can characterize the direction of the rotation axis, and the magnitude of the rotation vector can characterize the rotation amplitude. After converting the rotation increment into a rotation vector, the multiple components of the rotation vector can be filtered separately to obtain a smooth angular velocity estimation result.

[0118] Filtering multiple components separately can refer to smoothing the components of the rotation vector in each of the three directions individually. Each component can correspond to an independent filtering process to estimate the angular velocity changes of the robot base in different directions. By using discrete filtering, mutual interference between attitude changes in different directions can be reduced, improving the stability of the base angular velocity estimation.

[0119] Angular velocity in the body coordinate system refers to the angular velocity expressed with the robot body or robot base as the reference coordinate system. Compared with angular velocity in the external world coordinate system, angular velocity in the body coordinate system is easier to use directly in the robot control process because robot control data can be generated based on the robot's own posture and coordinate relationships.

[0120] Incremental speed clamping refers to a method of limiting the speed change when the speed-related quantity in the target control cycle changes too much compared to the speed output in the previous control cycle. Incremental speed clamping does not necessarily change the direction of the speed-related quantity itself, but rather limits the maximum range of change of the speed-related quantity between adjacent control cycles, thereby reducing the impact of sudden increases or decreases in speed on robot movements.

[0121] Velocity-related quantities refer to data used to characterize the movement speed of a robot or robot components. Velocity-related quantities can include at least one of the following: base linear velocity, base angular velocity, and joint velocities. Base linear velocity characterizes the translational speed of the robot base, base angular velocity characterizes the speed of change in the robot base's posture, and joint velocities characterize the speed of change in the angles of each joint of the robot.

[0122] For velocity increment clamping, in some embodiments, the base angular velocity can be estimated using the aforementioned base angular velocity estimation method. In other embodiments, the base angular velocity can also be calculated using the attitude angle difference method. For example, the robot base attitude corresponding to adjacent control cycles can be converted into base attitude angles, and the base angular velocity can be calculated based on the difference between the base attitude angles corresponding to adjacent control cycles and the control cycle duration.

[0123] The velocity output corresponding to the previous control cycle refers to the velocity output result already used for robot motion control in the adjacent control cycle preceding the target control cycle. The velocity output corresponding to the previous control cycle can serve as a reference for determining whether a sudden change has occurred in the current velocity-related quantity. By comparing the velocity-related quantity with the velocity output corresponding to the previous control cycle, the velocity change can be determined.

[0124] The velocity change can refer to the difference or magnitude of the velocity-related quantity corresponding to the target control cycle and the velocity output quantity corresponding to the previous control cycle. The velocity change can be determined separately for different velocity components, such as the base linear velocity change, base angular velocity change, and joint velocity change. The velocity change is used to determine whether velocity increment clamping processing is required.

[0125] The speed increment threshold refers to the maximum allowable speed change between adjacent control cycles. The speed increment threshold can be set based on the robot actuator's capabilities, control frequency, motion safety requirements, and the continuity requirements of teleoperation actions. For example, the speed increment threshold can be set to 0.15 to limit the maximum increment of speed-related quantities within each control cycle relative to the speed output of the previous control cycle.

[0126] The processed speed-related quantity can refer to the speed data obtained after speed increment clamping. If the speed change exceeds the speed increment threshold, the processed speed-related quantity can be obtained by adding the speed increment limited by the speed increment threshold to the speed output corresponding to the previous control cycle; if the speed change does not exceed the speed increment threshold, the processed speed-related quantity can be directly adopted from the speed-related quantity corresponding to the target control cycle.

[0127] Velocity freeze processing refers to a process that sets the velocity-dependent quantities used to generate robot control data to zero when velocity freeze conditions are met. Velocity freeze processing can be applied to at least one of the base linear velocity, base angular velocity, and joint velocity, limiting the robot's velocity output while position-dependent quantities can still participate in control, thereby enabling the robot to enter a more stable motion control state.

[0128] A velocity freeze condition can be any condition that triggers the velocity freeze process. Velocity freeze conditions can include teleoperation pause conditions, debugging conditions, and abnormal motion state conditions. When a velocity freeze condition is met, it can be assumed that the robot teleoperation control method needs to limit the velocity-dependent quantities from continuing to drive the robot's motion, thereby setting the velocity-dependent quantities to zero.

[0129] In this embodiment, the target robot control data can be generated based on the base angular velocity estimation result, the processed velocity correlation quantity, the velocity correlation quantity set to zero, and the position correlation quantity.

[0130] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each target control cycle lasting 10 milliseconds. In the base angular velocity estimation process, at the 300th target control cycle, the robot base is in the first pose; at the 301st target control cycle, the robot base rotates approximately 0.02 radians relative to the first pose. The robot teleoperation control method can convert this rotational change into a rotation vector and filter the three components of the rotation vector to obtain the angular velocities of the robot base in the body coordinate system, such as 0.8 radians per second, 0.1 radians per second, and 0.0 radians per second.

[0131] In velocity increment clamping, assuming the output velocity of a joint in the previous control cycle was 0.20 radians per second, and the velocity correlation value obtained from the teleoperation data in the target control cycle is 0.50 radians per second, with a velocity change of 0.30 radians per second. If the velocity increment threshold is 0.15 radians per second, the robot teleoperation control method limits the joint velocity to 0.35 radians per second, using this as the processed velocity correlation value, and generates the target robot control data based on this processed velocity correlation value. If the velocity correlation value obtained from the teleoperation data in the target control cycle is 0.28 radians per second, with a velocity change of 0.08 radians per second, which does not exceed the velocity increment threshold of 0.15 radians per second, the robot teleoperation control method can use 0.28 radians per second as the processed velocity correlation value.

[0132] In the velocity freeze process, if the velocity freeze condition is met in the 302nd target control cycle, the robot teleoperation control method can set the base linear velocity, base angular velocity, and joint velocity to 0, and generate target robot control data based on the current joint angle and robot base position. For example, the robot's right elbow joint angle is maintained at 45 degrees, the robot base position is maintained at the current position, and the robot enters a velocity-limited stable motion state based on this target robot control data.

[0133] In some embodiments, the link status information indicating that the teleoperation link meets the security intervention conditions may include: the link status information corresponding to the control data publishing link indicating that the data freshness does not meet the data validity conditions.

[0134] The process of determining the data freshness may include: determining the acquisition time information of the most recently generated target robot control data based on the process status information corresponding to the control data release process; and determining the data freshness based on the time difference between the current control time and the acquisition time information of the most recently generated target robot control data.

[0135] The security intervention strategy may include release control processing.

[0136] The motion control of the robot based on the safety intervention strategy may include: performing a release control process on the most recently generated target robot control data to restrict the output of the most recently generated target robot control data to the robot, so that the robot enters a safe holding posture.

[0137] Data freshness refers to the time validity of the most recently generated target robot control data relative to the current control time. Data freshness can be determined based on the time difference between the current control time and the acquisition time of the most recently generated target robot control data.

[0138] Data validity conditions refer to the conditions under which robot control data is still allowed to be output to the robot. Data validity conditions can include data freshness being less than or equal to a set time difference. By setting data validity conditions, the robot can be prevented from executing expired robot control data for extended periods.

[0139] The most recently generated target robot control data can refer to robot control data that was successfully generated by the robot teleoperation control method and determined to be usable for robot motion control before the current control moment. The most recently generated target robot control data can serve as reference data in scenarios with short-term input anomalies or short-term output maintenance, but if the data freshness does not meet the data validity condition, it is necessary to restrict further output to the robot.

[0140] Acquisition time information refers to the time when the most recently generated target robot control data was acquired by the control data publishing stage. Acquisition time information can be represented by a timestamp or a control cycle number. Based on the acquisition time information, the time difference between the most recently generated target robot control data and the current control time can be calculated, thereby determining the data freshness.

[0141] Release control processing refers to a method of controlling the output of robot control data based on data freshness and validity conditions. Release control processing operates on the robot control data output side to reduce the risk of the robot continuously executing outdated control data.

[0142] Restricting the output of the most recently generated target robot control data to the robot can include stopping output, reducing the output frequency, outputting safety-holding data, or switching to control data corresponding to the safety-holding posture. Different restriction methods can be determined based on data freshness, the duration of teleoperation link anomalies, the robot's current posture, and the robot controller's requirements for control data reliability.

[0143] For example, if the data freshness exceeds the first valid duration but not the second valid duration, and the duration of the teleoperation link anomaly indicates short-term jitter or short-term delay, the output frequency of the most recently generated target robot control data can be reduced, or the frequency of updating the control output based on that target robot control data can be reduced. In this way, the impact of expired control data on robot motion changes can be reduced during short-term anomalies, while maintaining the basic control input required by the robot controller.

[0144] If the data freshness exceeds the second valid duration but not the third valid duration, and the robot's current posture is still within the allowable holding range, the output of target robot control data for continuing to follow teleoperated actions can be stopped, and instead, safe holding data can be output. Safe holding data may include at least one of the following: current joint angle, current base position, zero velocity data, posture holding flag, or safe holding flag. By outputting safe holding data, the robot can reduce the likelihood of new motion changes arising from continuing to follow outdated teleoperated data.

[0145] If the data freshness exceeds the third valid time period, or if the most recently generated target robot control data no longer meets the control reliability requirements, the control data corresponding to the safe posture maintenance can be switched to. The control data corresponding to the safe posture maintenance can be used to control the robot to maintain the current stable posture, maintain the preset standing posture, set the speed-related quantities to zero, or enter a controlled stop state.

[0146] In this embodiment, by determining the acquisition time information of the most recently generated valid target robot control data based on the process status information corresponding to the control data release stage, a time basis can be provided for determining whether the robot control data has expired. By determining data freshness based on the acquisition time information, the validity of the robot control data relative to the current control moment can be quantified. By performing release control processing when the data freshness does not meet the data validity condition, the continued output of expired robot control data to the robot can be restricted. By putting the robot into a safe holding posture, the risk of the robot continuing to execute expired robot control data, leading to loss of control or delayed actions, can be reduced. This embodiment addresses the problem that the robot may continue to execute expired robot control data after the teleoperation link is interrupted or delayed during robot teleoperation by adopting a mechanism for release control processing based on data freshness and data validity conditions, thereby improving the safety and controllability of robot teleoperation control.

[0147] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each target control cycle lasting 10 milliseconds. The data validity condition is set to a data freshness of no more than 300 milliseconds. In the 100th target control cycle, the robot teleoperation control method effectively generates a set of robot control data. The acquisition time information corresponding to this robot control data is 1000 milliseconds. The robot control data includes a right shoulder joint angle of 20 degrees, a right elbow joint angle of 45 degrees, and the robot base position as the current position.

[0148] In the 120th target control cycle, the current control time is 1200 milliseconds. Based on the acquired time information of 1000 milliseconds, the data freshness is determined to be 200 milliseconds, which does not exceed 300 milliseconds. The data freshness meets the data validity condition, and the robot teleoperation control method can allow the target robot control data generated most recently to continue to participate in the control output.

[0149] In the 135th target control cycle, the current control time is 1350 milliseconds. Based on the acquisition time information of 1000 milliseconds, the data freshness is determined to be 350 milliseconds, which exceeds 300 milliseconds, and the data freshness does not meet the data validity condition. At this time, the robot teleoperation control method determines a safety intervention strategy, including issuing control processing and restricting the continued output of target robot control data effectively generated in the 100th target control cycle to the robot, causing the robot to enter a safe posture maintenance state. For example, the robot can maintain a right shoulder joint angle of 20 degrees, a right elbow joint angle of 45 degrees, and the current base position, and restrict further new motion changes based on expired robot control data.

[0150] In some embodiments, the multiple teleoperation markers on the robot's corresponding operating end may include at least one pre-set first marker and at least one second marker.

[0151] The link status information indicates that the teleoperation link meets the security intervention conditions, and may include: the link status information corresponding to the tag status management link indicates that the first tag is available and at least some of the second tags are unavailable.

[0152] The security intervention strategy may include constraint adjustment processing.

[0153] The motion control of the robot based on the safety intervention strategy may include: adjusting the constraints of the candidate motion solution constraints corresponding to the multiple teleoperation marks based on the link state information corresponding to the mark state management link to obtain the target motion solution constraints; performing motion solution based on the target motion solution constraints to generate the target robot control data corresponding to this control process; and performing motion control of the robot based on the target robot control data.

[0154] Teleoperation tags can refer to tagged information used by an operator to represent the operator's body parts, end-effector position, posture, or intention to move. Teleoperation tags can come from motion capture devices, wearable sensors, or visual recognition devices.

[0155] The first marker can be one of multiple teleoperation markers that maintains the robot's basic teleoperation functions. The first marker may include a pelvic marker for determining the robot's base position, a torso marker for determining the robot's torso posture, a hand marker for determining the position of the robot's end effector, or a foot marker for determining the position of the robot's feet. When the first marker is available, the robot teleoperation control method can assume that the robot still possesses the key input conditions for solving basic motion problems.

[0156] The second marker can be one of multiple teleoperation markers used to supplement motion solution accuracy, improve posture constraints, or support fine motion control. The second marker may include an elbow reference marker, a knee reference marker, a finger auxiliary marker, or other teleoperation markers used to supplement motion direction, joint posture, and end-effector details. When the second marker is unavailable, the robot teleoperation control method may reduce some motion accuracy, but can still continue to perform limited teleoperation control if the first marker is available.

[0157] In this embodiment, the safety intervention condition may include the stage state information corresponding to the stage state management stage, indicating that the first marker is available and at least some of the second markers are unavailable. This safety intervention condition is used to indicate that the robot teleoperation control method still retains the basic teleoperation input, but some supplementary or fine-grained input is missing. Therefore, it can trigger constraint adjustment processing instead of directly putting the robot into a safe holding posture. Thus, the robot teleoperation control method can continue to control the robot by adjusting the motion to solve the constraints, instead of interrupting the teleoperation control.

[0158] Constraint adjustment processing refers to a method of adjusting candidate motion solution constraints based on the state information of the corresponding stage in the marker state management stage. Constraint adjustment processing can include removing candidate motion solution constraints corresponding to unavailable second markers, reducing the impact of candidate motion solution constraints corresponding to unavailable second markers, increasing the weight of candidate motion solution constraints corresponding to available markers, or supplementing motion state variables corresponding to unavailable markers based on the previous valid state. Through constraint adjustment processing, target motion solution constraints that can still be used for motion solution can be formed even when some second markers are unavailable.

[0159] Candidate motion solution constraints refer to constraints that are pre-determined or generated in real-time based on multiple teleoperated markers and can participate in motion solving. These can include robot base position constraints, robot torso posture constraints, hand end-effector position constraints, foot position constraints, elbow direction constraints, knee direction constraints, or finger joint constraints. Before constraint adjustment, candidate motion solution constraints can correspond to all teleoperated markers; after constraint adjustment, they can form the target motion solution constraints.

[0160] The target motion solution constraint refers to the constraint conditions that actually participate in the motion solution after constraint adjustment. The target motion solution constraint can include the basic motion solution constraint corresponding to the first marker, and the supplementary motion solution constraint corresponding to the available second marker. The target motion solution constraint does not necessarily include all constraints corresponding to the unavailable second marker, thus reducing the interference of unavailable markers on the motion solution results and improving the usability of the motion solution process in scenarios where markers are missing.

[0161] In this embodiment, by dividing multiple teleoperation markers into first markers and second markers, it is possible to distinguish between critical inputs that maintain basic teleoperation functions and auxiliary inputs used to supplement accuracy or fine movements. By triggering constraint adjustment processing when the first marker is available and at least some of the second markers are unavailable, motion solving can continue while the critical inputs remain reliable. By adjusting candidate motion solution constraints based on the link state information corresponding to the marker state management link, the interference of constraints corresponding to unavailable markers on the motion solution results can be reduced. By generating target robot control data based on target motion solution constraints, the robot can continue to execute restricted but controllable teleoperation actions even when some of the second markers are unavailable. This embodiment addresses the problem that directly interrupting teleoperation control when some teleoperation markers are unavailable during robot teleoperation reduces continuity, while continuing to use constraints corresponding to unavailable markers may reduce the reliability of motion solutions. It adopts a mechanism of adjusting motion solution constraints based on the availability of markers, thereby improving the stability and robustness of robot motion control while maintaining basic teleoperation functions.

[0162] For example, the operator terminal is equipped with six teleoperation markers. The first marker includes a pelvic marker, a left-hand marker, and a right-hand marker. The second marker includes a left elbow reference marker, a right elbow reference marker, and a finger auxiliary marker. The constraints for solving candidate motions include the robot base position constraint corresponding to the pelvic marker, the left hand end-effector position constraint corresponding to the left-hand marker, the right hand end-effector position constraint corresponding to the right-hand marker, the left elbow direction constraint corresponding to the left elbow reference marker, the right elbow direction constraint corresponding to the right elbow reference marker, and the finger joint constraints corresponding to the finger auxiliary marker.

[0163] Within a specific target control cycle, the state information representations of the pelvic marker, left-hand marker, right-hand marker, and left elbow reference marker in the marker state management stage are all available, but the right elbow reference marker and finger auxiliary marker are unavailable. At this point, the robot teleoperation control method determines a safety intervention strategy, including constraint adjustment processing, and adjusts the candidate motion solution constraints. For example, the robot teleoperation control method retains the robot base position constraint, left-hand end-effector position constraint, right-hand end-effector position constraint, and left elbow direction constraint, removes the right elbow direction constraint and finger joint constraint, and can increase the weights of the left-hand end-effector position constraint and right-hand end-effector position constraint. The adjusted target motion solution constraints are used for motion solving to generate target robot control data, enabling the robot to continue performing teleoperation actions based on the pelvic marker, left-hand marker, right-hand marker, and left elbow reference marker.

[0164] In some embodiments, the robot control data may include foot contact state, which may be determined based on foot height and foot speed.

[0165] The method may further include: updating the foot contact state to a contact state when the foot contact state is a non-contact state, the foot height is less than a first height threshold, and the foot speed is less than or equal to a contact speed threshold; updating the foot contact state to a non-contact state when the foot contact state is a contact state, the foot height is greater than a second height threshold, and / or the foot speed is greater than a ground clearance speed threshold; maintaining the foot contact state as a contact state when the foot contact state is a contact state, the foot height is less than or equal to the second height threshold, and the foot speed is less than or equal to a stationary speed threshold; and performing control processing based on the updated or maintained foot contact state.

[0166] The updated or maintained foot contact state can be used to control the robot's motion; the second height threshold can be greater than the first height threshold; the stationary speed threshold can be less than or equal to the contact speed threshold, and the contact speed threshold can be less than or equal to the ground takeoff speed threshold.

[0167] Foot contact state refers to the contact relationship between the robot's foot and the supporting surface. Foot contact state can include both contact and non-contact states. A contact state indicates that the robot's foot is considered to be on the ground or in a supporting state, while a non-contact state indicates that the robot's foot is considered to be off the ground or in a swinging state. Foot contact state can be included as part of the robot's control data in control processing to improve the stability of the robot's gait or posture control.

[0168] Foot height refers to the height of a robot's foot relative to a support surface or reference plane. Foot height can be determined based on the position of the toes, the sole of the foot, foot markers, or the tip of the foot. For example, a smaller foot height indicates that the foot is closer to the support surface; a larger foot height indicates that the foot is more likely to leave the support surface.

[0169] Foot speed refers to the movement speed of the robot's foot relative to the support surface or reference coordinate system. Foot speed can be determined based on changes in foot position between adjacent control cycles. Foot speed is used to assist in determining the foot's contact state. For example, if the foot height is low but the foot speed is high, it may indicate that the foot is rapidly passing near the support surface, in which case it should not be directly determined as stable contact.

[0170] The first height threshold can refer to the height judgment threshold used when the foot contact state switches from a non-contact state to a contact state. The first height threshold can be set to a small value, such as 0.005 meters.

[0171] The second height threshold can be a height judgment threshold used when the foot contact state switches from a contact state to a non-contact state. The second height threshold is greater than the first height threshold, for example, it can be set to 0.04 meters. By setting the second height threshold to be greater than the first height threshold, a hysteresis interval can be formed when the foot height is between the first height threshold and the second height threshold, reducing the frequent switching of the foot contact state caused by fluctuations in foot height around the threshold.

[0172] The contact speed threshold refers to the speed threshold used to determine when a foot changes from a non-contact state to a contact state. The contact speed threshold is used to prevent excessive foot speed from being mistakenly identified as a contact state.

[0173] The ground-lift speed threshold refers to the speed threshold used to determine when the foot's contact state changes from a contact state to a non-contact state. The ground-lift speed threshold can be greater than or equal to the contact speed threshold. When the foot is in a contact state and the foot speed is greater than the ground-lift speed threshold, even if the foot height does not increase significantly temporarily, it can be considered that the foot has a tendency to leave the support surface, thus updating the foot contact state to a non-contact state.

[0174] The stationary speed threshold refers to the speed threshold used to determine whether the foot is in a low-speed or near-stationary state. The stationary speed threshold is less than or equal to the contact speed threshold. When the foot is in contact, the foot height is not greater than the second height threshold, and the foot speed is not greater than the stationary speed threshold, the foot can be maintained in contact to reduce the risk of being misjudged as non-contact during the low-speed support phase.

[0175] Different threshold values ​​can be used to determine when the foot enters and leaves a contact state. In this embodiment, a first height threshold is used when the foot switches from a non-contact state to a contact state, and a second height threshold is used when the foot switches from a contact state to a non-contact state, with the second height threshold being greater than the first height threshold. This hysteresis-based determination mechanism reduces state jitter caused by fluctuations in foot height around a single threshold.

[0176] The updated or maintained foot contact state can be used for motion control of the robot, referring to the process of performing motion control on the robot based on the updated or maintained foot contact state. This process may include adjusting the robot's gait, support judgment, posture maintenance, or foot movement control based on the foot contact state. By incorporating the foot contact state into the motion control process, the robot control process can more accurately distinguish between the foot support phase and the foot swing phase.

[0177] In some embodiments, the robot teleoperation control method can determine the robot's current foot support relationship based on the updated or maintained foot contact state, and perform control processing based on the foot support relationship. The foot support relationship can include at least one of two-foot support, left-foot single support, right-foot single support, airborne state, or abnormal contact state. Specifically, if both the left-foot and right-foot contact states are non-contact states, or if the foot contact state does not match the robot's current action phase, the robot can be determined to be in an airborne state or an abnormal contact state.

[0178] In some embodiments, after determining the supporting foot, the robot teleoperation control method can increase the position constraint weight and / or posture constraint weight of the supporting foot in the control channel corresponding to the supporting foot, or control the end position of the supporting foot to remain continuous, so as to limit abrupt changes in the end position of the supporting foot. The supporting foot can serve as a stable reference for robot posture control and whole-body balance control, thereby improving the stability of the robot during standing, step changing, or posture transitions.

[0179] In some embodiments, after determining the swing foot, the robot teleoperation control method can control the swing foot to perform actions such as lifting, swinging forward, swinging backward, swinging sideways, or landing in the control channel corresponding to the swing foot, based on the target motion information. If the swing foot is detected to have regained contact condition, it can be switched to a supporting foot, and the next supporting phase can begin.

[0180] In some embodiments, when the robot is in a bipedal support state, the robot teleoperation control method can reduce the amount of foot sway control and enhance the control of trunk posture, pelvic posture, or whole-body posture maintenance, enabling the robot to maintain stability in standing, ready-to-start, stop transition, or bipedal landing transition states. Control processing in the bipedal support state can reduce unnecessary foot swaying and enhance overall posture constraints.

[0181] In some embodiments, when the foot contact state is maintained due to sensor malfunction, short-term frame loss, or detection instability, the robot teleoperation control method can continue to use the maintained foot contact state for foot support relationship determination and motion control. By continuing to use the maintained foot contact state, the frequent switching between the supporting foot and the swinging foot caused by momentary foot contact state jitter can be reduced, thereby improving the stability of robot foot control, gait switching, and whole-body balance control.

[0182] In other words, by inputting the updated or maintained foot contact state into the motion control processing, the robot teleoperation control method can determine the robot's current supporting foot and swinging foot, and adjust foot constraints, gait switching, posture maintenance weights or foot motion control strategies accordingly, thereby enabling the robot to more stably distinguish between the foot support phase and the foot swinging phase.

[0183] For example, the robot teleoperation control method operates at a control frequency of 100 Hz, with each control cycle lasting 10 milliseconds. The first height threshold is set to 0.005 meters, the second height threshold is set to 0.04 meters, the stationary speed threshold is set to 0.005 meters per second, the contact speed threshold is set to 0.12 meters per second, and the takeoff speed threshold is set to 0.12 meters per second.

[0184] In the 10th control cycle, the foot contact state is non-contact, the foot height is 0.004 meters, and the foot speed is 0.06 meters per second. Since the foot height is less than the first height threshold of 0.005 meters and the foot speed is not greater than the contact speed threshold of 0.12 meters per second, the foot contact state is updated to contact state.

[0185] During the 11th to 20th control cycles, the foot height fluctuated between 0.010 meters and 0.030 meters, and the foot speed was 0.003 meters per second. Since the foot was in contact, the foot height did not exceed the second height threshold of 0.04 meters, and the foot speed did not exceed the stationary speed threshold of 0.005 meters per second. Therefore, the foot contact state was maintained to reduce the repeated changes in the foot contact state caused by small fluctuations in foot height.

[0186] In the 21st control cycle, the foot contact state is active, the foot height increases to 0.045 meters, and the foot speed is 0.08 meters per second. Since the foot height exceeds the second height threshold of 0.04 meters, the foot contact state is updated to a non-contact state. Alternatively, when the foot height is 0.030 meters but the foot speed increases to 0.15 meters per second, the foot contact state can also be updated to a non-contact state because the foot speed exceeds the ground clearance threshold of 0.12 meters per second. Based on the updated or maintained foot contact state, the robot teleoperation control method executes the corresponding control processing.

[0187] See Figure 2 , Figure 2 This is a structural block diagram of a robot teleoperation control system provided in an embodiment of this application.

[0188] This application also provides a robot teleoperation control system, which includes a teleoperation input module, a status acquisition module, a status analysis module, and a motion control module.

[0189] The teleoperation input module is used to receive teleoperation data generated by the operator for the robot.

[0190] The status acquisition module is used to acquire the link status information of the teleoperation link, which is the processing link from receiving the teleoperation data to outputting robot control data.

[0191] The status analysis module is used to determine the security intervention strategy corresponding to the teleoperation link based on the link status information, when the link status information indicates that the teleoperation link meets the security intervention conditions.

[0192] The motion control module is used to control the robot's motion based on the safety intervention strategy.

[0193] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0194] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0195] The computer program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the computer program product of the embodiments of this application is not limited thereto, and the computer program product may be any combination of one or more computer-readable media.

[0196] See Figure 3 , Figure 3 This is a structural block diagram of a computer device provided in an embodiment of this application.

[0197] This application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above methods.

[0198] The embodiments of this application do not limit the computer device, which may be, for example, a local computer device, a cloud computer device, a distributed computer device, etc.

[0199] The computer device may include: a memory 110, a processor 120, and a communication interface 130. The memory 110, the processor 120, and the communication interface 130 are connected through internal connection paths.

[0200] The memory 110 is used to store computer programs, which in some implementations may include code for implementing the methods of the embodiments of this application.

[0201] The processor 120 executes the computer program stored in the memory 110 to control the communication interface 130 to receive input data and information, and output operation results and other data. In some implementations, when the solutions of the embodiments of this application are implemented by software or firmware, the computer program used to implement the solutions of the embodiments of this application can be stored in the processor 120 and executed by the processor 120.

[0202] The memory 110 may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM). It should be noted that the memory 110 described herein is intended to include, but is not limited to, any memory of these and other suitable types. As an example, the memory 110 includes random access memory (RAM), cache memory, and read-only memory (ROM). The memory 110 stores a computer program that can be executed by processor 120, causing processor 120 to implement the steps of any of the methods described above.

[0203] The processor 120 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor 120 can be any conventional processor.

[0204] In implementation, each step of the above method can be completed by the integrated logic circuitry of the hardware in the processor 120 or by instructions in software form. The method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor 120. The software modules can be located in mature storage media in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in the memory 110, and the processor 120 reads the information in the memory 110 and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0205] In some implementations, in addition to the hardware units described above, computer devices may also include software modules, such as operating systems, basic input / output systems (BIOS), and application software.

[0206] An operating system is used to manage the hardware and / or software resources of a computer device; it is the kernel and foundation of the computer. The operating system handles fundamental tasks such as managing and configuring memory, determining the priority of system resource allocation, controlling input and output devices, operating the network, and managing the file system. To facilitate user operation, most operating systems provide a user interface for interaction with the system.

[0207] The BIOS is used to perform hardware initialization during the power-on boot phase and to provide runtime services for the operating system and applications. In some implementations, the BIOS can also monitor and display processor temperature and execute temperature protection strategies.

[0208] Application software, also known as an application program, is software written for a specific user application purpose and is one of the main categories of computer software. For example, application software can be a program used to achieve purposes such as power control and temperature management.

[0209] It is understood that the specific examples in this application are only intended to help those skilled in the art better understand the implementation of this application, and are not intended to limit the scope of protection of this application.

[0210] It is understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application.

[0211] It is understood that the various implementation methods described in this application can be implemented individually or in combination, and this application does not limit them.

[0212] Unless otherwise stated, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0213] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design intervention conditions of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0214] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the embodiments described above can be referred to the corresponding processes in other embodiments, and will not be repeated here.

[0215] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0216] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the technical solution in this application, depending on actual needs.

[0217] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0218] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, essentially, or the part that contributes to related technologies, or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0219] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for remotely controlling a robot, characterized in that, The method includes: Obtain the link status information of the teleoperation link, which is the processing link from receiving teleoperation data to outputting robot control data; When the link status information indicates that the teleoperation link meets the security intervention conditions, the security intervention strategy corresponding to the teleoperation link is determined based on the link status information. Based on the aforementioned safety intervention strategy, the robot's motion is controlled; The link status information includes link status information corresponding to at least one processing link. The multiple teleoperation markers on the robot's corresponding operating end include at least one pre-set first marker and at least one second marker. The link status information indicates that the teleoperation link meets safety intervention conditions, including: the link status information corresponding to the marker status management link in the teleoperation link indicates that the first marker is available and at least some of the second markers are unavailable; the first marker includes a pelvic marker for determining the robot's base position, a torso marker for determining the robot's torso posture, a hand marker for determining the robot's hand end-effector position, or a foot marker for determining the robot's foot position; the second marker includes an elbow reference marker, a knee reference marker, a finger auxiliary marker, or a teleoperation marker used to supplement movement direction, joint posture, and end-effector details. The security intervention strategy includes constraint adjustment processing; The step of controlling the robot's motion based on the safety intervention strategy includes: adjusting the constraints of the candidate motion solution constraints corresponding to the multiple teleoperation marks based on the link state information corresponding to the mark state management link to obtain the target motion solution constraints; performing motion solution based on the target motion solution constraints to generate the target robot control data corresponding to this control process; and controlling the robot's motion based on the target robot control data. When the first marker is available, the robot still has the key input conditions to perform basic motion solutions; when the second marker is unavailable, the robot continues to perform limited teleoperation control even when the first marker is available.

2. The robot teleoperation control method according to claim 1, characterized in that, The teleoperation link also includes at least one of the following processing steps: teleoperation data receiving, data conversion, motion solving, motion state protection, and control data publishing. The status information of the process includes status type information and status duration information; wherein, the status type information includes at least one of teleoperation input status, data conversion status, motion solving status, motion status protection status, control data release status, and marked availability status; the status duration information includes at least one of the following: the duration for which the corresponding status type information meets the safety intervention conditions, the number of continuous control cycles, and the number of consecutive occurrences.

3. The robot teleoperation control method according to claim 2, characterized in that, The security intervention strategy also includes at least one of input retention processing, result reuse processing, motion state protection processing, and release control processing.

4. The robot teleoperation control method according to claim 3, characterized in that, The teleoperation data includes multiple teleoperation data frames generated in a time sequence. The link status information indicates that the teleoperation link meets the safety intervention conditions. It also includes: the link status information corresponding to the teleoperation data receiving link indicates that no teleoperation data frame was received within the target control cycle, and the number of input hold counts did not reach the upper limit of the hold count. The security intervention strategy also includes input retention processing; The method of controlling the robot's actions based on the safety intervention strategy further includes: performing input hold processing based on the previous valid teleoperation data frame to generate target robot control data corresponding to the current control processing, and performing action control on the robot based on the target robot control data.

5. The robot teleoperation control method according to claim 3, characterized in that, The link status information indicates that the teleoperation link meets the safety intervention conditions, and also includes: for any processing link in the data conversion link and / or the motion solving link, the link status information corresponding to any processing link in the data conversion link and / or the motion solving link indicates that the processing time of the link exceeds the corresponding set processing time, and the number of consecutive reuses has not reached the reuse limit. The security intervention strategy also includes result reuse processing; The method of controlling the robot's motion based on the safety intervention strategy further includes: performing result reuse processing on the processing results of any processing step in the data conversion step and / or the motion solving step corresponding to the previous control cycle to generate target robot control data corresponding to the current control processing, and performing motion control on the robot based on the target robot control data.

6. The robot teleoperation control method according to claim 3, characterized in that, The link status information indicates that the teleoperation link meets the safety intervention conditions, and also includes: the link status information corresponding to the motion state protection link indicates that the position residual exceeds a set residual; wherein, the link status information includes filter status information and position observation information, and the process of determining the position residual includes: determining the filter status of the robot's motion state filter in the previous control cycle based on the filter status information; predicting the predicted position quantity corresponding to the target control cycle based on the filter status of the motion state filter in the previous control cycle; determining the position correlation quantity corresponding to the target control cycle based on the position observation information; and determining the position residual based on the position correlation quantity corresponding to the target control cycle and the predicted position quantity. The safety intervention strategy also includes motion state protection processing, which includes position jump protection processing; The method of controlling the robot's motion based on the safety intervention strategy further includes: performing position jump protection processing based on the position-related quantities corresponding to the target control cycle, generating target robot control data corresponding to the current control processing, and performing motion control on the robot based on the target robot control data; The position-related quantities include joint angles and / or robot base positions; the set residuals include a first set residual corresponding to the joint angles, and / or a second set residual corresponding to the robot base positions.

7. The robot teleoperation control method according to claim 3, characterized in that, The motion state protection process also includes a speed output protection process, which includes at least one of base angular velocity estimation process, speed increment clamping process, and speed freeze process.

8. The robot teleoperation control method according to claim 3, characterized in that, The link status information indicates that the teleoperation link meets the safety intervention conditions, and also includes: the link status information corresponding to the control data publishing link indicates that the data freshness does not meet the data validity conditions; wherein, the process of determining the data freshness includes: determining the acquisition time information of the most recently valid target robot control data based on the link status information corresponding to the control data publishing link; and determining the data freshness based on the time difference between the current control time and the acquisition time information of the most recently valid target robot control data. The security intervention strategy also includes release control processing; The method of controlling the robot's actions based on the safety intervention strategy further includes: performing a release control process on the most recently generated target robot control data to restrict the output of the most recently generated target robot control data to the robot, thereby causing the robot to enter a safe holding posture.

9. The robot teleoperation control method according to any one of claims 4 to 6 and 8, characterized in that, The target robot control data includes foot contact state, which is determined based on foot height and foot speed; The method further includes: If the foot contact state is non-contact, the foot height is less than a first height threshold, and the foot speed is less than or equal to a contact speed threshold, the foot contact state is updated to contact. If the foot is in a contact state and the foot height is greater than a second height threshold and / or the foot speed is greater than a ground clearance threshold, the foot contact state is updated to a non-contact state. When the foot is in contact, the foot height is less than or equal to the second height threshold, and the foot speed is less than or equal to the stationary speed threshold, the foot is maintained in contact. The updated or maintained foot contact state is used to control the robot's motion; the second height threshold is greater than the first height threshold; the stationary speed threshold is less than or equal to the contact speed threshold, and the contact speed threshold is less than or equal to the ground takeoff speed threshold.

10. A robot teleoperation control system, characterized in that, The system includes: The status acquisition module is used to acquire the link status information of the teleoperation link, which is the processing link from receiving teleoperation data to outputting robot control data. The status analysis module is used to determine the security intervention strategy corresponding to the teleoperation link based on the link status information when the link status information indicates that the teleoperation link meets the security intervention conditions. The motion control module is used to control the robot's motion based on the safety intervention strategy. The link status information includes link status information corresponding to at least one processing link. The multiple teleoperation markers on the robot's corresponding operating end include at least one pre-set first marker and at least one second marker. The link status information indicates that the teleoperation link meets safety intervention conditions, including: the link status information corresponding to the marker status management link in the teleoperation link indicates that the first marker is available and at least some of the second markers are unavailable; the first marker includes a pelvic marker for determining the robot's base position, a torso marker for determining the robot's torso posture, a hand marker for determining the robot's hand end-effector position, or a foot marker for determining the robot's foot position; the second marker includes an elbow reference marker, a knee reference marker, a finger auxiliary marker, or a teleoperation marker used to supplement movement direction, joint posture, and end-effector details. The security intervention strategy includes constraint adjustment processing; The step of controlling the robot's motion based on the safety intervention strategy includes: adjusting the constraints of the candidate motion solution constraints corresponding to the multiple teleoperation marks based on the link state information corresponding to the mark state management link to obtain the target motion solution constraints; performing motion solution based on the target motion solution constraints to generate the target robot control data corresponding to this control process; and controlling the robot's motion based on the target robot control data. When the first marker is available, the robot still has the key input conditions to perform basic motion solutions; when the second marker is unavailable, the robot continues to perform limited teleoperation control even when the first marker is available.

11. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program that, when executed by the processor, implements the steps of the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images