Robot cloud reasoning time delay compensation method and system for safe and stable execution

Abstract

The application discloses a kind of robot cloud reasoning time delay compensation method and system for security and stable execution, it is related to the robot control field under the cloud edge collaborative architecture.Method includes: maintaining to-be-executed action sequence;According to current effective time delay, time delay sensing switching is carried out between basic control strategy and compensation control strategy;Wherein, the compensation control strategy generates action sequence with current observation information and the to-be-executed action sequence as input, and the basic control strategy generates action sequence with current observation information as input;The action sequence returned by cloud is expired and is cut off, and the remaining effective action is continuously connected with the remaining action in the to-be-executed action sequence which has not been executed, and is issued for execution.The application significantly reduces the risk of missing and missing in dynamic grabbing, conveying belt sorting and other scenes, and is suitable for industrial robots and remote reasoning control scene.

Description

Technical Field

[0001] This application relates to the field of robot control under a cloud-edge collaborative architecture, specifically to a method and system for compensating for robot cloud inference latency for safe and stable execution. Background Technology

[0002] With the increasing application of imitation learning, generative policies, and visual motion control models in robot operation, the rapid growth in the number of model parameters has made it difficult for local inference devices to handle the ever-increasing computational demands. Robot control systems are gradually evolving from traditional local closed-loop control to a cloud-edge collaborative architecture of "cloud inference, local execution." In this architecture, the robot body is responsible for collecting visual information, joint states, and end effector states, and sending the observed data to a cloud server. The cloud server runs the policy model to generate action sequences, which are then distributed to the local controller for execution. This architecture reduces the computational burden on the robot body, facilitates the deployment of large-parameter policy models, and allows for the sharing of policy capabilities among multiple robots. It is particularly suitable for industrial scenarios where edge computing power is limited and the model size is too large to be directly deployed to the local controller.

[0003] However, in a cloud-edge collaborative control architecture, the system inevitably introduces latency in sensor upload, network transmission, cloud inference, and command feedback. These latencies not only have fixed components but also fluctuate with network load, server computing load, and task concurrency, resulting in latency jitter and asynchronous updates. For tasks requiring continuous action output, such as robotic arm grasping, conveyor belt sorting, and dynamic target manipulation, the observations used for action generation often lag behind the robot's and environment's states at the actual moment of action execution, leading to a time misalignment between observation and action execution.

[0004] In practical deployments, if the system cannot effectively handle the aforementioned time misalignment, the robot will continue executing the remaining actions in the old action plan before the new action sequence returns. In this case, the newly returned action sequence from the cloud may be discontinuous with the locally executing action state, leading to issues such as sudden action changes, path deviations, grasping failures, collisions, missed grasps, or even triggering a safety shutdown. Especially in scenarios involving dynamic transport, moving target grasping, or human-robot collaboration, the execution mismatch caused by latency directly impacts system safety and stability.

[0005] In the existing technology, the closest technical solutions to this invention are mainly concentrated in the field of cloud-edge collaborative robot systems. Related research typically unfolds from the following key aspects: first, the system deployment method, i.e., the architecture design of cloud inference and local execution; second, the action execution mechanism, including synchronous or asynchronous execution of action sequences; and finally, the compensation and control methods adopted for communication latency and inference latency during system operation. Based on the above classification, the closest existing technologies to this invention can be mainly summarized into the following categories: cloud-based inference and local execution deployment methods, asynchronous action execution methods, and latency compensation control methods, etc.

[0006] The first type is the synchronous execution scheme: the local robot waits for the latest inference result from the cloud before executing each action, and then begins to execute the next action. This scheme can avoid serious misalignment between actions and observations to some extent, but it will increase the robot's waiting time and cause obvious pauses in actions. In dynamic scenarios, it often fails to complete the task due to insufficient response speed.

[0007] The second type is the ordinary asynchronous execution scheme: while waiting for a new action sequence from the cloud, the robot continues to execute the remaining part of the previous action plan. When the new action returns, the expired action is directly truncated and the robot switches to the new action sequence. Although this scheme improves execution continuity and throughput efficiency, it does not explicitly consider the "state of actions that the robot has already committed to execute," which makes it prone to discontinuities during action switching, leading to control oscillations and safety risks.

[0008] The third category is a single delay compensation scheme: This type of scheme attempts to use a single control model to uniformly adapt to both low-latency and high-latency scenarios, or to handle latency issues by using methods such as state history stacking, fixed-length delay modeling, and explicit dynamic prediction. However, this type of scheme either relies on strong system model assumptions or struggles to simultaneously ensure control performance in low-latency scenarios and safety robustness in high-latency scenarios, thus limiting its applicability in engineering deployments.

[0009] Therefore, there is an urgent need for a robot latency adaptive control scheme for stable execution, which can handle latency and latency jitter in cloud-edge collaborative scenarios without significantly increasing engineering complexity, and reduce problems such as action mismatch, false capture, missed capture, trajectory deviation and reduced task success rate caused by outdated observation and discontinuous action switching. Summary of the Invention

[0010] This application provides a method and system for compensating for latency in robot cloud inference for safe and stable execution. It can solve the technical problems in the existing cloud-edge collaborative architecture, such as the misalignment of observation and execution time caused by latency, the discontinuity of new and old actions, and the inability of a single strategy to simultaneously achieve low latency performance and high latency security robustness.

[0011] In a first aspect, this application provides a method for compensating for latency in robot cloud inference for secure and stable execution, comprising the following steps: Maintain a sequence of actions to be executed, which represents the actions that the local controller is expected to continue to execute while waiting for a new action sequence to be returned from the cloud; The system performs latency-aware switching between the basic control strategy and the compensation control strategy based on the current effective latency; wherein, the compensation control strategy generates an action sequence with the current observation information and the action sequence to be executed as input, and the basic control strategy generates an action sequence with the current observation information as input. The action sequence returned from the cloud is truncated due to expiration, and the remaining valid actions are continuously connected with the remaining actions in the action sequence to be executed before being sent out for execution.

[0012] Furthermore, before performing the delay-aware switching between the basic control strategy and the compensation control strategy based on the current effective delay, the following steps are also included: During the system initialization phase, probe data is sent to the cloud server through the local controller, and the receiving time of action feedback is recorded, thereby obtaining multiple sets of end-to-end round-trip latency samples. The average round-trip time is calculated based on the delay sample, and the average delay steps are calculated. The average delay steps are used to determine the preset length and action truncation position of the action sequence to be executed.

[0013] Furthermore, the delay-aware switching between the basic control strategy and the compensation control strategy based on the current effective delay specifically includes: A first switching threshold and a second switching threshold are set, wherein the second switching threshold is greater than the first switching threshold, to form a hysteresis interval; When the number of delay steps corresponding to the current effective delay is less than the first switching threshold, the basic control strategy is selected; When the number of delay steps is greater than the second switching threshold, the compensation control strategy is selected; When the delay step number is between the first switching threshold and the second switching threshold, the strategy mode of the previous control cycle remains unchanged.

[0014] Furthermore, the step of truncating expired action sequences returned from the cloud and then continuously connecting the remaining valid actions with the remaining actions in the action sequence to be executed before sending them down for execution specifically includes the following steps: Based on the current time and corresponding delay, remove the previous actions in the action sequence that have become outdated due to observation lag, and only retain the action segments that are still valid after the current time as executable action sequences. The remaining actions that have not yet been completed in the executable action sequence and the action sequence to be executed are time-aligned and issued step by step according to the control cycle.

[0015] Furthermore, the training process of the compensation control strategy includes: Extract a motion segment of preset length from the time-delay-free teaching trajectory, and extract the corresponding preceding motion steps as the action sequence to be executed according to the set delay steps, while retaining the complete motion segment as the target action sequence; The sequence of actions to be executed is converted into a fixed-length delay feature, which is then uniformly adjusted to the preset maximum delay length by padding with zeros, and a validity flag is added to distinguish between expired actions and valid actions. The loss is calculated using the truncated effective action segments as supervision signals, and the model is trained to learn the action generation relationship under time delay conditions.

[0016] Furthermore, it also includes the following steps: During the execution of the action, the output of the safety monitoring module is used to detect collision risks, boundary crossing risks, or capture failures. When an anomaly is detected, the action is terminated, switched to conservative mode, or a safety shutdown is performed.

[0017] Secondly, this application provides a robot cloud inference latency compensation control system for safe and stable execution, comprising: The sensor acquisition module is used to acquire environmental images and robot body status information; The cloud-based strategy reasoning module is deployed on a cloud server and is configured with basic control strategies and compensation control strategies. It is used to generate candidate action sequences based on the received data. A local controller, deployed on the robot body, is used to implement the method described above; the local controller includes a sequence of actions to be executed unit, a latency monitoring unit, a strategy switching unit, and an action connection unit. The communication module is used to realize data transmission between the sensing acquisition module and the cloud-based strategy reasoning module; An actuator is used to drive the robot to move according to the action instructions issued by the local controller.

[0018] Furthermore, the compensation control strategy in the cloud-based strategy inference module is obtained by efficiently adapting parameters based on the basic control strategy model; The adaptation method includes injecting low-rank trainable parameters into selected linear layers of the base model, keeping the main parameters of the base model frozen, and only updating the low-rank adaptation parameters and the conditional coding modules related to time delay compensation.

[0019] Furthermore, the local controller is also used to send new observation data to the cloud in advance to request the next round of action sequence when the average delay corresponds to a number of action steps, before the current action sequence has any remaining actions corresponding to the number of delay steps that have not been completed. This ensures that the cloud has enough time to process and return the new action sequence, and avoids the robot from stalling due to waiting.

[0020] Furthermore, it also includes a safety monitoring module for detecting abnormal end force, abnormal gripper closure, or trajectory deviation; The security monitoring module is connected to the local controller for feedback. When a security shutdown condition is triggered, the local controller interrupts the currently executing sequence of actions to be executed and the sequence of actions sent from the cloud.

[0021] The beneficial effects of the technical solutions provided in this application include at least the following: By maintaining the sequence of actions to be executed and explicitly modeling the committed execution state, the problem of action mismatch caused by outdated observations is solved. Through a latency-aware switching mechanism, basic control strategies are enabled to maintain efficiency in low-latency scenarios, while compensation control strategies are enabled to improve the continuity and consistency of new and old actions in high-latency scenarios. By truncating and continuously connecting the action sequences returned from the cloud, action jumps and switching oscillations are avoided. Ultimately, in static grasping and dynamic conveyor belt sorting scenarios, misgrabbing, missed grasping, and task failures are reduced, improving the system's execution stability and task success rate.

[0022] By maintaining the sequence of actions to be executed, the robot explicitly models the execution state promised by the robot before the new action in the cloud returns, effectively solving the problem of action mismatch caused by outdated observations; Through the latency-aware switching mechanism, the basic control strategy is enabled in low-latency scenarios to maintain the original control efficiency, and the compensation control strategy is enabled in high-latency scenarios. New actions are generated with the sequence of actions to be executed as input, so that the cloud can perceive the current execution status, which significantly improves the continuity and consistency of new and old actions. By truncating expired action sequences returned from the cloud and continuously connecting the remaining valid actions with the remaining actions that have not yet been completed in the action sequence to be executed, action jumps and switching oscillations are avoided. Ultimately, this reduced mis-grabbing, missed grabbing, and task failures in scenarios such as static grasping and dynamic conveyor belt sorting, thereby improving the overall execution stability and task success rate of the system. Attached Figure Description

[0023] Figure 1 A flowchart of a robot cloud inference latency compensation method for secure and stable execution provided in an embodiment of this application; Figure 2The diagram shows the latency-aware switching and hysteresis control logic in the robot cloud inference latency compensation method for safe and stable execution provided in the embodiments of this application. Figure 3 A schematic diagram illustrating a secure execution application scenario for the robot cloud inference latency compensation method for secure and stable execution provided in this application embodiment; Figure 4 A schematic diagram of the action-to-be-executed cache and continuous execution timing in the robot cloud inference latency compensation method for safe and stable execution provided in the embodiments of this application; Figure 5 This is an overall flowchart of a robot cloud inference latency compensation method for safe and stable execution provided in the embodiments of this application; Figure 6 This is a diagram illustrating the architecture of a robot cloud-based inference latency compensation system for secure and stable execution, provided in an embodiment of this application. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0026] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0027] DDIM: Denoising Diffusion Implicit Models; LMS: Linear Multistep Methods; DDPM: Denoising Diffusion Probabilistic Models; VP-SDE: Variance Preserving Stochastic Differential Equation; Firstly, such as Figure 1As shown, this application provides a method for compensating for robot cloud inference latency for safe and stable execution, including the following steps: Step S1: Maintain the sequence of actions to be executed. The sequence of actions to be executed represents the actions that the local controller is expected to continue to execute while waiting for the cloud to return a new sequence of actions. Step S2: Switch between the basic control strategy and the compensation control strategy based on the current effective delay; wherein, the compensation control strategy generates an action sequence with the current observation information and the action sequence to be executed as input, and the basic control strategy generates an action sequence with the current observation information as input. Step S3: Truncate the expired action sequence returned from the cloud, and then continuously connect the remaining valid actions with the remaining actions in the action sequence to be executed before sending them out for execution.

[0028] This embodiment effectively solves the problem of action mismatch caused by outdated observations by maintaining the sequence of actions to be executed and explicitly modeling the execution state that the robot has promised before the new action in the cloud returns. Through the latency-aware switching mechanism, the basic control strategy is enabled in low-latency scenarios to maintain the original control efficiency, and the compensation control strategy is enabled in high-latency scenarios. New actions are generated with the sequence of actions to be executed as input, so that the cloud can perceive the current execution status, which significantly improves the continuity and consistency of new and old actions. By truncating expired action sequences returned from the cloud and continuously connecting the remaining valid actions with the remaining actions that have not yet been completed in the action sequence to be executed, action jumps and switching oscillations are avoided. Ultimately, this reduced mis-grabbing, missed grabbing, and task failures in scenarios such as static grasping and dynamic conveyor belt sorting, thereby improving the overall execution stability and task success rate of the system.

[0029] In one embodiment, before maintaining the sequence of actions to be executed, which represents the actions that the local controller is expected to continue executing while waiting for a new action sequence to be returned from the cloud, step S1 further includes the following steps: Collect observation information at the current time or the most recently available time; further, including the following steps: Step S01: In response to the trigger signal of the robot control cycle, the local controller collects environmental images, robot joint states, end effector states, pose information and task-related state information through the sensor acquisition module, and generates multimodal observation data covering the current moment or the most recently available moment.

[0030] Step S02: The local controller sends the collected observation information to the cloud-based policy inference module through the communication module, providing input for the generation of subsequent action sequences.

[0031] Specifically, the observation information includes, but is not limited to, environmental images, joint positions, joint velocities, end-effector poses, gripper opening and closing states, target object states, and task context information.

[0032] This embodiment ensures that the cloud-based policy reasoning module can obtain the latest state of the robot and its environment at the current moment through parallel acquisition, structured encapsulation, and reliable transmission of multimodal sensor data. This provides an accurate data foundation for subsequent time delay estimation, policy switching, and action generation, effectively avoiding control deviations caused by missing or misaligned observation data.

[0033] In one embodiment, step S1: Maintaining a sequence of actions to be executed, which represents the actions that the local controller is expected to continue executing while waiting for a new action sequence to be returned from the cloud, specifically includes the following steps: Step S11: Generate a sequence of actions to be executed based on the action plan that was not fully executed in the previous round, and store it in the action sequence cache; more specifically, obtain the action plan that was received in the previous round but not fully executed, form a sequence of actions to be executed through caching operations, and output the sequence of actions to be executed that represents the content that the robot is expected to execute before the new action returns from the cloud.

[0034] The sequence of actions to be executed may include joint targets, end pose targets, velocity commands, or gripper control commands for several future control cycles.

[0035] Step S12: Based on the pre-measured average delay, the number of delay steps is calculated. When the number of unexecuted steps in the sequence of actions to be executed matches the number of delay steps, a new observation data request is sent to the cloud to pre-fetch observation data in order to avoid the robot from pausing while waiting for a response from the cloud and to maintain the continuity of action execution. The formula for calculating the average latency is as follows: ; ; Among them, The average time delay is N, where N is the number of measurement rounds. The time for sending observations in the i-th round is... Let i be the time for receiving actions in the i-th round. This refers to the duration of a single step. This represents the delay step.

[0036] The observation data consists of environmental images, joint positions, joint velocities, end-effector poses, gripper opening and closing states, and task-related status information acquired by the local sensor acquisition module. These data are sent to the cloud-based strategy reasoning module to generate action sequences.

[0037] This embodiment solves the problem of action mismatch caused by outdated observations by maintaining the sequence of actions to be executed, explicitly modeling the robot's committed execution state, and combining it with a prefetching observation mechanism.

[0038] In one embodiment, step S2: performing a delay-aware switching between the basic control strategy and the compensation control strategy based on the current effective delay; wherein, the compensation control strategy generates an action sequence with the current observation information and the sequence of actions to be executed as input, and the basic control strategy generates an action sequence with the current observation information as input, specifically including the following steps: Step S21: Collect end-to-end latency data, cloud inference time, and command transmission time, and obtain the current effective latency through statistical analysis; Specifically, before the formal movement, the system first performs several rounds of time delay measurement. The local controller sends the observation data to the cloud and records the return time of the action result to obtain the end-to-end round-trip time delay of multiple rounds. Then, the average value of the multiple rounds of time delay is taken, and then divided by the single step duration corresponding to the action control frequency to convert it into the average number of delay steps and round it down. The calculation method of the average number of delay steps is as shown in step S12.

[0039] Step S22: Based on the current effective delay, a hysteresis interval is constructed by the first switching threshold and the second switching threshold. The interval in which the delay is located is determined to generate a policy switching decision, wherein the second switching threshold is greater than the first switching threshold. like Figure 2 As shown, the strategy switching decision is generated based on the interval in which the latency is located: When the number of delay steps corresponding to the current effective delay is less than the first switching threshold, the basic control strategy is selected; When the delay step count exceeds the second switching threshold, a compensation control strategy is selected. When the delay steps are between two thresholds, the strategy mode of the previous control cycle remains unchanged to avoid frequent switching caused by latency fluctuations. The strategy mode... The selection can be based on the current effective delay. With the first switching threshold and the second switching threshold The size relationship is used to determine this, as follows: ; in, This represents the strategy mode to be executed in the t-th control cycle. Indicates the basic control strategy. This indicates a compensation control strategy. This represents the current effective delay in the t-th control cycle. Indicates the first switching threshold. Indicates the second switching threshold, and > .

[0040] Step S23: Combining the current observation information with the sequence of actions to be executed, and based on the strategy switching decision, call the corresponding control strategy to generate a candidate sequence of actions to be issued; Specifically, if the basic control strategy is selected The cloud-based strategy reasoning module then relies solely on the currently observed information. Generate basic candidate action sequences The corresponding formula is: ; If a compensation control strategy is selected The cloud-based strategy reasoning module will then use the current observed information. In addition, receive the sequence of actions to be executed. Alternatively, the constructed action state features can be used as additional input to generate compensation candidate action sequences. The corresponding formula is: ; This design ensures that new actions are time-sequential with the current execution state.

[0041] This embodiment maintains the original control efficiency in low-latency scenarios and enhances safety robustness in high-latency scenarios through a latency-aware switching mechanism and a dual-strategy division of labor design. It avoids the performance trade-off of a single strategy and improves the continuity and execution stability of new and old actions.

[0042] In one embodiment, to address the action mismatch problem caused by network latency and ensure stable robot operation in the aforementioned complex scenarios, step S3: expired action sequences returned from the cloud are truncated, and the remaining valid actions are continuously linked with the remaining actions in the action sequence to be executed before being sent for execution. Stable linking of action sequences is achieved through the following sub-steps: like Figure 3 As shown in the diagram, this embodiment illustrates two typical robot operation tasks to demonstrate the practical significance of continuous action connection in this step: in, Figure 3 (a) shows a diagram illustrating the robot's task of picking up green peppers from a table and placing them in a basket in a static, cluttered environment. Figure 3 (b) shows a diagram illustrating the robot's task of tracking and grabbing a moving green pepper on a conveyor belt in a dynamic environment.

[0043] The above scenarios require a high degree of continuity in actions. If the transition between old and new actions is not continuous, it can easily lead to mis-capture, missed capture, or trajectory deviation.

[0044] Step S31: Based on the action sequence returned from the cloud, the current time, and the current valid delay, calculate the truncation position, determine the boundary between expired action segments and valid action segments, and use it as an index identifier for subsequent truncation operations; Specifically, the truncation position is calculated based on the delay step d, which is obtained by monitoring in step S2. Valid action segment The formula for returning the portion of the action sequence starting from the (d+1)th action in the cloud is as follows: ; in, Indicates the first The action executed in the t-th control cycle belongs to the action sequence generated in the t-th control cycle, and H is the total length of the action sequence.

[0045] Step S32: Based on the determined boundary between expired action segments and valid action segments, the remaining valid action sequence is obtained by removing the previous expired actions; specifically, the truncation operation removes the expired previous actions in the action sequence returned from the cloud according to the current time and the corresponding delay, and only retains the action segments that are still valid after the current time as the remaining valid action sequence.

[0046] Step S33: As Figure 4 As shown, by aligning the remaining valid action sequences with the remaining actions that have not yet been executed in the action sequences to be executed on the time axis, an executable continuous action sequence is generated by merging them. Specifically, during the connection process, the control cycle is used as a reference. The remaining valid action sequence is aligned with the remaining actions that have not yet been completed in the action sequence to be executed on the time axis to ensure that the intervals between adjacent actions are consistent and to avoid jumps or offsets. Figure 4 The upper part shows the state update logic of the action buffer to be executed: each control cycle saves the remaining actions that have not been executed to maintain the continuity of the task context; the lower part shows the alignment process of the continuous execution sequence: the newly generated remaining valid action sequence is spliced ​​with the old actions in the buffer on a unified time axis, and the phase difference caused by the delay is eliminated by overlapping windows or timestamp calibration to achieve seamless connection.

[0047] Step S34: Based on the continuous sequence of executable actions, drive the actuator by gradually pushing according to the control cycle to output the action command on the robot body side; Specifically, the executable action sequence is progressively sent to the actuators according to the control cycle, driving the robotic arm, gripper, etc., to complete the target task, while updating the action sequence to be executed to reflect the latest execution status. During the execution of the action, the local controller, in conjunction with the output of the safety monitoring module, detects abnormal end force, abnormal gripper closure, trajectory deviation, collision risk, boundary risk, or grasping failure. When an abnormality is detected, it triggers action abort, switches to conservative mode, or executes a safety shutdown, realizing a safety closed-loop update.

[0048] This embodiment eliminates the discontinuity problem when switching between old and new actions by using an expired truncation and continuous connection mechanism, avoids action jumps and trajectory deviations, improves the stability of the execution process, and reduces the risk of false captures, missed captures, and task failures caused by outdated observations.

[0049] like Figure 5 The diagram illustrates the overall hardware architecture and closed-loop control logic of the cloud-based robot scheduling system, as detailed below: The left-hand frame depicts the interaction between the cloud and the local machine: the cloud deploys a "latency-adaptive switching diffusion strategy" module, which sends a "continuous action sequence" to the local machine via a "communication module". On the local end, the task is performed by the "local robotic arm," whose status is divided into "currently executing" and "historical actions." And the current action "pending execution". The robot body collects "observations" in real time. This information is then transmitted back to the cloud-based strategy module as a feedback signal, forming a complete closed loop of "perception-decision-execution".

[0050] The middle section clarifies the system's multimodal input sources. The upper input is "latency steps," used to quantify the current network and computation latency; the lower input is "visual and torque" data, specifically represented as scene images containing the operational target. The core module on the right demonstrates the policy switching logic: the system switches based on the "latency steps." (Basic Strategy) and (Delay Steps) The judgment condition of "(compensation strategy)" is used to dynamically select the control mode through the branch processing module (Switch) to achieve adaptive response to different time delay environments.

[0051] The lower right corner of the diagram reveals in detail the internal generation mechanism of the "delay-adaptive switching diffusion strategy". The strategy network receives "observation noise" and "normal control" signals, combines them with the state of the "action to be executed", and applies the conditional probability formula. Generate predicted actions; where, This is a strategy model controlled by parameter θ. The purple area at the bottom lists four core algorithm kernels: DDIM, LMS, DDPM, and VP-SDE. This indicates that the strategy integrates the denoising and sampling capabilities of multiple diffusion models, ultimately outputting accurate "model prediction" results, which are then sent to the robotic arm to complete the operation.

[0052] Secondly, such as Figure 6 As shown, this application provides a robot cloud inference latency compensation system for safe and stable execution, including: The sensor acquisition module is used to acquire environmental images and robot body status information; The cloud-based strategy reasoning module is deployed on a cloud server and is configured with basic control strategies and compensation control strategies. It is used to generate candidate action sequences based on the received data. A local controller, deployed on the robot body, is used to implement the above method; the local controller includes a sequence of actions to be executed unit, a latency monitoring unit, a strategy switching unit, and an action connection unit; The communication module is used to realize data transmission between the sensor acquisition module and the cloud-based strategy inference module. The actuator is used to drive the robot to move according to the motion instructions issued by the local controller.

[0053] In one embodiment, the compensation control strategy in the cloud-based strategy inference module is obtained by efficiently adapting parameters based on the basic control strategy model; The adaptation method involves injecting low-rank trainable parameters into selected linear layers of the base model, keeping the main parameters of the base model frozen, and only updating the low-rank adaptation parameters and the conditional coding modules related to time delay compensation.

[0054] In one embodiment, the local controller is further configured to send new observation data to the cloud in advance to request the next round of action sequence when the average delay corresponds to a number of action steps and there are still actions in the current action sequence that are not completed before the number of delay steps is finished.

[0055] In one embodiment, a safety monitoring module is also included to detect abnormal end force, abnormal gripper closure, or trajectory deviation; The security monitoring module is connected to the local controller for feedback. When a security shutdown condition is triggered, the local controller interrupts the currently executing sequence of actions to be executed and the sequence of actions sent from the cloud.

[0056] The functions of each module in the above-mentioned robot cloud inference latency compensation system for safe and stable execution correspond to the steps in the above-mentioned robot cloud inference latency compensation method embodiment for safe and stable execution. Their functions and implementation processes will not be described in detail here.

[0057] Thirdly, embodiments of this application also provide a readable storage medium.

[0058] The present application stores a robot cloud inference latency compensation program for safe and stable execution on a readable storage medium, wherein when the robot cloud inference latency compensation program for safe and stable execution is executed by a processor, it implements the steps of the robot cloud inference latency compensation method for safe and stable execution as described above.

[0059] The method implemented when the robot cloud inference latency compensation program for safe and stable execution is executed can be referred to in the various embodiments of the robot cloud inference latency compensation method for safe and stable execution in this application, and will not be repeated here.

[0060] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0061] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods of the various embodiments of this application.

[0062] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for compensating for latency in robot cloud-based inference for safe and stable execution, characterized in that, Includes the following steps: Maintain a sequence of actions to be executed, which represents the actions that the local controller is expected to continue to execute while waiting for a new action sequence to be returned from the cloud; Based on the current effective latency, perform latency-aware switching between the basic control strategy and the compensation control strategy; The specific steps of switching between the basic control strategy and the compensation control strategy based on the current effective delay include: A first switching threshold and a second switching threshold are set, wherein the second switching threshold is greater than the first switching threshold, to form a hysteresis interval; When the number of delay steps corresponding to the current effective delay is less than the first switching threshold, the basic control strategy is selected; When the number of delay steps is greater than the second switching threshold, the compensation control strategy is selected; When the number of delay steps is between the first switching threshold and the second switching threshold, the strategy mode of the previous control cycle remains unchanged; The compensation control strategy generates an action sequence by taking the current observation information and the action sequence to be executed as inputs, while the basic control strategy generates an action sequence by taking the current observation information as inputs. The training process of the compensation control strategy includes: Extract a motion segment of preset length from the time-delay-free teaching trajectory, and extract the corresponding preceding motion steps as the action sequence to be executed according to the set delay steps, while retaining the complete motion segment as the target action sequence; The sequence of actions to be executed is converted into a fixed-length delay feature, which is then uniformly adjusted to the preset maximum delay length by padding with zeros, and a validity flag is added to distinguish between expired actions and valid actions. The loss is calculated using the truncated effective action segments as supervision signals, and the model is trained to learn the action generation relationship under time delay conditions. The compensation control strategy is obtained by adapting parameters based on the basic control strategy model; The adaptation method includes injecting low-rank trainable parameters into selected linear layers of the base model, keeping the main parameters of the base model frozen, and only updating the low-rank adaptation parameters and the conditional coding module related to time delay compensation. The action sequence returned from the cloud is truncated due to expiration, and the remaining valid actions are continuously connected with the remaining actions in the action sequence to be executed before being sent out for execution.

2. The robot cloud inference latency compensation method for safe and stable execution according to claim 1, characterized in that, Before performing the delay-aware switching between the basic control strategy and the compensation control strategy based on the current effective delay, the following steps are also included: During the system initialization phase, probe data is sent to the cloud server through the local controller, and the receiving time of action feedback is recorded, thereby obtaining multiple sets of end-to-end round-trip latency samples. The average round-trip time is calculated based on the delay sample, and the average delay steps are calculated. The average delay steps are used to determine the preset length and action truncation position of the action sequence to be executed.

3. The robot cloud inference latency compensation method for safe and stable execution according to claim 1, characterized in that, The process of truncating expired action sequences returned from the cloud and then continuously connecting the remaining valid actions with the remaining actions in the action sequence to be executed before sending them down for execution includes the following steps: Based on the current time and corresponding delay, remove the previous actions in the action sequence that have become outdated due to observation lag, and only retain the action segments that are still valid after the current time as executable action sequences. The remaining actions that have not yet been completed in the executable action sequence and the action sequence to be executed are time-aligned and issued step by step according to the control cycle.

4. The robot cloud inference latency compensation method for safe and stable execution according to claim 1, characterized in that, It also includes the following steps: During the execution of the action, the output of the safety monitoring module is used to detect collision risks, boundary crossing risks, or capture failures. When an anomaly is detected, the action is terminated, switched to conservative mode, or a safety shutdown is performed.

5. A robot cloud-based inference latency compensation control system for safe and stable execution, characterized in that, include: The sensor acquisition module is used to acquire environmental images and robot body status information; The cloud-based strategy reasoning module is deployed on a cloud server and is configured with basic control strategies and compensation control strategies. It is used to generate candidate action sequences based on the received data. A local controller, deployed on the robot body, is used to implement the robot cloud inference latency compensation control method for safe and stable execution as described in any one of claims 1 to 4; the local controller includes a sequence of actions to be executed, a latency monitoring unit, a strategy switching unit, and an action connection unit. The communication module is used to realize data transmission between the sensing acquisition module and the cloud-based strategy reasoning module; An actuator is used to drive the robot to move according to the action instructions issued by the local controller.

6. The robot cloud-based inference delay compensation control system according to claim 5, characterized in that, The local controller is also used to send new observation data to the cloud in advance to request the next round of action sequence when the average delay corresponds to a number of action steps and there are still actions in the current action sequence that are not completed before the number of delay steps is finished.

7. The robot cloud-based inference delay compensation control system according to claim 5, characterized in that, It also includes a safety monitoring module for detecting abnormal end force, abnormal gripper closure, or trajectory deviation; The security monitoring module is connected to the local controller for feedback. When a security shutdown condition is triggered, the local controller interrupts the currently executing sequence of actions to be executed and the sequence of actions sent from the cloud.

Images