Rolling closed-loop diffusion policy control method and system for robotic dexterous manipulation

Abstract

The application relates to the technical field of robot control, and discloses a rolling closed-loop diffusion strategy control method and system for robot dexterous operation. The method comprises the following steps: acquiring an initial action sequence, and performing conditional denoising through a denoising network to extract an initial action window; in each control cycle, observation data containing the latest environment feedback is acquired, the observation data is input into the denoising network, denoising iteration is performed on the current action window, noise-free actions are obtained, and the noise-free actions are sent to a robot for execution; the remaining noisy action sequence after the robot executes is shifted forward by one time step as a whole, a new random noise sample is added at the end of the window, and an action window for the next control cycle is formed; the application not only eliminates the dependence on a fixed action block size, significantly improves the calculation efficiency and the smoothness of an action trajectory, but also greatly enhances the instantaneous closed-loop response capability of the robot to environment changes, and effectively improves the success rate of complex tasks.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and specifically to a rolling closed-loop diffusion strategy control method and system for dexterous robot operation. Background Technology

[0002] In the field of robot manipulation skill learning, with the rapid development of generative artificial intelligence, diffusion policies have shown great potential to integrate multimodal information (such as images, point clouds, and text) and uniformly output actions in various tasks (such as pick-and-place and articulated object interaction). However, robot control tasks are fundamentally different from traditional image or text generation tasks. Robot control not only requires the continuity and smoothness of action output, but also places extremely stringent demands on real-time feedback and low-latency inference. Currently, most mainstream diffusion policy-based control frameworks adopt an action chunking mechanism. Under this mechanism, the model generates a fixed-length sequence of actions (i.e., action chunks) in batches based on a continuous historical observation data.

[0003] This existing action block paradigm has significant limitations: First, it lacks real-time action-level feedback. Throughout the entire execution cycle of action block generation, the model only utilizes observation data at the starting point of action block generation, failing to continuously integrate the latest observation data from environmental feedback during subsequent actions. This results in poor robustness to environmental disturbances and slow response of the robot. Second, the system is computationally inefficient and prone to motion discontinuities. The inherent multi-step denoising process of the diffusion model is a significant computational bottleneck. Generating an action block requires numerous iterative denoising steps, leading to high inference latency and failing to meet the demands of high-frequency control. Furthermore, the boundaries between adjacent action blocks often exhibit motion discontinuities, easily causing abrupt changes in robot joint velocity or acceleration, posing safety hazards, especially in dexterity operations where stability and safety are paramount (such as pouring liquids). To address these issues, while existing technologies have attempted to introduce auxiliary networks or use consistency models for acceleration, they have not fundamentally solved the challenge of integrating the latest perceptual feedback in real-time and efficiently during the diffusion generation process. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a rolling closed-loop diffusion strategy control method and system for dexterous robot operation.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In a first aspect, the present invention provides a rolling closed-loop diffusion strategy control method for dexterous robot manipulation, comprising: S1. Obtain the initial action sequence of the robot composed of pure Gaussian noise, and perform conditional denoising on the initial action sequence through a pre-trained denoising network to extract the initial action window with a progressive noise structure. The progressive noise structure refers to the action noise level near the current time being lower than the action noise level far away from the current time within the action window. S2, in each control cycle, acquires observation data containing the latest environmental feedback, inputs the observation data as a condition to the denoising network, performs denoising iteration on the current action window, obtains the noise-free action at the current moment, and sends it to the robot for execution; the observation data includes the robot's body state and point cloud data of the operation scene; S3, shift the remaining noisy action sequence after the robot has executed forward by one time step, and add a new random noise sample at the end of the window to form the action window for the next control cycle; S4. Repeat steps S2 and S3 until the task is completed.

[0006] In one embodiment, extracting the initial action window with a progressive noise structure specifically includes: Set time Noise level Action window below for: ; in, This represents the noise level increment between consecutive actions within the action window; T represents the maximum noise level; and H represents the fixed field of view length for action prediction. Indicates at time and noise level The generated d-dimensional action vector; Indicates definition, Represents the set of real numbers; within the initial action window, the closer to the current time... The action is assigned a lower noise level.

[0007] In one embodiment, the step of acquiring observation data containing the latest environmental feedback, inputting the observation data as a condition into a denoising network, performing denoising iterations on the current action window to obtain the noise-free action at the current moment, and sending it to the robot for execution, specifically includes: Based on a given time Length is Observational data sequence Using a denoising network, for time... Noise level is Action window below After a step size of After the denoising iteration, the overall noise level of all actions within the action window is reduced. : ; For the denoised action window, Indicates at time and noise level The generated action vector, where H represents the fixed field of view length for action prediction; express Time-lapse observation data; noise-reduced action window Includes Partially denoised action vectors, and the first fully denoised action vector. .

[0008] In one embodiment, the elements of the motion vector include one or more of the following: robot joint position control, joint velocity control, end effector pose increment, end effector velocity control, and end effector opening / closing control (e.g., opening / closing control of a gripper or dexterous hand actuator).

[0009] In one embodiment, the denoising network is implemented based on conditional denoising score matching and inverse stochastic differential equations; Conditional denoising score matching: The denoising network uses the following parameters: One-dimensional conditional U-Net network Achieve parameterized approximation of observed data sequences and noise level Conditional fractional function The conditional fractional function is used to characterize the log probability density gradient of the action window under conditional observation data, that is: ; Indicates the action window, Indicates to Find the partial derivative; The training process of the denoising network uses Fisher divergence as the loss function. : ; This represents the expectation for the training data, noise level, and random noise. This represents the observation-action distribution in the teaching data, used in imitation learning. Approximates the true conditional fractional function. For gradient operators; In practical reasoning, the denoising network will include the latest observation data sequence of environmental feedback. As a conditional input, inverse stochastic differential equation evolution is performed to update the action window in real time and track the target action distribution.

[0010] In a second aspect, the present invention provides a rolling closed-loop diffusion strategy control system for dexterous robot operation, comprising: The noise module is initialized to obtain the robot's initial action sequence composed of pure Gaussian noise. The initial action sequence is then conditionally denoised using a pre-trained denoising network to extract an initial action window with a progressive noise structure. The progressive noise structure refers to the action noise level closer to the current moment within the action window being lower than the action noise level further away from the current moment. The rolling denoising module acquires observation data containing the latest environmental feedback in each control cycle, inputs the observation data as a condition into the denoising network, performs denoising iteration on the current action window, obtains the noise-free action at the current moment, and sends it to the robot for execution; the observation data includes the robot's body state and point cloud data of the operation scene; The window update sliding module shifts the remaining noisy action sequence after the robot has executed forward by one time step, and adds a new random noise sample at the end of the window to form the action window for the next control cycle. Repeatedly scroll the noise reduction module and the window update slider module until the task is completed.

[0011] In one embodiment, extracting the initial action window with a progressive noise structure specifically includes: Set time Noise level Action window below for: ; in, This represents the noise level increment between consecutive actions within the action window; T represents the maximum noise level; and H represents the fixed field of view length for action prediction. Indicates at time and noise level The generated d-dimensional action vector; Indicates definition, Represents the set of real numbers; within the initial action window, the closer to the current time... The action is assigned a lower noise level.

[0012] In one embodiment, the step of acquiring observation data containing the latest environmental feedback, inputting the observation data as a condition into a denoising network, performing denoising iterations on the current action window to obtain the noise-free action at the current moment, and sending it to the robot for execution, specifically includes: Based on a given time Length is Observational data sequence Using a denoising network, for time... Noise level is Action window below After a step size of After the denoising iteration, the overall noise level of all actions within the action window is reduced. : ; For the denoised action window, Indicates at time and noise level The generated action vector, where H represents the fixed field of view length for action prediction; express Time-lapse observation data; noise-reduced action window Includes Partially denoised action vectors, and the first fully denoised action vector. .

[0013] In one embodiment, the elements of the motion vector include one or more of the following: robot joint position control, joint velocity control, end effector pose increment, end effector velocity control, and end effector opening / closing control.

[0014] In one embodiment, the denoising network is implemented based on conditional denoising score matching and inverse stochastic differential equations; Conditional denoising score matching: The denoising network uses the following parameters: One-dimensional conditional U-Net network Achieve parameterized approximation of observed data sequences and noise level Conditional fractional function The conditional fractional function is used to characterize the log probability density gradient of the action window under conditional observation data, that is: ; Indicates the action window, Indicates to Find the partial derivative; The training process of the denoising network uses Fisher divergence as the loss function. : ; This represents the expectation for the training data, noise level, and random noise. This represents the observation-action distribution in the teaching data, used in imitation learning. Approximates the true conditional fractional function. For gradient operators; In practical reasoning, the denoising network will include the latest observation data sequence of environmental feedback. As a conditional input, inverse stochastic differential equation evolution is performed to update the action window in real time and track the target action distribution.

[0015] The system and method in this invention correspond to each other; the specific technical solutions applicable to the method are also applicable to the system.

[0016] Compared with the prior art, the beneficial technical effects of the present invention are: Achieving true real-time closed-loop control: Abandoning the traditional action block prediction paradigm, this invention immediately incorporates the latest sensor observation data when generating each individual action. This high-frequency, fine-grained action-level feedback replaces coarse block-level feedback, enabling the robot to react instantaneously to environmental dynamics and external disturbances.

[0017] Extremely high time efficiency and inference speed: Due to the adoption of a rolling window mechanism and linear noise scheduling, during the stable operation phase, this invention only needs to perform a single denoising iteration in each control step, completely eliminating the need for a complete long chain of denoising calculations from pure noise to clear action, reducing inference latency, and easily meeting the high-frequency control requirements for agile operation.

[0018] This invention improves the smoothness and stability of motion, eliminating discontinuities caused by action block boundaries. The motion sequences generated by this invention are naturally smooth in terms of velocity and acceleration, reducing jitter during robot execution and improving the safety and success rate of complex tasks (such as precision assembly and liquid pouring).

[0019] Reduced parameter sensitivity: The method of this invention is naturally adapted to different observation window lengths and prediction horizons, without the need for tedious parameter tuning (such as tuning action block size) for specific tasks, as is the case with traditional diffusion strategies. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention.

[0021] Figure 2This is a schematic diagram comparing the timing noise initializer in this invention with the traditional diffusion strategy (DP) generation mechanism. Detailed Implementation

[0022] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0023] like Figure 1 As shown, the present invention proposes a rolling closed-loop diffusion strategy control method for dexterous robot operation, comprising the following steps: S1. Obtain the initial action sequence of the robot composed of pure Gaussian noise, and perform conditional denoising on the initial action sequence through a pre-trained denoising network to extract the initial action window with a progressive noise structure. The progressive noise structure refers to the action noise level near the current time being lower than the action noise level far away from the current time within the action window. S2, in each control cycle, acquires observation data containing the latest environmental feedback, inputs the observation data as a condition to the denoising network, performs denoising iteration on the current action window, obtains the noise-free action at the current moment, and sends it to the robot for execution; the observation data includes the robot's body state and point cloud data of the operation scene; S3, shift the remaining noisy action sequence after the robot has executed forward by one time step, and add a new random noise sample at the end of the window to form the action window for the next control cycle; S4. Repeat steps S2 and S3 until the task is completed.

[0024] The purpose of this invention is to overcome the shortcomings of existing diffusion control strategies based on action blocks in terms of real-time responsiveness, closed-loop feedback capability, and motion smoothness. By introducing a temporal noise initialization and a single-step rolling local denoising mechanism, it breaks the constraints of traditional fixed action blocks and achieves true closed-loop control that integrates the latest environmental observation data for each action output. The method of this invention includes constructing a temporal noise initializer, rolling denoising windows, and an environment module.

[0025] The Temporal Noise Initializer is used to construct a rolling denoising window with a progressive noise structure during the task initiation phase. Unlike traditional methods that apply a uniform noise level to the entire action block, this initializer places actions closer to the current time within the window at a lower noise level, while actions further in the future are placed at a higher noise level, providing a temporally consistent starting point for subsequent rolling denoising. The Rolling Denoise Windows are used to continuously generate control actions during the robot's operation. At each decision time step, the window only takes one short step (…). The denoising network iterates through the environment module, outputting the single optimal action for the current moment. Then, the window slides forward, integrating the latest observation data acquired by the environment module, and appends new random noise to the end of the window for the next iteration. The environment module is responsible for real-time perception, processing the robot's own state and environmental sensor data (such as 3D point cloud data of the operating scene), and providing this as conditional input to the denoising network to ensure that the generated actions always closely track the current environmental state.

[0026] The overall process of this invention is as follows: First, a closed-loop control framework including a temporal noise initializer, a rolling denoising window, and an environment module is constructed, and offline training of the denoising score network is completed. Second, during the task startup phase, the environment module collects the robot's initial observation data. The temporal noise initializer receives this initial observation data, performs multiple predictions on standard Gaussian noise sampling, and extracts and combines an initial action window with a linearly increasing noise level. Subsequently, during the task execution phase, at each control time step, the environment module collects the latest observation data in real time and forms an observation history with the most recent historical data. The rolling denoising window performs a denoising iteration on the current action window based on the above observation history, obtaining a noise-free single action at the current moment, which is then executed by the robot. Simultaneously, unexecuted actions within the window are shifted to the left, and pure Gaussian noise is appended to the end to dynamically update the action window. Finally, the above process is repeated to complete continuous closed-loop control until the specified robot operation task is completed.

[0027] Example: The core idea of ​​this invention is to shift the learning objective of the diffusion model from traditional global trajectory modeling to local conditional probability transition modeling. Traditional diffusion strategies attempt to establish a complete inverse path from pure Gaussian noise to a long sequence of optimal actions; while this invention maintains a rolling denoising window, focusing on short time steps. Within the system, the local trajectory is corrected in real time based on the latest observation data, thereby dynamically tracking the ever-changing optimal target distribution. The method of this invention mainly consists of three parts: a temporal noise initializer, a rolling denoising window, and an environment module.

[0028] 1. Model construction and offline training.

[0029] First, an environment module is constructed to handle heterogeneous input data. For the robot's body state (such as joint angles and velocities), a multilayer perceptron (MLP) is used for feature extraction to obtain body features. For the 3D point cloud data of the operating scene, farthest point sampling is used first, followed by MLP and max pooling operations to extract point cloud features. After concatenating the body features and point cloud features, a feature linear modulation layer (FiLM) is used to reduce the noise level during the diffusion process. This is injected into the observed features to form a unified contextual condition vector. Simultaneously, mathematical modeling of the core denoising network is performed. The prediction field length of the action window is set to... The maximum noise level is The step size (noise increment) is Defined at any time Its action window is (in (For the action dimension), its structure includes a linearly increasing noise level: .

[0030] The system state evolution is controlled by forward stochastic differential equations (Forward SDEs): , The corresponding inverse stochastic differential equation for denoising is: , Among them, the drift term Includes a score function with respect to the conditional distribution. . This represents the diffusion coefficient function, used to describe the intensity of random noise injection at a diffusion time t. This represents the Standard Wiener process in a forward stochastic differential equation. This represents the standard Wiener process in the inverse stochastic differential equation. This invention uses a one-dimensional conditional U-Net network (C-Unet). To approximate the scoring function.

[0031] The denoising network uses one-dimensional conditional U-Net network parameters, which are trained offline by sampling observation data sequences from the offline demonstration dataset. The loss function of the one-dimensional conditional U-Net network is optimized based on the score matching formula of Fisher divergence and the corresponding action blocks. : .

[0032] This represents the expectation for the training data, noise level, and random noise. This represents the observation-action distribution in the teaching data, used in imitation learning. Approximate true conditional fractional function.

[0033] 2. Task startup phase and timing noise initialization.

[0034] At the initial moment when the robot begins to perform its task ( This utilizes a time-series noise initializer to break the cold start deadlock. First, a pure noise action sequence is sampled from a standard Gaussian distribution. The initial observation data sequence will then be... As a conditional input, the pre-trained denoising network is invoked. According to step size Iterative denoising yields Arrange these action sequences from different stages in rows to form an evolution matrix. , such that the first matrix The corresponding noise level for each row is The action block. Extract. The diagonal elements, i.e., the corresponding asymptotic noise levels at each time position, are extracted to construct an initial action window with a linearly increasing noise structure: .

[0035] Finally, perform another denoising step on the initial action window to obtain a fully initialized action window. .

[0036] 3. Task execution phase and rolling noise reduction control loop.

[0037] After the initial window is generated, for The rolling noise reduction stage. The environment module first collects the latest observation data. The observation data sequence is obtained by combining historical observation data. Next, for the incomplete denoising process retained from the previous step, a newly sampled Gaussian noise process is appended to the end. This constitutes the input action window at the current moment. At this time, the actions in the action window will sequentially have... The noise. Then and It is fed as input to a one-dimensional conditional U-Net network. Since the network has already learned the local conditional probability distribution, it only needs to pass through the corresponding step size once. The inverse stochastic differential equation operation can reduce the overall noise of the window by one level, and output the denoised action window. .

[0038] extract The first element This is a completely noise-free optimal motion, which is sent to the robot joint controller for immediate execution. The remaining, not fully denoised motions in the window are retained and carried over to the next control loop.

[0039] 4. Control execution and performance evaluation.

[0040] To ensure efficient system operation and verify the effectiveness of the invention, relevant hyperparameters need to be set and the system evaluated during the control execution phase. In a specific control execution embodiment, the action prediction field of view length is set. The range is 8 to 32 (preferably 8), and the length of the observation data sequence is... The range is 2 to 4 (preferably 2). The maximum number of denoising steps depends on the complexity of the task being performed. The configuration ranges from 80 to 360, thus corresponding to the single-pass noise reduction step size. It ranges from 10 to 45. In the execution of specific long-cycle control tasks, compared to the traditional action-block-based diffusion strategy, the total processing length is... Tasks need to be executed in a concentrated manner In the subsequent noise reduction inference, the rolling closed-loop strategy of this invention was executed in total approximately [number missing]. Secondary reasoning.

[0041] The pseudocode corresponding to the method of this invention is as follows: 1: Input: The trained closed-loop model and timing noise initialization network Maximum length of the action ; 2: Use initialization : 3: Settings , where initially ; 4: Create an empty list ; 5: Sampling ; 6: Settings ; 7: Length is ; 8: ; 9: ; 10: ; 11: ; 12: Add noise level to The action; 13: End for (length is...) ; 14: Settings and ; 15: Acquisition ; The length is ; 16: Begin controlling the loop; 17: ; 18: And sampling ; 19: Sampling level is noise; 20: ; twenty one: ; twenty two: ; 23: Calculation via inverse stochastic differential equations ; Reverse step; 24: End for; 25: ; 26: ; 27: End for.

[0042] In the pseudocode above, the first line represents the input to the trained closed-loop denoising network. Temporal noise initialization network and the maximum number of control steps allowed for task execution. Among them, the closed-loop denoising network Used to estimate the conditional score function based on the current action window, noise level, and latest observation data during the task execution phase; Temporal noise initialization network. Used to conditionally denoise pure Gaussian noise action sequences during the task initiation phase, in order to construct an initial action window with a progressive noise structure.

[0043] The second line indicates that the network is initialized using temporal noise. For the initial action window Perform initialization. Line 3 indicates the construction of the initial observation data sequence. ,in This represents the observation data collected by the robot at the initial moment. This represents the observation data from the previous moment; since there is no actual observation data from the previous moment at the initial moment of the mission, let... Line 4 indicates the creation of an empty list. This is used to sequentially store action vectors extracted from different noise levels. The 5th row represents the vectors extracted from the standard Gaussian distribution. Medium-sampled pure noise action sequence ,in Indicates the maximum noise level. This represents the identity matrix. Line 6 indicates setting the index variable. This is used to extract actions sequentially from the end of the action sequence during initialization.

[0044] Lines 7 through 13 represent the timing noise initialization loop during the task startup phase. Specifically, line 7 indicates the noise level. from By step size Gradually decreasing to 0, the cycle length is the action prediction visual field length. Line 8 indicates the current noisy action sequence. Initial observation data sequence and noise level Input timing noise initialization network The noise predicted by the network is obtained. Line 9 indicates the denoising update formula based on the diffusion model, from... The action sequence at a lower noise level was calculated. Line 10 indicates the sequence of actions. Extract the first One action, and add it to This creates a set of actions at different time locations corresponding to different noise levels. Line 11 indicates that the index variable... Increment by one so that the next iteration can extract actions from adjacent positions. Line 12 indicates the current addition. Actions related to noise level The following action. Line 13 indicates the end of the timing noise initialization loop.

[0045] Line 14 indicates that... Reverse arrangement to obtain the initial scrolling action window And the first action pops up from that window. This represents the action being performed at the current moment. Line 15 indicates the action being performed by the robot's environment. And return the observation data for the next time step. At this point, since the first action has already been executed, the window... The number of remaining actions is Line 16 indicates the start of the rolling closed-loop control cycle.

[0046] Lines 17 to 27 represent the rolling closed-loop denoising control process during the task execution phase. Line 17 represents the control time step. Increasing from 1 to That is, at most execute One control cycle. Line 18 indicates the control cycle based on the observation data from the previous moment. and current observation data Constructing observation data sequences and from the standard Gaussian distribution Mid-sample new random noise Line 19 indicates the maximum noise level corresponding to this random noise. This is used to append to the end of the action window. Line 20 indicates that the action window retained from the previous control cycle is being added. With newly sampled random noise The noisy action window of the current control cycle is formed by splicing the data together. Lines 21 to 24 indicate the adjustment of the noise level of the current action window. The reverse denoising process to noise level 0. Line 22 indicates the use of a closed-loop denoising network. Based on the current action window Noise level and observation data sequence Estimating the conditional score function Line 23 shows the denoised action window calculated using the inverse stochastic differential equation based on this conditional score function. Line 24 indicates the end of the denoising iteration in the current control cycle. Line 25 indicates the start of the action window after denoising. The first fully denoised action popped up in the middle. Line 26 indicates the action performed by the robot. The environment then returns the observation data for the next moment. Line 27 indicates the end of the rolling closed-loop control cycle.

[0047] Although the theoretical total computational load tends to be consistent over a long period, this invention evenly distributes the massive batch denoising computational load across each control loop, requiring only a single, extremely short denoising operation at each step. This execution mechanism completely eliminates the task execution stuttering or pauses caused by batch action generation in traditional methods, significantly reducing the latency of single-step inference. In real-world physics experiments, the overall execution time is shortened by approximately 46%. Simultaneously, the high-frequency motion-level control output effectively avoids motion discontinuities at the boundaries of adjacent action blocks, resulting in a natural and smooth motion trajectory of the robot's end effector (TCP) across six degrees of freedom (including three-dimensional linear velocity and three-dimensional angular velocity), greatly reducing abrupt changes in joint velocity and acceleration. When faced with unavoidable sensor noise, actuator hysteresis, or unforeseen human interference in the physical environment, such as the target being accidentally moved during grasping, the system can immediately suspend the original planned trajectory and quickly respond with actions such as re-tracking and grasping, because the robot can instantaneously acquire the latest observation history and make local strategy corrections in each control cycle. This true real-time closed-loop feedback mechanism significantly improves the robot's dynamic anti-interference ability, operational stability, and final task success rate when performing high-precision and dexterous operations such as bucket lifting, socket assembly, block stacking, and continuous picking and placing.

[0048] The contents not described in detail in this invention are existing technologies known to those skilled in the art.

[0049] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0050] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0051] Based on the description of the above method embodiments, the present invention also provides a system. The system may be a system that uses software (applications), modules, components, servers, clients, etc., using the methods described in the embodiments of this specification, combined with necessary implementation hardware. Since the implementation schemes and methods for solving the problem are similar, the specific system implementations in the embodiments of this specification can be found in the implementations of the foregoing methods, and repeated details will not be described again. Although the system described in the embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0052] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0053] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0054] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A rolling closed-loop diffusion strategy control method for dexterous robot manipulation, characterized in that, include: S1, Obtain the robot's initial action sequence composed of pure Gaussian noise, and conditionally denoise the initial action sequence using a pre-trained denoising network to extract the initial action window with a progressive noise structure: Set the time... Noise level Action window below for: ; This represents the noise level increment between consecutive actions within the action window; T represents the maximum noise level; H represents the fixed field of view length for action prediction. Indicates at time and noise level The generated d-dimensional action vector; Indicates definition, Represents the set of real numbers; within the initial action window, the closer to the current time... The action is assigned a lower noise level; the progressive noise structure refers to the action noise level closer to the current moment within the action window being lower than the action noise level further away from the current moment. S2, in each control cycle, acquires observation data containing the latest environmental feedback, uses this observation data as a conditional input to the denoising network, performs denoising iterations on the current action window, obtains the noise-free action at the current moment, and sends it to the robot for execution: based on a given moment... Length is Observational data sequence Using a denoising network, for time... Noise level is Action window below After a step size of After the denoising iteration, the overall noise level of all actions within the action window is reduced. : ; For the denoised action window, Indicates at time and noise level The generated action vector, where H represents the fixed field of view length for action prediction; express Observational data at any given time; Denoising action window Includes Partially denoised action vectors, and the first fully denoised action vector. The observation data includes point cloud data of the robot's physical state and the operating scene. S3, shift the remaining noisy action sequence after the robot has executed forward by one time step, and add a new random noise sample at the end of the window to form the action window for the next control cycle; S4. Repeat steps S2 and S3 until the task is completed.

2. The rolling closed-loop diffusion strategy control method for dexterous robot operation according to claim 1, characterized in that, The elements of the motion vector include one or more of the following: robot joint position control, joint velocity control, end effector pose increment, end effector velocity control, and end effector opening / closing control.

3. The rolling closed-loop diffusion strategy control method for dexterous robot operation according to claim 1, characterized in that, The denoising network is implemented based on conditional denoising score matching and inverse stochastic differential equations; Conditional denoising score matching: The denoising network uses the following parameters: One-dimensional conditional U-Net network Achieve parameterized approximation of observed data sequences and noise level Conditional fractional function The conditional fractional function is used to characterize the log probability density gradient of the action window under conditional observation data, that is: ； Indicates the action window, Indicates to Find the partial derivative; The training process of the denoising network uses Fisher divergence as the loss function. : ； This represents the expectation for the training data, noise level, and random noise. This represents the observation-action distribution in the teaching data, used in imitation learning. Approximates the true conditional fractional function. For gradient operators; In practical reasoning, the denoising network will include the latest observation data sequence of environmental feedback. As a conditional input, inverse stochastic differential equation evolution is performed to update the action window in real time and track the target action distribution.

4. A rolling closed-loop diffusion strategy control system for dexterous robot operation, characterized in that, include: The noise-adding module is initialized to obtain the robot's initial action sequence composed of pure Gaussian noise. A pre-trained denoising network is then used to conditionally denoise the initial action sequence, extracting the initial action window with a progressive noise structure. The time interval is set. Noise level Action window below for: ;in, This represents the noise level increment between consecutive actions within the action window; T represents the maximum noise level; H represents the fixed field of view length for action prediction. Indicates at time and noise level The generated d-dimensional action vector; Indicates definition, Represents the set of real numbers; within the initial action window, the closer to the current time... The action is assigned a lower noise level; the progressive noise structure refers to the action noise level closer to the current moment within the action window being lower than the action noise level further away from the current moment. The rolling denoising module acquires observation data containing the latest environmental feedback in each control cycle, inputs this observation data as a condition into the denoising network, performs denoising iterations on the current action window, obtains the noise-free action at the current moment, and sends it to the robot for execution, including: based on a given moment... Length is Observational data sequence Using a denoising network, for time... Noise level is Action window below After a step size of After the denoising iteration, the overall noise level of all actions within the action window is reduced. : ; For the denoised action window, Indicates at time and noise level The generated action vector, where H represents the fixed field of view length for action prediction; express Time-lapse observation data; noise-reduced action window Includes Partially denoised action vectors, and the first fully denoised action vector. The observation data includes point cloud data of the robot's physical state and the operating scene. The window update sliding module shifts the remaining noisy action sequence after the robot has executed forward by one time step, and adds a new random noise sample at the end of the window to form the action window for the next control cycle. Repeatedly scroll the noise reduction module and the window update slider module until the task is completed.

5. A rolling closed-loop diffusion strategy control system for dexterous robot operation according to claim 4, characterized in that, The elements of the motion vector include one or more of the following: robot joint position control, joint velocity control, end effector pose increment, end effector velocity control, and end effector opening / closing control.

6. A rolling closed-loop diffusion strategy control system for dexterous robot operation according to claim 4, characterized in that, The denoising network is implemented based on conditional denoising score matching and inverse stochastic differential equations; Conditional denoising score matching: The denoising network uses the following parameters: One-dimensional conditional U-Net network Achieve parameterized approximation of observed data sequences and noise level Conditional fractional function The conditional fractional function is used to characterize the log probability density gradient of the action window under conditional observation data, that is: ； Indicates the action window, Indicates to Find the partial derivative; The training process of the denoising network uses Fisher divergence as the loss function. : ； This represents the expectation for the training data, noise level, and random noise. This represents the observation-action distribution in the teaching data, used in imitation learning. Approximates the true conditional fractional function. For gradient operators; In practical reasoning, the denoising network will include the latest observation data sequence of environmental feedback. As a conditional input, inverse stochastic differential equation evolution is performed to update the action window in real time and track the target action distribution.

Images