A robot control method and device based on uncertainty perception and diffusion strategy fusion

Abstract

The application discloses a robot control method and device based on uncertainty perception and diffusion strategy fusion, takes a state space sequence visual backbone as a perception front end, and introduces a joint scheme of a fusion adapter, an uncertainty perception sampling unit and state-vision alignment distillation. The fusion adapter obtains time-consistent features and outputs visual uncertainty through channel re-calibration, scale alignment and cross-frame gating. The uncertainty perception sampling unit adjusts the temperature and step length of diffusion sampling according to the uncertainty monotone, and triggers action resampling when the confidence is too low. The state-vision alignment distillation loss takes the task state as an anchor to reduce the interference of appearance artifacts on the action distribution. The method of the application can still maintain high robustness and generalization ability in a small sample and open environment, and the robot action performance is smoother.

Description

A robot manipulation method and device based on the fusion of uncertainty perception and diffusion strategy Technical Field

[0001] This invention belongs to the field of machine learning and robot control technology, and particularly relates to a robot manipulation method and device based on the fusion of uncertainty perception and diffusion strategy. Background Technology

[0002] Existing policy learning methods based on diffusion models (hereinafter referred to as "diffusion strategies") demonstrate strong imitation learning capabilities in vision-dominated tasks such as desktop manipulation, grasping, and assembly. However, when shifting between visual domains across scenes, lighting conditions, and materials, action degradation often occurs due to observation drift. For example, when lighting, materials, background, or viewpoint changes, the appearance statistics and geometric cues of consecutive frames undergo abrupt changes, the features output by the traditional visual backbone become unstable over time, and the focus of attention drifts with each frame, thus inducing high-frequency control noise in the diffusion strategy, manifesting as grasping jitter, empty grasping, or lag.

[0003] Existing diffusion strategies typically employ fixed temperatures and fixed step sizes for denoising and sampling during the inference phase. Even in areas with strong reflections, occlusion, or low texture, increased uncertainty at the perception end cannot be transmitted to the control end. This leads to the continued use of high temperatures and large step sizes even when visibility is poor, amplifying decision-making risks. When the appearance changes but the task objective remains the same, the lack of explicit alignment with task states such as end-effector pose and contact / geometric constraints causes visual latent variables to drift with the appearance, resulting in a drift in the distribution of dragging actions.

[0004] In summary, existing policy learning methods based on diffusion models are sensitive to appearance changes, lack sufficient modeling of occlusion and long-term dependencies, and lack coupling between visual confidence and action sampling noise. To alleviate these problems, researchers typically replace the visual backbone or perform domain randomization. However, simply replacing the backbone cannot fundamentally solve the mismatch between "visual uncertainty and action diffusion," resulting in insufficient generalization. This invention aims to provide an implementable, calibrable, and computationally controllable joint solution to address the core technical contradictions that arise when deploying diffusion strategies in open environments. Summary of the Invention

[0005] This invention addresses the various problems existing in the prior art by providing a robot manipulation method and device based on the fusion of uncertainty perception and diffusion strategy.

[0006] According to a first aspect of the present invention, a robot manipulation method based on the fusion of uncertainty perception and diffusion strategy is provided, comprising the following steps:

[0007] S1. Acquire a certain number of frame images. and corresponding status information This forms an image sequence and a state information sequence, and performs data preprocessing.

[0008] S2. Extracting temporal features using a state-space-based sequence visual backbone. and uncertain priors ;

[0009] S3, the time sequence features and uncertain priors Input the fusion adapter to obtain the fusion features. With visual uncertainty ;

[0010] S4. Fusion Features Based on Diffusion Strategy Network Training is performed, and the training objective is constituted by the joint loss function L;

[0011] S5. Based on visual uncertainty Adjusting the temperature during diffusion sampling With step size ,when Exceeding the threshold The action is resampled N times, where N is a natural number.

[0012] S6. The prediction results are converted into control commands for the robot execution unit through the motion decoder.

[0013] Further, in step S1 above, the image sequence is composed of a demonstration dataset D, which is a mixture of real-sampled data D_{real} and synthetic perturbation data D_{syn}; the ratio of real-sampled data D_{real} to synthetic perturbation data D_{syn} is p∶(1−p), where .

[0014] Furthermore, in step S2 above, the sequence visual backbone is a structured visual model based on state space modeling, which is used to enhance the modeling capability of long temporal dependencies.

[0015] Furthermore, the fusion adapter includes, in sequence, channel recalibration, scale alignment, timing alignment, and an uncertainty estimation header.

[0016] Furthermore, the uncertainty estimation head fuses the feature entropy. Monte Carlo Dropout Variance Reconstruction error Output visual uncertainty :

[0017] ,

[0018] Among them, coefficient ,i=1,2,3, and .

[0019] Furthermore, in step S4 above, the joint loss function L:

[0020]

[0021] in, For behavioral cloning loss, For state-visual alignment distillation loss, For action-state cycle consistency loss, For cross-perspective / style consistency loss, For the uncertainty calibration loss, is a coefficient.

[0022] Furthermore, in step S5 above, the visual uncertainty... Used to calculate sampling temperature With step size ,satisfy: and Monotonically increasing, and Monotonically decreasing; when Exceeding the threshold When N action resampling is triggered, the optimal action is selected based on the risk cost function, which is used to minimize the task failure rate and trajectory jitter.

[0023] According to a second aspect of the present invention, a robot control device based on the fusion of uncertainty perception and diffusion strategy is provided, the device specifically comprising:

[0024] Visual acquisition unit, used to acquire images ;

[0025] The state acquisition unit is used to acquire the robot's state. ;

[0026] Visual backbone units are used to extract temporal features. and uncertain priors ;

[0027] Fusion adapter unit, used to output fused features With visual uncertainty ;

[0028] The diffusion policy training unit is used to train the policy network based on the joint loss function L;

[0029] Uncertainty-aware sampling unit, used to... Adjusting the sampling temperature With step size ;

[0030] The action decoder unit is used to convert the prediction results into control commands.

[0031] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the aforementioned methods.

[0032] In a fourth aspect, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any of the aforementioned methods.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) In view of the problems of feature drift and attention instability that often occur in cross-domain deployment based on visual backbone, this invention designs a fusion adapter, introduces cross-frame gating and learnable delay compensation, so that the output features have structured cross-frame consistency, and simultaneously outputs visual uncertainty estimates, which solves the problems of "visual drift-control jitter" and "sensor-control asynchrony and insufficient uncertainty quantification", ensures the stability and consistency of continuous frame features, avoids attention drift caused by sudden changes in lighting, material and viewpoint, alleviates sensing-control delay, and quantifies observation confidence.

[0035] (2) To address the motion distribution drift caused by appearance artifacts, this invention adopts the InfoNCE-type state-visual alignment distillation loss (SVD Loss), which explicitly constrains the visual latent variables with the task state as the anchor point, so that the latent variables are firmly anchored in the task state space, effectively removing appearance factors that are irrelevant to the task, fundamentally eliminating the interference of appearance artifacts on the motion distribution, ensuring the stability of the motion distribution, and in conjunction with the action-state loop consistency constraint, ensuring that the learned strategy is more in line with the real task requirements.

[0036] (3) This invention employs an Uncertainty Awareness Sampling Unit (UANT), which directly maps visual uncertainty to sampling temperature and step size through a monotonic function and triggers threshold resampling. For the first time, it transmits perceptual uncertainty to the sampling control loop and visual risk to the action control loop, realizing "slowing down when it's unclear and accelerating when it's clear," significantly reducing motion jitter and failure rate. This invention differs from the traditional diffusion strategy inference mode where sampling temperature and step size are fixed, avoiding the risk of "progressing in a high-risk manner even if the vision is blurred or severely obstructed."

[0037] (4) This invention addresses the problem that large-scale re-annotation and expensive domain adaptation are often impossible in industrial and laboratory deployments. This invention can be effective with small-scale demonstration data. It achieves controllable coverage of real deployment noise through cross-domain hybrid data pipeline (style perturbation, temporal occlusion, lens / exposure jitter and short scrolling online enhancement), and training and inference are guaranteed to be implemented on the end side by linear complexity. Attached Figure Description

[0038] Figure 1. Specific steps of the robot manipulation method based on the fusion of uncertainty perception and diffusion strategy provided by the present invention;

[0039] Figure 2. Obtaining fusion features via a fusion adapter as provided by the present invention. With uncertainty Steps;

[0040] Figure 3. Fusion features based on diffusion strategy network provided by the present invention. The steps for conducting training;

[0041] Figure 4. The steps provided by the present invention to directly map visual uncertainty to sampling temperature and step size using a monotonic function;

[0042] Figure 5(a) shows the robot provided by the present invention performing a small object grasping task;

[0043] Figure 5(b) shows the robot provided by the present invention performing a multi-object classification and placement task;

[0044] Figure 5(c) shows the robot provided by the present invention performing a complex tool operation task;

[0045] Figure 6. Architecture diagram of the robot control device based on uncertainty perception and diffusion strategy provided by the present invention. Detailed Implementation

[0046] The techniques described below can be modified in various ways and have multiple embodiments, which are described in detail below with reference to the accompanying drawings. However, this does not mean that the techniques described below are limited to the specific embodiments. It should be understood that the present invention includes all similar modifications, equivalents, and substitutions without departing from the spirit and scope of the techniques described below.

[0047] Example 1

[0048] This invention provides a robot manipulation method based on the fusion of uncertainty perception and diffusion strategy. It uses a state-space sequence visual backbone as the perception front end and introduces a joint scheme of fusion adapter, uncertainty perception sampling unit, and state-visual alignment distillation. The fusion adapter obtains temporally consistent features and outputs visual uncertainty through channel recalibration, scale alignment, and cross-frame gating. The uncertainty perception sampling adjusts the temperature and step size of the diffusion sampling based on the uncertainty monotonicity and triggers action resampling when the confidence level is too low. The state-visual alignment distillation loss is anchored to the task state to reduce the interference of appearance artifacts on action distribution.

[0049] As shown in Figure 1, the robot manipulation method based on the fusion of uncertainty perception and diffusion strategy of the present invention specifically includes the following steps:

[0050] Step S1: Acquire a certain number of frame images and corresponding status information This forms an image sequence and a state information sequence, and performs data preprocessing.

[0051] Image sequence The demonstration dataset D consists of a mixture of real-sampled data D_{real} and synthetic perturbation data D_{syn}. The ratio of real-sampled data D_{real} to synthetic perturbation data D_{syn} is p∶(1−p), where... The preferred value is p=0.5. Aligned time series samples:

[0052] ,

[0053] in, For the image / frame at time t, This refers to the state information at time t (such as end-effector pose, contact / geometric constraints, etc.). The action is at time t. A sequence of images of length T.

[0054] This invention provides a cross-domain hybrid data pipeline that offers systematic perturbation coverage, enabling it to adapt to complex deployment scenarios without large-scale annotation, and improving robustness and generalization ability under small sample constraints. Specifically, at the data level, it employs a hybrid training approach using both real-world data and synthetic perturbation data, including:

[0055] Input: The original dataset D_{real} and the synthetic perturbation library D_{syn}.

[0056] deal with:

[0057] Style Disturbance Bank: Brightness [0.6, 1.4], Color Temperature Shift [−800K, +800K], 50 Material Replacements;

[0058] Occlusion simulation: scale [5%, 25%];

[0059] Exposure / Lens Shake: Normal Distribution , σ∈[0.01,0.05];

[0060] Short scroll enhancement: k∈[2,8], trigger condition is .

[0061] Output: Perturbation-enhanced sample set .

[0062] To address the challenges of large-scale relabeling and expensive domain adaptation in industrial and laboratory deployments, the method of this invention is effective even with small-scale demonstration data. It achieves controllable coverage of real-world deployment noise through a cross-domain hybrid data pipeline (style perturbation, temporal occlusion, lens / exposure jitter, and short scrolling online enhancement), and training and inference are guaranteed to be implemented on the edge with linear complexity.

[0063] In addition to the combination of style perturbation, occlusion simulation, and short scroll enhancement, random style transfer based on generative adversarial networks (GANs), simulation domain synthesis based on physically rendered renderers, and random cropping / frame interpolation methods based on video temporal perturbations can also be extended to synthesized-real-sampling hybrid datasets to be compatible with the training framework of this invention.

[0064] Step S2: Extract temporal features using a state-space-based sequence visual backbone. and uncertain priors ;

[0065] In step S1 As input, temporal features are obtained through state-space-based sequence visual backbone processing. And generate uncertain priors. The output of step S2 is .

[0066] Specifically, a state-space based backbone (such as MambaVision) is adopted, and long-term dependencies are modeled through State-Space Hybrid Blocks (SSM-Conv) to ensure cross-frame consistency. Each block contains a one-dimensional state-space layer and a two-dimensional convolutional layer, outputting a multi-scale feature pyramid. The complexity is O(n log n). Where T is the number of frames and d is the number of channels.

[0067] Besides employing a state-space-based backbone (such as MambaVision), a convolutional-state-space hybrid can be used, where convolutional layers extract local textures and state-space layers maintain long-range dependencies; alternatively, an improved temporal Transformer can be used, where attention computation is constrained to linear complexity, functionally equivalent to a state-space backbone. Under these alternatives, the structure and function of the fusion adapter, UANT, and SVD losses remain unchanged, still achieving cross-frame consistency and uncertainty propagation.

[0068] Step S3: The time series features and uncertain priors Input the fusion adapter to obtain the fusion features. With visual uncertainty ;

[0069] As shown in Figure 2, the fusion adapter includes, in sequence, channel recalibration, scale alignment, and timing alignment (Δt and...). And uncertainty estimation head. The input is processed by the fusion adapter to obtain features. With visual uncertainty (0 = high confidence, 1 = low confidence). Specifically:

[0070] Channel recalibration: SE module or 1×1 convolution, output dimension ;

[0071] Scale alignment: linear layer mapping to dimension;

[0072] Timing alignment: Includes learnable delay compensation Δt∈{0,1,2} frames, cross-frame gating:

[0073] ;

[0074] in, Let W be the Sigmoid function and W be the learnable weights.

[0075] Uncertainty estimation head: fusion feature entropy Monte Carlo Dropout Variance Reconstruction error Output :

[0076] ,

[0077] Among them, coefficient (i=1,2,3), and .

[0078] To address the issues of feature drift and attention instability that often occur in cross-domain deployments based on visual backbones, this invention designs a fusion adapter that introduces cross-frame gating and learnable latency compensation. This enables the output features to have structured cross-frame consistency and simultaneously outputs visual uncertainty estimates, solving the problems of "visual drift-control jitter" and "sensor-control asynchrony and insufficient uncertainty quantification". It ensures stable and consistent features across consecutive frames, avoids attention drift caused by sudden changes in illumination, material, and viewpoint, alleviates sensor-control latency, and quantifies observation confidence.

[0079] This invention's uncertainty estimation head employs a linear fusion approach using multiple metrics (entropy, dropout variance, and reconstruction error) to estimate visual uncertainty. Furthermore, it can also utilize uncertainty prediction based on Bayesian neural networks, or use confidence scores based on instance segmentation or depth estimation as indirect proxies. Regardless of the estimation method used, its output format is as follows: It is also necessary to maintain a monotonic mapping relationship with UANT to ensure the adaptability of diffusion sampling.

[0080] Step S4: Based on the diffusion strategy network, perform fusion feature analysis. Training is performed, and the training objective is constituted by the joint loss function L;

[0081] As shown in Figure 3, during the training phase, the fused features (visual latent variables) are constrained by the task state as the anchor point, thereby ensuring that the latent variables are stably mapped to the task state space. The training process includes five types of loss functions, namely... , , , .Will( , The network parameters are updated using five loss functions (g, g) as input, where g is the task cue, and the training objective is formed by combining these five loss functions to update the diffusion strategy network parameters. Output the trained Joint loss function L:

[0082]

[0083] in:

[0084] , where is the behavioral cloning loss, used to maintain consistency in imitation learning, where Standard Gaussian noise, For noise prediction in diffusion strategy networks, c represents conditional input;

[0085] For state-visual alignment distillation loss, Where sim is the cosine similarity. Ψ is a learnable mapping, and τ is a temperature coefficient;

[0086] , where h is the action-state loop consistency loss, and h is the dynamic approximator. The next time step task state is predicted by the model / forward operator;

[0087] Cross-viewpoint / style consistency loss is used to maintain the consistency of feature representations under different viewpoints or style conditions;

[0088] To calibrate the loss for uncertainty, the reliability of the model's uncertainty predictions is improved by comparing the prediction confidence histogram with the error distribution histogram. ,in To predict the confidence histogram, This is a histogram of the error distribution.

[0089] is a coefficient.

[0090] To address the motion distribution drift caused by appearance artifacts, this invention employs InfoNCE-type State-Visual Alignment Distillation Loss (SVD Loss). By explicitly constraining visual latent variables with the task state as the anchor point, the latent variables are stably anchored in the task state space, effectively removing appearance factors irrelevant to the task. This fundamentally eliminates the interference of appearance artifacts on motion distribution, ensuring stable motion distribution. Furthermore, combined with action-state loop consistency constraints, it ensures that the learned strategy is more closely aligned with the requirements of real tasks.

[0091] Step S5: Based on visual uncertainty Adjusting the temperature during diffusion sampling With step size ,when Exceeding the threshold Trigger N action resampling times;

[0092] As shown in Figure 4, in this step, visual uncertainty... The sampled temperature was obtained through a monotonic mapping function. With step size ,when Exceeding the threshold Action resampling is triggered at any time. Specifically, this step is implemented by UANT, which will... As input, according to Adjusting the sampling temperature With step size :

[0093] ,

[0094] ,

[0095] in

[0096] And satisfy the monotonicity constraint , .

[0097] when At that time, trigger a re-observation or action resampling N times. and according to the cost function Choose the best, among which For dynamic / geometric approximators To perform actions Predicting the state of the next moment (which can be pose or key point state). The distance metric for the task space is preferably Euclidean / pose-weighted distance. The trajectory curvature / smoothness metric is defined as follows: ,in The weighting factor (preferably satisfying) ,like ; k is the length of the historical action window used for smoothness evaluation (preferred) ).

[0098] This invention employs a Unidirectional Anomaly Array (UANT) to directly map visual uncertainty to sampling temperature and step size via a monotonic function, triggering threshold resampling. For the first time, it transmits perceptual uncertainty to the sampling control loop and visual risk to the action control loop, achieving "slowing down when visibility is poor and accelerating when visibility is clear," significantly reducing motion jitter and failure rate. This invention differs from traditional diffusion strategies that use fixed sampling temperature and step size during inference, avoiding the risk of "progressing in a high-risk manner even with blurred vision or severe occlusion."

[0099] In this step, according to Adjusting the sampling temperature With step size In addition to using monotonic mapping functions (linear functions), piecewise linear functions can also be used, with different slopes in different uncertainty intervals, or monotonic neural network approximation functions can be used, with training ensuring... .

[0100] Step S6: Convert the prediction results into control commands for the robot execution unit using the motion decoder.

[0101] sampling action As input, the motion decoder maps the data into low-dimensional control instructions, and outputs executable robot instructions, including pose control vector Δx∈ℝ³, joint angle control Δq∈ℝ^n, force / torque control τ∈ℝ^n, etc., to complete the perception-control closed loop.

[0102] Figures 5(a)-(c) show the differences in technical performance between the method provided by the present invention and the prior art (which only replaces the baseline of the visual backbone) in three types of scenarios (illumination disturbance, multi-target interference, and reflective occlusion).

[0103] Figure 5(a) illustrates the robot performing a small object grasping task. In a desktop environment, the robot needs to grasp a red square from its starting position and place it in a designated container. The environment contains random lighting disturbances. The input sequence length is T=20 frames, the backbone uses a state-space based visual model, and the fusion adapter enables latency compensation Δt=1. The image in the second row of Figure 5(a) shows that the attention heatmap of the comparative example exhibits significant drift in consecutive frames, with attention to the background area in some frames. In ten repeated experiments, the task was completed 6 / 10 times, and trajectory jitter occurred. The image in the third row of Figure 5(a) shows that the heatmap using the method of this invention is stably focused on the square area. In ten repeated experiments, the task was completed 9 / 10 times, and the trajectory is smooth. This fully verifies that the method of this invention, based on the gating and latency compensation mechanism of the fusion adapter, has better temporal consistency and action stability.

[0104] Figure 5(b) illustrates the robot performing a multi-object classification and placement task. The robot faces multiple colored cubes on a table and needs to classify and place them according to preset rules: for example, placing a yellow cube into a red cup and a purple cube into a brown cup. Other colored cubes are also present in the scene, creating multi-object interference. The second row of Figure 5(b) shows that the attention heatmap of the comparative example shifts in some frames, focusing on irrelevant cubes, leading to incorrect action selection or placement failure. In ten repeated experiments, the overall task completion rate was only 5 / 10. The third row of Figure 5(b) shows that the attention heatmap of the method of this invention can stably focus on the target cube and its corresponding cup, successfully completing the classification and placement actions. In ten repeated experiments, the success rate increased to 8 / 10. Compared to the comparative example, the method provided by this invention reduces the action error rate by approximately 20–40% and significantly reduces trajectory deviation. This fully verifies the uncertainty estimation and cross-view consistency loss based on the fusion adapter output of this invention. State-visual alignment distillation loss The combined effect of [these factors] enhances robustness and generalization ability in multi-target classification and interference scenarios.

[0105] Figure 5(c) illustrates a robot performing a complex instrument manipulation task. The robot needs to grasp a transparent glass and place it. The scene contains strong reflections and partial occlusion. The second row of the image in Figure 5(c) shows that in the comparative example, when the glass is occluded or reflects light, the heatmap is dispersed, and gripper jitter and delay occur during the action execution. In ten repeated experiments, the task success rate is 4 / 10. The third row of the image in Figure 5(c) shows that the method of the present invention automatically reduces the diffusion sampling step size and triggers resampling when uncertainty increases, resulting in smoother movements. In ten repeated experiments, the success rate increases to 7 / 10. Compared to the comparative example, the method of the present invention has a lower motion trajectory curvature and less jitter. This fully verifies the adaptability of the present invention, which uses the UANT module to transmit visual confidence to the sampling control loop, in high-uncertainty scenarios.

[0106] In summary, across three scenarios (illumination disturbance, multi-target interference, and reflective occlusion), the method provided by this invention, compared to existing technologies (which only replace the baseline of the visual backbone), achieves an average improvement of 20–30% in cross-domain success rate, a decrease of 25–40% in action failure rate, and a decrease of 15–25% in trajectory jitter index (RMS jerk). This clearly demonstrates that the method of this invention maintains high robustness and generalization ability even in small sample sizes and open environments, and provides smoother action performance.

[0107] Example 2

[0108] The present invention also provides a robot control device based on the fusion of uncertainty perception and diffusion strategy, as shown in Figure 6. The device includes an input module, a perception part, a strategy part, and an output module.

[0109] The input module includes a visual acquisition unit and a status acquisition unit. The visual acquisition unit is used to acquire images. The state acquisition unit is used to acquire the robot's state. ;

[0110] The perception component includes a sequence vision backbone unit and a fusion adapter unit. The sequence vision backbone unit is used to extract temporal features. and uncertain priors The fusion adapter unit is used to output fusion features. With visual uncertainty ;

[0111] The policy component includes a diffusion policy training unit and an uncertainty-aware sampling unit. The diffusion policy training unit is used to train the policy network based on the joint loss function L; the uncertainty-aware sampling unit is used to train the policy network based on the joint loss function L. Adjusting the sampling temperature With step size ;

[0112] The output module includes an action decoder unit and a robot execution unit. The action decoder unit is used to convert the prediction results into control commands.

[0113] Example 3

[0114] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the aforementioned methods.

[0115] Example 4

[0116] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of any of the aforementioned methods.

[0117] Although the present invention has been described in detail above with general descriptions and specific embodiments, some modifications or improvements can be made to it. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Other changes and modifications made by those skilled in the art without departing from the spirit and scope of the present invention are still included within the scope of protection of the present invention.

Claims

1. A robot manipulation method based on the fusion of uncertainty perception and diffusion strategy, characterized in that, Includes the following steps: S1. Acquire a certain number of frame images. and corresponding status information S1) Construct an image sequence and a state information sequence, and perform data preprocessing; S2) Extract temporal features through a state-space-based sequence visual backbone. and uncertain priors S3. The time-series features and uncertain priors Input the fusion adapter to obtain the fusion features. With visual uncertainty The fusion adapter sequentially includes channel recalibration, scale alignment, temporal alignment, and an uncertainty estimation header; the uncertainty estimation header fuses feature entropy. Monte Carlo Dropout Variance Reconstruction error Output visual uncertainty ： , where the coefficient i=1,2,3, and S4. Based on diffusion strategy network for fusion features Training is performed, and the training objective is composed of the joint loss function L; S5, based on visual uncertainty Adjusting the temperature during diffusion sampling With step size ,when Exceeding the threshold S6. Trigger N motion resampling times, where N is a natural number; S7. Convert the prediction results into control instructions for the robot execution unit through the motion decoder.

2. The method according to claim 1, characterized in that, In step S1 above, the image sequence consists of a demonstration dataset D, which is a mixture of real-sampled data D_{real} and synthetic perturbation data D_{syn}; the ratio of real-sampled data D_{real} to synthetic perturbation data D_{syn} is... ,in 。 3. The method according to claim 1, characterized in that, In step S2 above, the sequence visual backbone is a structured visual model based on state space modeling, which is used to enhance the modeling capability of long temporal dependencies.

4. The method according to claim 1, characterized in that, In step S4 above, the joint loss function L is: ;in, For behavioral cloning loss, For state-visual alignment distillation loss, For action-state cycle consistency loss, For cross-perspective / style consistency loss, For the uncertainty calibration loss, is a coefficient.

5. A robot control device based on the fusion of uncertainty perception and diffusion strategy, used to perform the steps of the method as described in any one of claims 1 to 4, characterized in that, The device includes: a visual acquisition unit for acquiring images. The state acquisition unit is used to acquire the robot's state. Visual backbone units are used to extract temporal features. and uncertain priors ; Fusion adapter unit, used to output fused features With visual uncertainty The diffusion policy training unit is used to train the policy network based on the joint loss function L; the uncertainty-aware sampling unit is used to... Adjusting the sampling temperature With step size The action decoder unit is used to convert the prediction results into control commands.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4.

Images