Robot basic large model reasoning depolarization method based on strategy comparison decoding

Abstract

The invention discloses a robot basic large model reasoning depolarization method based on strategy comparison decoding. The method comprises the following steps of 1, obtaining a behavior track of a robot basic large model; 2, a target object in the visual observation image os in the behavior track is masked, and a masked visual observation image # imgabs0 # S is obtained and serves as the maximum time step length; 3, respectively calculating action probabilities pt and # imgabs2 of the visual observation image os and the masked visual observation image # imgabs1 #; 4, calculating comparison action probability distribution # imgabs4 # according to the action probabilities pt and # imgabs3 #, taking a maximum value in the comparison action probability distribution to obtain an action corresponding to the maximum action probability, and executing the action under the condition; according to the method, the robustness of the RFMs reasoning result is improved through the object tracking mask; attention of RFMs is redirected to visual clues related to instructions in the reasoning process, and false correlation in visual observation is eliminated.

Description

Technical Field

[0001] The present invention relates to the technical field of robot basic large models, and in particular to a robot basic large model reasoning and debiasing method based on strategy contrast decoding. Background Art

[0002] In recent years, Robotic Foundation Models (RFMs), or robotic manipulation policies, have shown great potential for realizing flexible and versatile robotic systems. The primary goal of such models is to enable users to instruct robots to perform arbitrary tasks through verbal or non-verbal communication, thereby enabling autonomous task execution with minimal human intervention. However, during reasoning, RFMs' predicted actions are significantly biased towards spurious correlations derived from visual observations, which weakens the model's reasoning robustness. They often generate actions with high confidence even when the target object specified in the verbal instruction is absent from the RFM's visual observations, and they cease generating subsequent actions once the gripper reaches a closed state in an intermediate observation. This bias poses persistent challenges, undermining the performance and reliability of RMFs. Summary of the Invention

[0003] Aiming at the problems existing in the prior art, the present invention provides a robot basic large model reasoning debiasing method based on strategy contrast decoding.

[0004] The technical solution adopted in the present invention is:

[0005] A robot basic large model reasoning debiasing method based on policy contrast decoding includes the following steps:

[0006] Step 1: Obtain the behavior trajectory of the robot basic model;

[0007] Step 2: Visual observation image o in the behavior trajectory s Mask the target object in the image to obtain the masked visual observation image S is the maximum time step;

[0008] Step 3: Calculate the visual observation image o separately s and the masked visual observation image The action probability p t and t is the action dimension number, t = 1, ..., M, M is the action dimension;

[0009] Step 4: According to the action probability p t and Calculate and compare the probability distribution of actions Take the maximum value from the comparison action probability distribution to obtain the action corresponding to the maximum action probability, and execute the action under this condition.

[0010] Further, in step 2, the object tracking mask method Track2Mask is used for masking.

[0011] Further, the basic robot large model is a basic robot large model based on the diffusion strategy.

[0012] Further, in step 3, the kernel density estimation is used to calculate the action probability of the basic robot large model based on the diffusion strategy.

[0013] Further, the calculation process of the action probability is as follows:

[0014] Sample N noise vectors from the Gaussian distribution;

[0015] Parallel denoising of the noise vectors through the reverse diffusion process to generate N candidate action prediction values

[0016] Action a s The t-th action dimension a t The action probability distribution of is:

[0017]

[0018] In the formula: is the Gaussian kernel, and b is the bandwidth parameter that controls the smoothness of the distribution;

[0019] Calculate the action probabilities of M action dimensions in sequence to obtain the action probability.

[0020] Further, in step 4, the comparison action probability distribution is calculated by the policy comparison decoding method.

[0021] Further, the comparison action probability distribution The calculation process is as follows:

[0022]

[0023] In the formula: α is the modulation coefficient.

[0024] The beneficial effects of the present invention are:

[0025] (1) Through object tracking masking, the present invention eliminates the excessive attention of current RFMs to task-irrelevant features and improves the robustness of the RFMs inference results;

[0026] (2) The present invention uses the ratio of the original observation image to the action prediction result of the observation image after masking the target object as the final prediction result of the model; during the inference process, it redirects the attention of the RFMs to the instruction-related visual cues to eliminate the spurious correlations in visual observations. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic flowchart of the method of the present invention.

[0028] Figure 2 It is a schematic diagram of the method framework of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0030] A debiasing method for inference of a robot base large model based on policy contrast decoding includes the following steps:

[0031] Step 1: Obtain the behavior trajectory of the robot base large model;

[0032] Step 2: Mask the target object in the visual observation image o s in the behavior trajectory to obtain the masked visual observation image S is the maximum time step; the object tracking mask method Track2Mask is used for masking. The robot base large model is a robot base large model based on a diffusion policy. (It can also be applicable to the RFM for autoregressive prediction)

[0033] Step 3: Calculate the action probabilities p s of the visual observation image o and the masked visual observation image t and t is the action dimension serial number, t = 1,..., M, M is the action dimension; when the robot base large model is the RFM for autoregressive prediction, the action probability can be directly calculated; when the robot base large model is a robot base large model based on a diffusion policy, kernel density estimation is used to calculate the action probability of the robot base large model based on a diffusion policy.

[0034] The action probability calculation process is as follows:

[0035] Sample N noise vectors from a Gaussian distribution;

[0036] Parallel denoise the noise vectors through the reverse diffusion process to generate N candidate action prediction values a s (i) = [a1(i),..., a M (i)];

[0037] Action a s The t-th action dimension a t The action probability distribution is as follows:

[0038]

[0039] Where: is the Gaussian kernel, and b is the bandwidth parameter controlling the distribution smoothness;

[0040] The action probability can be obtained by calculating the action probabilities of M action dimensions in sequence.

[0041] Step 4: Calculate the comparison action probability distribution according to the action probability p t and Calculate the comparison action probability distribution Take the maximum value in the comparison action probability distribution to obtain the action corresponding to the maximum action probability, and execute the action under this condition. Calculate the comparison action probability distribution through the policy comparison decoding method; the comparison action probability distribution The calculation process is as follows:

[0042]

[0043] Where: α is the modulation coefficient, which is used to control the intervention degree of the comparison image.

[0044] As Figure 2 shown, the present invention first designs an object tracking mask method Track2Mask to accurately mask all visual observation images in the behavior trajectory at the lowest cost where the target object in it, and T represents the total number of steps. This strategy marks the target object for the input visual observation o i ∈O of the behavior trajectory through a human or expert model (such as the open vocabulary segmentation model: SED). Then, use the SAM2 model to track the target object in the visual observation images of different path points. In the inference stage, accurately mask the target object appearing in the visual observation image at the lowest human marking cost or without human participation to obtain

[0045] Taking the embodied large model OpenVLA as an example, based on the visual observation o i ∈O, the output action space is [x, y, z, x rot , y rot , z rot , s], where: [x, y, z], [x rot , y rot , z rot and [s] are the coordinates, rotation angle and closing state of the end effector of the robotic arm respectively.

[0046] Let M {θ,ω}represents the implementation of a large model after small sample learning, where θ is the pre-trained large model parameter and ω is the target task adapter parameter learned during the training phase. Let 1 represent the task language instruction. OpenVLA uses an autoregressive method to i ∈O to perform action prediction, the result is P = [p x ,...,p s ],in:

[0047]

[0048] Similarly, OpenVLA is used to observe the visual The action prediction result is expressed as:

[0049]

[0050] The model is applied to the original observation image o i And the observed image after masking the target object The ratio of the action prediction results is taken as the final prediction result.

[0051] By forcing the prediction results to be biased towards actions with higher probability values in the comparison action probability distribution.

[0052] Diffusion-based models cannot directly generate action probability distributions. We introduce a KDE-based probabilistic modeling (KDE-PM) scheme that computes the action probability distribution of diffusion-based RFMs using kernel density estimation (KDE). This invention can be used as a plug-in to improve the reasoning of RFMs. It can be directly applied to large embodied models based on autoregressive prediction (such as OpenVLA, RT-1, RT-2, RT-X), and can also be extended to large embodied models based on diffusion prediction (such as Octo, π-0, RDT) by leveraging KDE.

[0053] Example

[0054] like Figure 1 The specific reasoning process is as follows:

[0055] Input: RFM:π θ (a s |o s ,l), where the action Initial observation value o0; language instruction l; maximum time step S.

[0056] S1:s←0;

[0057] S2: Obtain observation value o through object tracking mask strategy (Track2Mask) s Masked observations

[0058] S3: Generate o using the KDE-PM method in RFM based on the diffusion strategy s The action probabilities [p1,..., p M and The action probabilities

[0059] RFMs based on the diffusion strategy construct action generation as an iterative denoising process. Let e s represent the multimodal embedding that fuses o s and l. Starting from Gaussian noise the reverse diffusion process optimizes the M dimensions of a s in parallel within K steps as follows:

[0060]

[0061] where is the noisy action at the k-th diffusion step; represents the denoising network; α, γ, σ are the parameters of the cosine noise schedule.

[0062] The action probability calculation process is as follows:

[0063] The model based on the diffusion strategy samples N noise vectors from the Gaussian distribution; where

[0064] Denoise the noise vectors in parallel through the reverse diffusion process to generate N candidate action prediction values a s (i) = [a1(i),..., a M (i)];

[0065] For the t-th action dimension a s of action a t the action probability distribution is:

[0066]

[0067] In the formula: is the Gaussian kernel b is the bandwidth parameter that controls the smoothness of the distribution; is the N candidate actions of the t-th dimension of action a s generated by the diffusion process.

[0068] Calculate the action probabilities of the M action dimensions in sequence to obtain the action probabilities [p1,..., p M .

[0069] The action probabilities The calculation method is as follows s The action probabilities [p1,..., p M . Calculation method

[0070] S4: Calculate the contrast action probability distribution through the measurement and comparison decoding method

[0071]

[0072] Where: α is the modulation coefficient, used to control the intervention degree of the contrast image. The larger the α value, the higher the amplification degree of the difference between the two distributions. When α = 0, it is simplified to the conventional prediction

[0073] Obtain the contrast action probability distribution, then select the action corresponding to the largest value among them and execute this action

[0074] S5: Based on Sample the action a s =[a1,..., a M ; (S5 - S9 is the resampling process, and the continuous inference debiasing process is thus realized)

[0075] S6: Execute the action a s =[a1,..., a M ;

[0076] S7: Obtain the new observation value o s+1 ;

[0077] S8: s ← s + 1;

[0078] S9: If s ≤ S, then return to execute from S2, otherwise the algorithm ends

[0079] To illustrate the effect of the method of the present invention, it is combined with the current advanced RFM based on autoregressive prediction - OpenVLA - 7b. As can be seen from the results, the success rate of the model in the three tasks of close_drawer, open_drawer, and pick_coke_can is significantly improved as shown in Table 1

[0080] Table 1. Comparison between the method of the present invention and the existing method

[0081]

[0082] As can be seen from Table 1, compared with the existing OpenVLA - 7b model, the success rate of the method of the present invention in the three tasks has been greatly improved

[0083] The present invention adopts an object tracking mask strategy. By using class expertise or off-the-shelf models (such as SED) to annotate target objects in the initial visual observation, and then adopting the SAM2 model to track these objects in subsequent observations in the trajectory. Precise object masking is achieved, greatly reducing manual intervention. By comparing the action prediction results of the RFMs for the original observation image and the observation image after masking the target object, a new comparative action probability distribution is obtained, and based on this, the attention of the RFMs is redirected to instruction-related visual cues during the inference process, thereby eliminating false correlations in visual observations and enhancing the inference robustness of the RFMs. Further, the action probability distribution of the RFMs based on the diffusion strategy is calculated through kernel density estimation, enhancing the inference result robustness of various types of RFMs.

Claims

1. A method for debiasing the inference of a large robot foundation model based on policy contrast decoding, characterized in that It includes the following steps: Step 1: Obtain the behavioral trajectory of the robot base large model; Step 2: Mask the target object in the visual observation image o in the behavior trajectory to obtain the masked visual observation image s ; S is the maximum time step; Step 3: Calculate the action probabilities p s of the visual observation image o and the masked visual observation image t respectively, where t is the action dimension index, t = 1,..., M, and M is the action dimension; Step 4: According to the action probability p t and calculate the comparison action probability distribution Take the maximum value in the comparison action probability distribution to obtain the action corresponding to the maximum action probability, and execute the action under this condition.

2. The method for debiasing the inference of a robot basic large model based on policy comparison decoding according to claim 1, characterized in that In step 2, the object tracking mask method Track2Mask is used for masking.

3. A method for debiasing the inference of a robot basic large model based on policy comparison decoding according to claim 1, characterized in that The robot base large model is a robot base large model based on a diffusion strategy.

4. A method for debiasing the inference of a robot's basic large model based on policy contrast decoding according to claim 3, characterized in that, In step 3, the kernel density estimation calculates the action probability of the robot base large model based on the diffusion strategy.

5. A method for debiasing the inference of a robot's basic large model based on policy comparison decoding, characterized in that, The calculation process of the action probability is as follows: Sample N noise vectors from the Gaussian distribution; Parallel denoising of the noise vector through the reverse diffusion process to generate N candidate action prediction values Action a s The t-th action dimension a of t The action probability distribution is as follows: In the formula: is the Gaussian kernel, and b is the bandwidth parameter that controls the smoothness of the distribution; The action probability can be obtained by calculating the action probabilities of M action dimensions in sequence.

6. The method for debiasing the inference of a robot basic large model based on policy contrast decoding according to claim 5, wherein In step 4, the contrast action probability distribution is calculated by the policy contrast decoding method.

7. A method for debiasing the inference of a robot basic large model based on policy comparison decoding, characterized in that The probability distribution of the comparison action The calculation process is as follows: In the formula: α is the modulation coefficient.

Images