Mechanical arm reasoning grabbing method and device based on anti-noise enhanced world model

Abstract

The invention discloses a mechanical arm reasoning grabbing method and device based on an anti-noise enhanced world model, and relates to the field of robot intelligent control. Although an existing large language model-assisted grabbing frame can infer a fuzzy instruction, the grabbing frame is easily interfered by noise in a complex scene. Therefore, the method comprises the following steps: firstly, constructing an information filtering world model, introducing a gating representation decomposition module and a contrast causal gating module, decomposing a multi-modal perception vector into a main effect component and a noise component, screening key dimensions related to the future under an action condition, and performing multi-step prediction in a cyclic state space; secondly, designing an information value constraint anti-noise reinforcement learning mechanism, optimizing and maximizing the mutual information of future and useful information and minimum and useless information through a value strategy, and inhibiting the interference of perception noise on decision optimization; according to the method, imagination reasoning of a world model and decision optimization of reinforcement learning are combined, noise interference is restrained and restrained through information value, and the robustness of the mechanical arm reasoning grabbing process is enhanced.

Description

Technical Field

[0001] This invention relates to the field of intelligent robot control, specifically to a method and apparatus for robotic arm reasoning and grasping based on a noise-resistant enhanced world model. Background Technology

[0002] In recent years, with the rapid development of intelligent manufacturing, warehousing and logistics, and home service robots, the robustness and adaptability of robotic arm autonomous grasping technology in complex dynamic environments have become key challenges. While existing grasping methods based on large language models possess certain command reasoning and generalization capabilities, their perception and decision-making processes are susceptible to irrelevant noise in real-world scenarios with sensor noise, lighting changes, occlusion, and dynamic interference, leading to a significant decrease in grasping success rate. Traditional grasping systems often rely on manual features or end-to-end reinforcement learning, lacking explicit modeling of the causal structure and key task information in the scene, making it difficult to achieve accurate state prediction and motion planning in noisy environments. Summary of the Invention

[0003] The purpose of this application is to propose a robotic arm reasoning and grasping method and device based on a noise-resistant enhanced world model to address the aforementioned technical problems.

[0004] In a first aspect, the present invention provides a robotic arm reasoning and grasping method based on a noise-resistant enhanced world model, comprising the following steps:

[0005] An information filtering world model, a value network, and a policy network are constructed. The information filtering world model includes an encoder, a gating representation decomposition module, a contrastive causal gating module, and a decoder. The information filtering world model, the value network, and the policy network are trained to obtain the trained information filtering world model, the trained value network, and the trained policy network.

[0006] The system acquires the fuzzy commands issued by the user at the current time step and inputs them into the BidirPR-Grasp framework to obtain the masked RGB image, masked depth map, and target item availability map for the current time step. Combined with the joint activity data of the robotic arm collected at the current time step, it constructs the first multimodal perception vector for the current time step. The first multimodal perception vector, action, and latent state of the current time step are then input into a trained information filtering world model. The first multimodal perception vector and latent state of the current time step are first processed by an encoder to obtain the representation vector of the current time step. The representation vector of the current time step is then input into a gating table. The feature decomposition module filters out the principal component and noise component of the representation vector at the current time step; the principal component of the representation vector at the current time step is input into the decoder to obtain the observation vector at the current time step; the observation vector, action, and the principal component and noise component of the representation vector at the current time step are input into the contrastive causal gating module to obtain the final principal representation at the current time step; the hidden state, action, and final principal representation at the current time step are recursively updated using a state transition function to obtain the hidden state at the next time step; the hidden state at the next time step is input into the trained policy network to obtain the action at the next time step.

[0007] Based on the motion control of the next time step, the robotic arm executes the grasping action of the next time step.

[0008] As a preferred embodiment, the gated representation decomposition module includes a dimension-wise gated network and an element-wise multiplication operation, and its calculation process is as follows:

[0009] ;

[0010] ;

[0011] ;

[0012] in, Indicates the first A gated vector for each time step. Indicates the first output of the encoder The representation vector at each time step, Represents the space of real numbers. Indicates dimension; This represents the Sigmoid activation function. This represents element-wise multiplication; This represents the weight matrix of a dimensionally gated network. Represents the bias vector of a dimension-wise gated network; and They represent the first The representation vector of each time step The main effect component and the noise component.

[0013] As a preferred embodiment, the comparative causal gating module includes an action sequence compressor, a dimension embedding module, a query-key pair construction module, and a gating weighting module, and its calculation process is as follows:

[0014] According to the The predicted action, predicted latent state, and predicted representation vector at the nth time step are recursively predicted using the state transition function. The predicted latent state and predicted representation vector at the nth time step, and the nth time step... The predicted latent state and predicted representation vector at the i-th time step are input into the policy network to obtain the i-th time step. For the predicted action at the nth time step, repeat the above steps until the nth time step is obtained. Predict the action at the nth time step, and the nth time step. The time step to the 1 The predicted action at the nth time step is constructed into the nth time step. The future at a time step Action sequence at each time step As shown in the following formula:

[0015] ;

[0016] ;

[0017] in, , and when hour, , , , Indicates the first The implicit state at each time step Indicates the first The action at each time step; Represents the policy network, with the parameter set as follows: ; Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Represents the state transition function. The set of parameters representing the state transition function;

[0018] The first The future at a time step The action sequence at time step n is input into the action sequence compressor and compressed to obtain the action sequence at time step n. Action condition vectors for each time step;

[0019] The first The principal effective component of the representation vector at the nth time step is input into the dimension embedding module for dimension adjustment, resulting in the nth time step. The principal components of the adjusted representation vector at each time step;

[0020] The first The action condition vector at each time step, the principal component of the adjusted representation vector, the observation vector, and the noise component of the representation vector are input into the query-key pair construction module to obtain the first time step. The query vector, principal key vector, and noise key vector for each time step are shown in the following formula:

[0021] ;

[0022] ;

[0023] ;

[0024] in, Indicates the first Action condition vectors at each time step This represents the compression function corresponding to the action sequence compressor. Indicates splicing; Indicates the first The principal effective components of the adjusted representation vector at each time step This indicates the function corresponding to the dimension embedding module; Indicates the first The principal component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step Represents a multilayer perceptron; and They represent the first The principal key vector and noise key vector at each time step;

[0025] The first The query vector, principal key vector, and noise key vector at each time step are input into the gated weighting module. First, based on the query vector, principal key vector, and noise key vector at each time step... The query vector, principal key vector, and noise key vector at the nth time step are calculated to obtain the nth time step. The contrastive log-likelihood vector for each time step is shown in the following equation:

[0026] ;

[0027] in, Indicates the first The contrastive log-likelihood vector at each time step This indicates temperature hyperparameters. Indicates the first The noise key vector at each time step. , Represents the similarity function;

[0028] According to the The comparison log-likelihood vector at the nth time step is calculated to obtain the th... The controllability score vector for each time step is shown in the following formula:

[0029] ;

[0030] in, Indicates the first The expectation at each time step Indicates the first Controllability score vector for each time step;

[0031] According to the The controllability score vector at the nth time step is calculated to obtain the first... The dimension-level gated vector at the nth time step, and for the nth time step We weight the principal effective components of the representation vector at the nth time step to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation:

[0032] ;

[0033] ;

[0034] in, Indicates the benchmark for the shift in scoring. Represents the temperature coefficient of the gated function; Indicates the first A dimensional gated vector at each time step; Indicates the first The final main effect characterization at each time step.

[0035] As a preferred option, the information filtering world model is constructed based on a cyclic state-space model. The cyclic state-space model includes a filtering system, an observation system, a dynamic system, and a reward system. In the filtering system, the information filtering system incorporates the information generated during the reasoning phase. The first multimodal sensing vector at the nth time step, or the vector generated during training by the nth time step. The first time step is constructed from the masked RGB image, the masked depth map, and the target item availability map. The second multimodal sensing vector at the nth time step is sequentially processed by the encoder and the gated representation decomposition module to obtain the nth time step. The principal component and noise component of the representation vector at each time step are shown in the following equation:

[0036] ;

[0037] in, Indicates the first The first or second multimodal sensing vector at each time step The parameter set is The encoder, This indicates the gated representation decomposition module;

[0038] The first The principal effective component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step is shown in the following equation:

[0039] ;

[0040] in, The parameter set is The decoder;

[0041] The first The observation vector, action, and principal component and noise component of the representation vector at the nth time step are input into the contrastive causal gating module to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation:

[0042] ;

[0043] in, This indicates a comparison of causal gating modules;

[0044] According to the The implicit state, action, and final principal effect representation at the nth time step are recursively updated using a state transition function to obtain the nth time step. The hidden state at each time step As shown in the following formula:

[0045] ;

[0046] During the training phase, the first The latent state and final main effect representation at the nth time step are input into the reward function to obtain the nth time step. The reward for each time step is shown in the following formula:

[0047] ;

[0048] in, The parameter set is The reward function, Indicates the first Rewards for each time step.

[0049] As a preferred approach, the training process for the information filtering world model, value network, and policy network is divided into two stages:

[0050] In the first stage, the information filtering world model is trained separately based on the comprehensive loss function to obtain the trained information filtering world model; the construction process of the comprehensive loss function is as follows:

[0051] The mean square error is used to measure the first step of the decoder output. The difference between the observation vector and the principal effective component of the representation vector at each time step is used to obtain the observation reconstruction loss, as shown in the following equation:

[0052] ;

[0053] in, Indicates the observation reconstruction loss;

[0054] Calculate the encoder output of the first The KL divergence between the posterior distribution of the hidden states at each time step and the standard Gaussian prior distribution is used to obtain the KL divergence regularization loss, as shown in the following equation:

[0055] ;

[0056] in, Indicates the first output of the encoder The posterior distribution of the representation vectors at each time step. This represents the standard Gaussian prior distribution. The KL divergence represents the distance between two probability distributions. This represents the KL divergence regularization loss;

[0057] According to the The decomposition loss of the gated representation decomposition module is calculated by combining the principal component and noise component of the representation vector at each time step, as shown in the following equation:

[0058] ;

[0059] in, This represents the decomposition loss of the gated representation decomposition module;

[0060] According to the The contrast loss of the contrastive log-likelihood vector at each time step is calculated for the contrastive causal gating module, as shown in the following formula:

[0061] ;

[0062] in, This represents the contrast loss of the contrast causality gating module;

[0063] The comprehensive loss function is constructed as shown in the following equation:

[0064] ;

[0065] in, Represents the comprehensive loss function. These are the weight coefficients corresponding to the observation reconstruction loss, KL divergence regularization loss, reward prediction loss, decomposition loss of the gated representation decomposition module, and contrast loss of the contrastive causal gating module, respectively.

[0066] In the second stage, the parameters of the trained information filtering world model are fixed, and information value-constrained noise-resistant reinforcement learning is used to train the value network and policy network respectively based on the total loss function, resulting in trained value networks and trained policy networks; the construction process of the total loss function is as follows:

[0067] First, construct the objective function for the λ-Return multi-step return, as shown in the following equation:

[0068] ;

[0069] ;

[0070] in, This represents the balance coefficient of the λ-Return multi-step return. Indicates the discount factor; Indicates the first The target value of the λ-Return multistep return at the time step is determined by the λ-Return multistep return at the λ-Return time step. The reward at the first time step and the first The value is obtained by recursively calculating the value after the hidden state at each time step is input into the value network. Indicates the first The target value of the multi-step λ-Return return at each time step; The parameter set is The value network Indicates the first The value obtained by inputting the hidden state at each time step into the value network; H represents the maximum time step of the λ-Return multi-step reward. This represents the target value of the multi-step return corresponding to the maximum time step;

[0071] Then, the training loss of the value network is constructed as shown in the following formula:

[0072] ;

[0073] in, This represents the training loss of the value network. This represents the Huber loss function. For threshold hyperparameters, This represents the weight hyperparameter of the Bellman consistency regularization. This represents the squared residual of the first-step TD. Indicates the first The value obtained by inputting the hidden state of each time step into the value network;

[0074] Next, the training loss of the policy network is constructed as shown in the following formula:

[0075] ;

[0076] in, This represents the training loss of the policy network; Indicates the first The hidden state at each time step Next strategy network selection Actions at each time step The probability, Indicates the first The hidden state at each time step The probability distribution of the next policy network choosing any action; Represents the hyperbolic tangent function; Indicates the first The advantage function at each time step This indicates that the gradient operation is stopped. This indicates the scaling hyperparameters. Represents the entropy regularity coefficient; Entropy represents the policy distribution;

[0077] Subsequently, the information compression decoupling contrast loss is constructed as shown in the following equation:

[0078] ;

[0079] in, This indicates the information compression and decoupling contrast loss; Represents mutual information; The weight hyperparameters representing the information bottleneck term. This represents the balance coefficient hyperparameter. This represents the noise weighting hyperparameter. This represents the contrastive learning weight hyperparameters; Indicates the first The observation vectors at each time step are input into the parameter set as follows: Summary encoder The compressed future summary; The InfoNCE loss under the denoising invariance task is expressed as follows:

[0080] ;

[0081] in, Indicates the first Each time step This represents the total number of time steps in the training phase.

[0082] Finally, the total loss function is constructed as shown in the following equation:

[0083] ;

[0084] in, Represents the total loss function; represents the weight hyperparameters of the training loss of the value network, the training loss of the policy network, and the information compression decoupling contrast loss, respectively.

[0085] Preferably, the process of constructing the first or second multimodal sensing vector at the current time step includes:

[0086] The masked RGB image, masked depth image, and target item availability map at the current time step are preprocessed and concatenated along the channel dimension to obtain an image vector. The preprocessing methods include spatial dimension alignment, mask consistency verification and repair, feature standardization, and dimension and channel adaptation. The image vector is used as the second multimodal perception vector.

[0087] The process of constructing the first multimodal sensing vector at the current time step also includes:

[0088] The joint activity data of the robotic arm at the current time step is standardized for features, adapted for dimensions and channels, and concatenated for channel dimensions to obtain the joint activity feature vector. The joint activity data includes joint angles, angular velocities, and / or torques.

[0089] The first multimodal sensing vector is obtained by concatenating the image vector with the joint activity feature vector.

[0090] Secondly, the present invention provides a robotic arm reasoning and grasping device based on a noise-resistant enhanced world model, comprising:

[0091] The model building module is configured to build an information filtering world model, a value network, and a policy network. The information filtering world model includes an encoder, a gating representation decomposition module, a contrastive causal gating module, and a decoder. The information filtering world model, the value network, and the policy network are trained to obtain a trained information filtering world model, a trained value network, and a trained policy network.

[0092] The action prediction module is configured to acquire the fuzzy commands issued by the user at the current time step and input them into the BidirPR-Grasp framework to obtain a masked RGB image, a masked depth map, and a target object availability map for the current time step. Combined with the joint activity data of the robotic arm collected at the current time step, a first multimodal perception vector for the current time step is constructed. This first multimodal perception vector, action, and latent state are then input into a trained information-filtering world model. The first multimodal perception vector and latent state for the current time step are first processed by an encoder to obtain a representation vector for the current time step. The input is fed into the gated representation decomposition module, which filters to obtain the principal component and noise component of the representation vector at the current time step. The principal component of the representation vector at the current time step is then fed into the decoder to obtain the observation vector at the current time step. The observation vector, action, and the principal and noise components of the representation vector at the current time step are then fed into the contrastive causal gating module to obtain the final principal representation at the current time step. Based on the hidden state, action, and final principal representation at the current time step, a state transition function is used for recursive updates to obtain the hidden state at the next time step. The hidden state at the next time step is then fed into the trained policy network to obtain the action at the next time step.

[0093] The execution module is configured to control the robotic arm to perform the grasping action of the next time step based on the motion of the next time step.

[0094] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0095] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0096] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0097] Compared with the prior art, the present invention has the following beneficial effects:

[0098] (1) The robotic arm reasoning grasping method based on the noise-resistant enhancement world model proposed in this invention constructs an information filtering world model, and achieves noise suppression and causal enhancement through the collaborative action of the gating representation decomposition module and the contrastive causal gating module. The gating representation decomposition module uses sparse gating mechanism and dimension-wise filtering to explicitly separate the task-related main component and the noise component that is irrelevant interference from the representation vector; the contrastive causal gating module strengthens the key dimensions that have causal contributions to future prediction in the hidden state through cross-temporal contrastive learning under action conditions, which significantly improves the robust perception ability of the model under occlusion, illumination change and sensor noise conditions, and solves the problems of low grasping success rate caused by perception redundancy and insufficient policy coupling in traditional grasping methods under noise interference and dynamic environment.

[0099] (2) The robotic arm reasoning and grasping method based on the noise-resistant reinforcement world model proposed in this invention proposes an information value-constrained noise-resistant reinforcement learning mechanism, which breaks through the limitations of traditional reinforcement learning in terms of large fluctuations in value estimation and unstable policy convergence in noisy environments. This mechanism integrates λ-Return multi-step reward estimation, Huber robust regression, Bellman consistency constraint, and information theory denoising to achieve smooth modeling of long-term rewards, preservation of the time consistency of the value function, and bounded optimization of policy gradients. Through mutual information minimization and maximization operations, it explicitly constrains the sensitivity of the policy learning process to past redundant information and future noise, thereby enhancing the model's ability to discriminate and utilize key task features.

[0100] (3) The robotic arm reasoning and grasping method based on the noise-resistant reinforcement world model proposed in this invention achieves a significant improvement in perception robustness in robotic arm reasoning and grasping tasks through the collaborative design and deep integration of the information filtering world model and the information value constraint noise-resistant reinforcement learning mechanism, providing a novel and effective solution for the reliable operation of embodied intelligent systems in complex real-world scenarios. Attached Figure Description

[0101] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0102] Figure 1 This is a flowchart illustrating the robotic arm reasoning and grasping method based on a noise-resistant enhanced world model, as an embodiment of this application.

[0103] Figure 2This is a schematic diagram illustrating the calculation process of the information filtering world model in the robotic arm reasoning and grasping method based on the noise-resistant and enhanced world model, which is an embodiment of this application.

[0104] Figure 3 This is a schematic diagram illustrating the calculation process of the gated representation decomposition module of the robotic arm reasoning and grasping method based on the noise-resistant enhanced world model, which is an embodiment of this application.

[0105] Figure 4 This is a schematic diagram of the calculation process of the comparative causal gating module in the robotic arm reasoning and grasping method based on the noise-resistant enhanced world model, which is an embodiment of this application.

[0106] Figure 5 This is a schematic diagram illustrating the construction process of the total loss function of the information value-constrained noise-resistant reinforcement learning in the robotic arm reasoning and grasping method based on the noise-resistant reinforcement world model, which is an embodiment of this application.

[0107] Figure 6 This is a schematic diagram of a robotic arm reasoning and grasping device based on a noise-resistant enhanced world model, which is an embodiment of this application.

[0108] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0109] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0110] Figure 1 An embodiment of this application illustrates a robotic arm reasoning and grasping method based on a noise-resistant enhanced world model, comprising the following steps:

[0111] S1. Construct an information filtering world model, a value network, and a policy network. The information filtering world model includes an encoder, a gating representation decomposition module, a contrastive causal gating module, and a decoder. Train the information filtering world model, the value network, and the policy network to obtain the trained information filtering world model, the trained value network, and the trained policy network.

[0112] Specifically, the hardware system corresponding to the robotic arm reasoning and grasping method based on the noise-resistant enhanced world model proposed in the embodiments of this application is built on a mature robotic arm grasping platform, which includes:

[0113] The execution unit includes the JAKA C7 six-axis collaborative robotic arm and its matching two-finger planar gripper;

[0114] The sensing unit includes an Orbbec Gemini 2 depth camera, which is fixedly mounted on the flange at the end of the robotic arm to acquire RGB images and depth maps of the working area in real time at a resolution of 480×640.

[0115] The interaction unit includes a programmable microphone array for receiving natural language commands from the user.

[0116] The computing unit consists of a local workstation configured with a CPU, GPU, and 32GB of RAM, running a Windows operating system. All model inference, information filtering world model prediction, and reinforcement learning decision-making processes are performed on this local workstation.

[0117] The embodiments of this application enable seamless integration with the upstream BidirPR-Grasp framework. The output of the BidirPR-Grasp framework serves as the input to the information filtering world model of the embodiments of this application, with the specific interface as follows:

[0118] The BidirPR-Grasp framework obtains masked RGB and masked depth maps during the visual localization stage. An affordance map centered on the optimal component keypoints is obtained during the affordance generation stage. The BidirPR-Grasp framework proposed in this application is prior art, based on content mentioned in the paper titled "Research on Robotic Arm Grasping Method Based on Large Language Model," and will not be elaborated upon here.

[0119] The second multimodal perception vector, which is formed by preprocessing and channel stitching the masked RGB image, the masked depth image, and the target item availability map, or the first multimodal perception vector, which is further constructed by combining the joint activity data of the robotic arm, is used as one of the inputs to the InfoFilter World Model (IFWM).

[0120] In a specific embodiment, the process of constructing the first multimodal sensing vector or the second multimodal sensing vector at the current time step includes:

[0121] The masked RGB image, masked depth image, and target item availability map at the current time step are preprocessed and concatenated along the channel dimension to obtain an image vector. The preprocessing methods include spatial dimension alignment, mask consistency verification and repair, feature standardization, and dimension and channel adaptation. The image vector is used as the second multimodal perception vector.

[0122] The process of constructing the first multimodal sensing vector at the current time step also includes:

[0123] The joint activity data of the robotic arm at the current time step is standardized for features, adapted for dimensions and channels, and concatenated for channel dimensions to obtain the joint activity feature vector. The joint activity data includes joint angles, angular velocities, and / or torques.

[0124] The first multimodal sensing vector is obtained by concatenating the image vector with the joint activity feature vector.

[0125] Specifically, the preprocessing and channel stitching steps for the masked RGB image, masked depth map, and target item availability map at the current time step in the embodiments of this application are as follows:

[0126] Step 1: Spatial Dimension Alignment, the specific process is as follows:

[0127] Resolution uniformity: Based on the resolution (H×W) of the masked RGB image output by the visual localization stage of the BidirPR-Grasp framework, the masked depth map and the target object availability map are resized and interpolated to ensure that the spatial dimensions of the three are completely consistent.

[0128] Coordinate benchmark alignment: Based on the coordinates of the optimal component key points output by BidirPR-Grasp, the target item availability map is fine-tuned by translation, cropping and other operations to ensure that its key areas completely overlap with the masked areas in the masked RGB map and the masked depth map, avoiding spatial misalignment that could lead to feature failure.

[0129] The second step, mask consistency verification and repair, is as follows:

[0130] Mask origin verification: Extract the mask channels of the masked RGB image and the masked depth image, and verify the intersection rate of the two masks. The intersection rate is used to quantify the degree of mask deviation. The lower the intersection rate, the greater the deviation. When the deviation is small, local inconsistencies can be eliminated through morphological restoration. When the deviation is large, a fallback mechanism such as mask removal or re-acquisition is required. If there is a slight deviation, the mask in the masked RGB image is used as a reference to perform morphological restoration on the mask in the masked depth image to ensure that the target areas of the two are completely consistent.

[0131] Enhanced Mask Application: The unified mask channel is applied to the target item availability map to ensure that the data retains only the effective features within the target item mask and removes background interference.

[0132] The third step is feature standardization, which is carried out as follows:

[0133] Pixel value normalization: First, for the masked RGB image: normalize the pixel values ​​of the 3 channels to the [0,1] range to eliminate the impact of pixel magnitude differences on subsequent feature fusion; Second, for the masked depth map: the depth value is the actual physical distance, and the range may fluctuate with the scene. The effective range of the depth map based on the BidirPR-Grasp output is adopted, that is, the minimum and maximum depth values ​​within the masked area, and the depth values ​​are mapped to the [0,1] range to retain the relative depth differences; Finally, for the target item availability map: its pixel value is the availability score, which is usually normalized to [0,1] by the BidirPR-Grasp framework, representing the probability that the position is suitable for the robotic arm action. If it is not normalized, the same normalization logic as the pixel value normalization is adopted to ensure that the magnitude is consistent with the first two types of data.

[0134] Step 4: Dimension and channel adaptation, the specific process is as follows:

[0135] Tensor format conversion: Converts three types of image data from 2D spatial matrices (H×W) to a 4D tensor format (B×C×H×W) compatible with deep learning models. Here, B represents the batch size (to meet the input requirements of IFWM, usually set to 1, i.e., single-frame input); C represents the number of channels (3 channels for masked RGB images, 1 channel for masked depth images, and 1 channel for target item availability maps).

[0136] Invalid channel removal: Check the channel dimensions of the three types of image data, remove any redundant masked channels that may exist in the masked RGB image, and retain only the redundant information of the R / G / B three channels and the masked depth map to ensure that the channels of each type of data are pure and avoid dimensional redundancy after stitching.

[0137] After the above preprocessing, the format of the three types of image data is unified into a tensor with dimensions B×C×H×W:

[0138] The dimensions of the masked RGB image are B×3×H×W;

[0139] The dimensions of the masked depth map are B×1×H×W;

[0140] The dimensions of the target item availability map are B×1×H×W.

[0141] By concatenating the channels, a second multimodal perception vector is finally generated, with dimensions B×(3+1+1)×H×W=B×5×H×W. This second multimodal perception vector can be directly used as input to IFWM during the training phase, realizing the integrated input of multimodal features, and then combined with actions at the same time step. and latent state The information is fed into the filtered world model to make accurate predictions of actions at the next time step.

[0142] In a specific embodiment, the gated representation decomposition module includes a dimension-wise gated network and an element-wise multiplication operation, and its calculation process is as follows:

[0143] ;

[0144] ;

[0145] ;

[0146] in, Indicates the first A gated vector for each time step. Indicates the first output of the encoder The representation vector at each time step, Represents the space of real numbers. Indicates dimension; This represents the Sigmoid activation function. This represents element-wise multiplication; This represents the weight matrix of a dimensionally gated network. Represents the bias vector of a dimension-wise gated network; and They represent the first The representation vector of each time step The main effect component and the noise component.

[0147] Specifically, based on the above input, the parameters of each module in the information filtering world model proposed in the embodiments of this application are constructed and initialized on a local workstation equipped with a GPU using the PyTorch framework.

[0148] Due to the limitation of 24GB of GPU memory on the local workstation, all large models need to be designed as lightweight versions, and the total number of parameters needs to be optimized to meet the requirements of real-time inference. The value network and policy network mentioned in the embodiments of this application adopt a medium-sized MLP structure.

[0149] The information filtering world model proposed in the embodiments of this application is used to perform multi-step simulation prediction of the grasping action of the target item, with reference to... Figure 2 , Figure 2Taking t=1, 2, 3 as an example, the same logic applies in subsequent time steps. The embodiments of this application propose an information filtering world model that introduces a gated representation decomposition module and a contrastive causal gated module. Using the latent states extracted from the information filtering world model as the environment, it obtains information favorable for grasping the target object from three types of image data: masked RGB images, masked depth maps, and target object availability maps. It evaluates the information gain of candidate grasping actions and selects the optimal action. Furthermore, it designs an information value-constrained noise-resistant reinforcement learning mechanism, improving perceptual robustness by designing future and past policy optimizations. The embodiments of this application combine the perceptual reasoning capabilities of the world model with the environmental modeling advantages of reinforcement learning. Through innovative noise-resistant theory of information bottlenecks, it enhances the robustness of the robotic arm in grasping target parts, making it suitable for various grasping task scenarios such as home robots.

[0150] refer to Figure 3 The gated representation disentanglement module (GRD) mentioned in the embodiments of this application mainly consists of a dimension-wise gated network and element-wise multiplication operations. First, the representation disentanglement module (GRD) obtained by the encoder... The representation vector of each time step Inputting a dimension-wise gated network, the 1st generation is obtained through linear transformation and Sigmoid activation. The gating vector at the nth time step, and then the gating vector at the th time step. The representation vector of each time step Weighting is applied to separate useful and useless information. In one embodiment, the encoder in this application uses a Transformer-type encoder; in other embodiments, the encoder in this application may also use a convolutional encoder.

[0151] In a specific embodiment, the comparative causal gating module includes an action sequence compressor, a dimension embedding module, a query-key pair construction module, and a gating weighting module, and its calculation process is as follows:

[0152] According to the The predicted action, predicted latent state, and predicted representation vector at the nth time step are recursively predicted using the state transition function. The predicted latent state and predicted representation vector at the nth time step, and the nth time step... The predicted latent state and predicted representation vector at the i-th time step are input into the policy network to obtain the i-th time step. For the predicted action at the nth time step, repeat the above steps until the nth time step is obtained. Predict the action at the nth time step, and the nth time step. The time step to the 1 The predicted action at the nth time step is constructed into the nth time step. The future at a time step Action sequence at each time step As shown in the following formula:

[0153] ;

[0154] ;

[0155] in, , and when hour, , , , Indicates the first The implicit state at each time step Indicates the first The action at each time step; Represents the policy network, with the parameter set as follows: ; Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Represents the state transition function. The set of parameters representing the state transition function;

[0156] The first The future at a time step The action sequence at time step n is input into the action sequence compressor and compressed to obtain the action sequence at time step n. Action condition vectors for each time step;

[0157] The first The principal effective component of the representation vector at the nth time step is input into the dimension embedding module for dimension adjustment, resulting in the nth time step. The principal components of the adjusted representation vector at each time step;

[0158] The first The action condition vector at each time step, the principal component of the adjusted representation vector, the observation vector, and the noise component of the representation vector are input into the query-key pair construction module to obtain the first time step. The query vector, principal key vector, and noise key vector for each time step are shown in the following formula:

[0159] ;

[0160] ;

[0161] ;

[0162] in, Indicates the first Action condition vectors at each time step This represents the compression function corresponding to the action sequence compressor. Indicates splicing; Indicates the first The principal effective components of the adjusted representation vector at each time step This indicates the function corresponding to the dimension embedding module; Indicates the first The principal effective component of the representation vector at time step n is input into the decoder to obtain the nth time step. The observation vector at each time step Represents a multilayer perceptron; and They represent the first The principal key vector and noise key vector at each time step;

[0163] The first The query vector, principal key vector, and noise key vector at each time step are input into the gated weighting module. First, based on the query vector, principal key vector, and noise key vector at each time step... The query vector, principal key vector, and noise key vector at the nth time step are calculated to obtain the nth time step. The contrastive log-likelihood vector for each time step is shown in the following equation:

[0164] ;

[0165] in, Indicates the first The contrastive log-likelihood vector at each time step This indicates temperature hyperparameters. Indicates the first The noise key vector at each time step. , Represents the similarity function;

[0166] According to the The comparison log-likelihood vector at the nth time step is calculated to obtain the th... The controllability score vector for each time step is shown in the following formula:

[0167] ;

[0168] in, Indicates the first The expectation at each time step Indicates the first Controllability score vector for each time step;

[0169] According to the The controllability score vector at the nth time step is calculated to obtain the first... The dimension-level gated vector at the nth time step, and for the nth time step We weight the principal effective components of the representation vector at the nth time step to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation:

[0170] ;

[0171] ;

[0172] in, Indicates the benchmark for the shift in scoring. Represents the temperature coefficient of the gated function; Indicates the first A dimensional gated vector at each time step; Indicates the first The final main effect characterization at each time step.

[0173] For details, please refer to Figure 4 The Contrastive Causal Gate Module (CCG) mentioned in the embodiments of this application mainly consists of an action sequence compressor, a dimension embedding module, a query-key pair construction module, and a gate weighting module. First, the principal component of the representation vector at the current time step is input into the dimension embedding module, and the action sequence from the current time step to several future steps is compressed into an action condition vector to capture the regulatory effect of actions on future states. Then, the principal component of the adjusted representation vector and the action condition vector are input into a multilayer perceptron to generate a query vector, which represents the predictive potential of the principal component of the adjusted representation vector for each dimension under given action conditions. Next, the noise components of the observation vector and the representation vector are mapped to the principal key vector and interference key vector, respectively. A controllability score vector for each dimension is calculated using contrastive loss, reflecting the contribution of that dimension's features to the future state. Finally, a temperature coefficient is used to convert the controllability score vector into a dimension-level gate vector to achieve weighted selection for each dimension. Then, the dimension-level gate vector is multiplied element-wise with the principal effect component to obtain the final principal effect representation, ensuring that the information of the key dimensions is preserved and the noise dimension is suppressed.

[0174] The first The latent states and representation vectors at each time step are used as conditional inputs. Actions are generated through a policy network and then expanded in a multi-step forward pass through the information filtering world model to construct the future. The sequence of actions in a step. Specifically, the policy network. With the first The hidden state and representation vector at each time step As input, generate the first Actions at each time step and the first Action input state transition function at each time step The policy network recursively predicts the hidden states for subsequent time steps; based on each predicted hidden state, it further generates the action for the next time step, until a sequence of lengths is formed. The action sequence obtained from this. This represents the future action trajectory jointly predicted by the policy network and the information filtering world model under the current state conditions. As the action condition input for the comparative causal gating module, this action sequence needs to be compressed through concatenation, weighted summation, and a mapping network to form an action condition vector. It is used to characterize the information on how actions affect state evolution over a future period of time under a given policy assumption, and participates in subsequent controllability assessment and gating weighted calculation.

[0175] In a specific embodiment, the information filtering world model is constructed based on a cyclic state-space model, which includes a filtering system, an observation system, a dynamic system, and a reward system. In the filtering system, the first... The first multimodal sensing vector at the nth time step, or the vector generated during training by the nth time step. The first time step is constructed from the masked RGB image, the masked depth map, and the target item availability map. The second multimodal sensing vector at the nth time step is sequentially processed by the encoder and the gated representation decomposition module to obtain the nth time step. The principal component and noise component of the representation vector at each time step are shown in the following equation:

[0176] ;

[0177] in, Indicates the first The first or second multimodal sensing vector at each time step The parameter set is The encoder, This indicates the gated representation decomposition module;

[0178] The first The principal effective component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step is shown in the following equation:

[0179] ;

[0180] in, The parameter set is The decoder;

[0181] The first The observation vector, action, and principal component and noise component of the representation vector at the nth time step are input into the contrastive causal gating module to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation:

[0182] ;

[0183] in, This indicates a comparison of causal gating modules;

[0184] According to the The implicit state, action, and final principal effect representation at the nth time step are recursively updated using a state transition function to obtain the nth time step. The hidden state at each time step As shown in the following formula:

[0185] ;

[0186] During the training phase, the first The latent state and final main effect representation at the nth time step are input into the reward function to obtain the nth time step. The reward for each time step is shown in the following formula:

[0187] ;

[0188] in, The parameter set is The reward function, Indicates the first Rewards for each time step.

[0189] Specifically, the information filtering world model mentioned in the embodiments of this application is based on the cyclic state-space model (RSSM), which consists of four parts, including:

[0190] The filtering system consists of two parts: First, the first or second multimodal sensing vector and the latent state at the current time step are fed into the encoder to obtain the representation vector, which is then sequentially input into the gated representation decomposition module and the contrastive causal gating module to further suppress irrelevant noise and enhance the causality of the state representation. Second, the gated representation decomposition module performs adaptive dimensionality filtering on the representation vector to decompose the noise component for subsequent noise suppression constraint training.

[0191] Observation system: The observation vector is constructed based on the principal component of the representation vector at the current time step.

[0192] Dynamical system: The implicit state and action of the current time step, along with the final principal effect representation after CCG filtering, are input into the dynamical system. The implicit state is recursively updated through the state transition function to obtain the implicit state of the next time step.

[0193] Reward system: The reward for the current time step is estimated synchronously based on the principal component and the implicit state of the current time step through the reward function.

[0194] It should be noted that the information filtering world model of the embodiments of this application needs to use the above-mentioned filtering system, observation system, dynamic system and reward system repeatedly during the training phase, while only the above-mentioned filtering system, observation system and dynamic system need to be used during the inference phase.

[0195] In a specific implementation, the training process for the information filtering world model, the value network, and the policy network is divided into two stages:

[0196] In the first stage, the information filtering world model is trained separately based on the comprehensive loss function to obtain the trained information filtering world model; the construction process of the comprehensive loss function is as follows:

[0197] The mean square error is used to measure the first step of the decoder output. The difference between the observation vector and the principal effective component of the representation vector at each time step is used to obtain the observation reconstruction loss, as shown in the following equation:

[0198] ;

[0199] in, Indicates the observation reconstruction loss;

[0200] Calculate the encoder output of the first The KL divergence between the posterior distribution of the hidden states at each time step and the standard Gaussian prior distribution is used to obtain the KL divergence regularization loss, as shown in the following equation:

[0201] ;

[0202] in, Indicates the first output of the encoder The posterior distribution of the representation vectors at each time step. This represents the standard Gaussian prior distribution. The KL divergence represents the distance between two probability distributions. This represents the KL divergence regularization loss;

[0203] According to the The decomposition loss of the gated representation decomposition module is calculated by combining the principal component and noise component of the representation vector at each time step, as shown in the following equation:

[0204] ;

[0205] in, This represents the decomposition loss of the gated representation decomposition module;

[0206] According to the The contrast loss of the contrastive log-likelihood vector at each time step is calculated for the contrastive causal gating module, as shown in the following formula:

[0207] ;

[0208] in, This represents the contrast loss of the contrast causality gating module;

[0209] The comprehensive loss function is constructed as shown in the following equation:

[0210] ;

[0211] in, Represents the comprehensive loss function. These are the weight coefficients corresponding to the observation reconstruction loss, KL divergence regularization loss, reward prediction loss, decomposition loss of the gated representation decomposition module, and contrast loss of the contrastive causal gating module, respectively.

[0212] In the second stage, the parameters of the trained information filtering world model are fixed, and information value-constrained noise-resistant reinforcement learning is used to train the value network and policy network respectively based on the total loss function, resulting in trained value networks and trained policy networks; the construction process of the total loss function is as follows:

[0213] First, construct the objective function for the λ-Return multi-step return, as shown in the following equation:

[0214] ;

[0215] ;

[0216] in, This represents the balance coefficient of the λ-Return multi-step return. Indicates the discount factor; Indicates the first The target value of the λ-Return multistep return at the time step is determined by the λ-Return multistep return at the λ-Return time step. The reward at the first time step and the first The value is obtained by recursively calculating the value after the hidden state at each time step is input into the value network. Indicates the first The target value of the multi-step λ-Return return at each time step; The parameter set is The value network Indicates the first The value obtained by inputting the hidden state at each time step into the value network; H represents the maximum time step of the λ-Return multi-step reward. This represents the target value of the multi-step return corresponding to the maximum time step;

[0217] Then, the training loss of the value network is constructed as shown in the following formula:

[0218] ;

[0219] in, This represents the training loss of the value network. This represents the Huber loss function. For threshold hyperparameters, This represents the weight hyperparameter of the Bellman consistency regularization. This represents the squared residual of the first-step TD. Indicates the first The value obtained by inputting the hidden state of each time step into the value network;

[0220] Next, the training loss of the policy network is constructed as shown in the following formula:

[0221] ;

[0222] in, This represents the training loss of the policy network; Indicates the first The hidden state at each time step Next strategy network selection Actions at each time step The probability, Indicates the first The hidden state at each time step The probability distribution of the next policy network choosing any action; Represents the hyperbolic tangent function; Indicates the first The advantage function at each time step This indicates that the gradient operation is stopped. This indicates the scaling hyperparameters. Represents the entropy regularity coefficient; Entropy represents the policy distribution;

[0223] Subsequently, the information compression decoupling contrast loss is constructed as shown in the following equation:

[0224] ;

[0225] in, This indicates the information compression and decoupling contrast loss; Represents mutual information; The weight hyperparameters representing the information bottleneck term. This represents the balance coefficient hyperparameter. This represents the noise weighting hyperparameter. This represents the contrastive learning weight hyperparameters; Indicates the first The observation vectors at each time step are input into the parameter set as follows: Summary encoder The compressed future summary; The InfoNCE loss under the denoising invariance task is expressed as follows:

[0226] ;

[0227] in, Indicates the first Each time step This represents the total number of time steps in the training phase.

[0228] Finally, the total loss function is constructed as shown in the following equation:

[0229] ;

[0230] in, Represents the total loss function; represents the weight hyperparameters of the training loss of the value network, the training loss of the policy network, and the information compression decoupling contrast loss, respectively.

[0231] For details, please refer to Figure 5The Information Value Bounded Robust Reinforcement Learning (IVBRL) used in the embodiments of this application mainly consists of a multi-step reward estimation unit, a value consistency fitting unit, a policy advantage optimization unit, and an information noise suppression consistency unit. First, the multi-step reward estimation unit takes the latent state at the current time as input and, following a backward recursive approach, uses the value estimate of the next time step as guidance to construct the objective function of the λ-Return multi-step reward, combining the reward at the current time to achieve a smooth characterization of long-term returns. Then, the value consistency fitting unit regresses the objective value of the λ-Return multi-step reward at the current time step with the value, employing a thresholded Huber loss to suppress outlier errors, and introducing a Bellman consistency regularization step to constrain the temporal relationship between adjacent time steps, thereby stabilizing the value curve in the presence of observation noise and contact transients, and constructing the training loss formula for the value network. Next, the strategy advantage optimization unit constructs an advantage based on the difference between the target value and the value of the multi-step λ-Return at the current time step. It then performs a bounded transformation on the advantage using stopping gradients and hyperbolic compression to reduce the amplification effect of noise spikes on the policy gradient. On this basis, an entropy regularization term of the policy distribution is superimposed to maintain moderate exploration and avoid premature convergence, thus constructing the training loss of the policy network. Finally, the information noise suppression and consistency unit constructs an information compression and decoupling contrastive loss based on the useful and useless components obtained from the representation vector. For useful components, information compression is implemented by minimizing mutual information with past feature representations to reduce redundancy, retaining only task-related information, and maximizing mutual information with future summaries under future actions, strengthening the predictive ability for controllable future results. For useless components, decoupling suppression is achieved by minimizing their mutual information with future summaries, weakening the interference of noise information on prediction and control. Simultaneously, stable representation is maintained through contrastive consistency constraints, thereby enhancing the ability to discriminate the future. Through the weighted synergy of the four units mentioned above, Information Value Constrained Noise-Resistant Reinforcement Learning (IVBRL) integrates the long-term nature of multi-step rewards, the temporal consistency of value learning, the robustness of policy updates, and the discriminative nature of information-theoretic noise suppression. This ultimately achieves stable learning and reliable control of the robotic arm in bidirectional perception tasks. The constructed total loss function uses weight hyperparameters to balance the contributions of each training objective to the total loss function. This total loss function is then used to train the information filtering world model, while simultaneously updating the parameters of the policy network and value network, resulting in a trained information filtering world model, a trained policy network, and a trained value network.The training process is divided into two phases. In the first phase, the parameters of the policy network and the value network are fixed, and the information filtering world model is trained separately. In the second phase, the parameters of the trained information filtering world model are fixed, and the policy network and the value network are trained.

[0232] S2: Obtain the fuzzy command issued by the user at the current time step and input it into the BidirPR-Grasp framework to obtain the masked RGB image, masked depth map, and target item availability map for the current time step. Combined with the joint activity data of the robotic arm collected at the current time step, construct the first multimodal perception vector for the current time step. Input the first multimodal perception vector, action, and latent state of the current time step into the trained information filtering world model. The first multimodal perception vector and latent state of the current time step first pass through the encoder to obtain the representation vector of the current time step. The representation vector of the current time step is then input into the gate. The control representation decomposition module filters out the principal component and noise component of the representation vector at the current time step. The principal component of the representation vector at the current time step is input into the decoder to obtain the observation vector at the current time step. The observation vector, action, and the principal and noise components of the representation vector at the current time step are input into the contrastive causal gating module to obtain the final principal representation at the current time step. Based on the hidden state, action, and final principal representation at the current time step, a state transition function is used for recursive updating to obtain the hidden state at the next time step. The hidden state at the next time step is input into the trained policy network to obtain the action at the next time step.

[0233] Specifically, the trained information filtering world model, the trained policy network, and the trained value network are deployed, and the deployment phase follows this process:

[0234] Upper-level instruction inference: The BidirPR-Grasp framework is functioning normally, that is, it receives fuzzy instructions issued by the user, infers the target item through DeepSeek-R1 and Qwen2.5-VL, and performs localization and segmentation through GroundingDINO and SAM, and generates a masked RGB map, a masked depth map and a target item availability map.

[0235] Noise-resistant enhancement decision: Combine the generated masked RGB image, masked depth map, and target item availability map with the joint activity data of the robotic arm to form the first multimodal perception vector. The input is fed into a trained information filtering world model. This model, through its internal gated representation decomposition module and contrastive causal gating module, performs noise-resistant filtering and causal enhancement on the input representation vector. The trained information filtering world model outputs a more robust latent state for the next time step, richer in task-critical information.

[0236] Policy optimization and action generation: The policy network of IVBRL receives the latent state of the next time step generated by the trained information filtering world model, and the policy generated based on this is the action for the next time step. This strategy, having undergone noise-resistant training, is better able to resist noise interference in the input, resulting in more stable and accurate grasping motion parameters. (Action per time step) It contains parameters for the robotic arm's grasping action, such as joint angles, angular velocity, and torque, in the next time step.

[0237] S3, based on the motion control of the next time step, the robotic arm executes the grasping action of the next time step.

[0238] Finally, the action output by the policy network for the next time step is converted into the true pose in the robot arm's base coordinate system through a pre-calibrated coordinate transformation matrix. The instruction to execute this true pose is sent through the robot arm control interface, driving the robot arm to perform the grasping action. Then, it enters the next perception-decision loop, realizing continuous and robust operation in dynamic environments.

[0239] Further reference Figure 6 As an implementation of the methods shown in the above figures, this application provides an embodiment of a robotic arm reasoning and grasping device based on a noise-resistant enhanced world model. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0240] This application provides a robotic arm reasoning and grasping device based on a noise-resistant enhanced world model, including:

[0241] Model building module 1 is configured to build an information filtering world model, a value network, and a policy network. The information filtering world model includes an encoder, a gating representation decomposition module, a contrastive causal gating module, and a decoder. The information filtering world model, the value network, and the policy network are trained to obtain a trained information filtering world model, a trained value network, and a trained policy network.

[0242] Action prediction module 2 is configured to acquire the fuzzy commands issued by the user at the current time step and input them into the BidirPR-Grasp framework to obtain the masked RGB image, masked depth map, and target object availability map for the current time step. Combined with the joint activity data of the robotic arm collected at the current time step, it constructs the first multimodal perception vector for the current time step. The first multimodal perception vector, action, and latent state of the current time step are then input into a trained information filtering world model. The first multimodal perception vector and latent state of the current time step are first processed by an encoder to obtain the representation vector of the current time step. The input is fed into the gated representation decomposition module, which filters to obtain the principal component and noise component of the representation vector at the current time step. The principal component of the representation vector at the current time step is then fed into the decoder to obtain the observation vector at the current time step. The observation vector, action, and the principal and noise components of the representation vector at the current time step are then fed into the contrastive causal gating module to obtain the final principal representation at the current time step. Based on the hidden state, action, and final principal representation at the current time step, a state transition function is used for recursive updates to obtain the hidden state at the next time step. The hidden state at the next time step is then fed into the trained policy network to obtain the action at the next time step.

[0243] Execution module 3 is configured to control the robotic arm to perform the grasping action of the next time step based on the motion of the next time step.

[0244] Figure 7 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 7 As shown, the electronic device of this embodiment includes a processor 701 and a memory 702; wherein the memory 702 is used to store computer execution instructions; and the processor 701 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0245] Alternatively, the memory 702 can be either standalone or integrated with the processor 701.

[0246] When the memory 702 is set up independently, the electronic device also includes a bus 703 for connecting the memory 702 and the processor 701.

[0247] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 701, implement the above method.

[0248] This invention also provides a computer program product, including a computer program that, when executed by a processor 701, implements the above-described method.

[0249] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0250] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0251] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0252] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 701 to execute some steps of the methods of the various embodiments of this application.

[0253] It should be understood that the processor 701 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 701 can be any conventional processor 701. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 701 executing the steps, or as a combination of hardware and software modules within the processor 701 executing the steps.

[0254] The memory 702 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0255] Bus 703 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 703 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 703 in the accompanying drawings of this application is not limited to only one bus 703 or one type of bus 703.

[0256] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0257] An exemplary storage medium is coupled to a processor 701, enabling the processor 701 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 701. The processor 701 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 701 and the storage medium can exist as discrete components in an electronic device or a host device.

[0258] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0259] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reasoning and grasping of a robot arm based on an anti-noise reinforced world model, characterized in that, The method comprises the following steps: constructing an information filtering world model, a value network and a strategy network, the information filtering world model comprising an encoder, a gated representation decomposition module, a contrastive causal gating module and a decoder; training the information filtering world model, the value network and the strategy network to obtain a trained information filtering world model, a trained value network and a trained strategy network; obtaining a fuzzy instruction issued by a user at a current time step and inputting the fuzzy instruction into a BidirPR-Grasp framework to obtain a masked RGB image, a masked depth image and a target object affordance map at the current time step, and combining joint activity data of a robot arm collected at the current time step to construct a first multi-modal perception vector at the current time step, inputting the first multi-modal perception vector at the current time step, an action and a hidden state into the trained information filtering world model, the first multi-modal perception vector at the current time step and the hidden state being inputted into the encoder to obtain a representation vector at the current time step, the representation vector at the current time step being inputted into the gated representation decomposition module to screen a main effective component and a noise component of the representation vector at the current time step, the main effective component of the representation vector at the current time step being inputted into the decoder to obtain an observation vector at the current time step, the observation vector at the current time step, the action and the main effective component and the noise component of the representation vector being inputted into the contrastive causal gating module to obtain a final main effective representation at the current time step, the hidden state at the current time step, the action and the final main effective representation being used to perform recursive update by using a state transition function to obtain a hidden state at a next time step, the hidden state at the next time step being inputted into the trained strategy network to obtain an action at the next time step; controlling the robot arm to perform a grasping action at the next time step based on the action at the next time step.

2. The anti-noise reinforced world model based robotic arm reasoning and grasping method according to claim 1, wherein, The gated representation decomposition module comprises a dimension-by-dimension gating network and an element-by-element multiplication operation, and the calculation process is as follows: ； ； ； wherein, represents a gating vector at the th time step, represents a representation vector of the encoder output at the th time step, represents a real space, represents a dimension; represents a sigmoid activation function, represents an element-wise multiplication; represents a weight matrix of the dimension-wise gating network, represents a bias vector of the dimension-wise gating network; and respectively represent a principal component and a noise component of the representation vector at the th time step.

3. The anti-noise reinforced world model based robotic arm reasoning and grasping method according to claim 2, characterized in that, The contrastive causal gating module comprises an action sequence compressor, a dimension embedding module, a query-key pair construction module and a gated weighting module, and the calculation process is as follows: According to the The predicted action, predicted latent state, and predicted representation vector at the nth time step are recursively predicted using the state transition function. The predicted latent state and predicted representation vector at the nth time step, and the nth time step... The predicted latent state and predicted representation vector at the nth time step are input into the policy network to obtain the nth time step. For the predicted action at the nth time step, repeat the above steps until the nth time step is obtained. Predict the action at the nth time step, and the nth time step. The time step to the 1 The predicted action at the nth time step is constructed into the nth time step. The future at a time step Action sequence at each time step As shown in the following formula: ； ； wherein, , and when , , , , denotes the hidden state of the th time step, denotes the action of the th time step; denotes the policy network, the parameter set being ; denotes the predicted representation vector of the th time step, denotes the predicted hidden state of the th time step, denotes the predicted action of the th time step, denotes the predicted representation vector of the th time step, denotes the predicted hidden state of the th time step, denotes the predicted action of the th time step, denotes a state transition function, denotes a parameter set of the state transition function; inputting the action sequence of the first time step into the action sequence compressor to compress an action condition vector of the first time step; ;​​ The first The principal effective component of the representation vector at the nth time step is input into the dimension embedding module for dimension adjustment to obtain the nth time step. The principal components of the adjusted representation vector at each time step; The action condition vector of the first time step, the main effect component of the adjusted representation vector, the observation vector, and the noise component of the representation vector are input into the query-key pair construction module to obtain the query vector, the main effect key vector, and the noise key vector of the first time step, as shown in the following formula: ； ； ； in, Indicates the first Action condition vectors at each time step This represents the compression function corresponding to the action sequence compressor. Indicates splicing; Indicates the first The principal effective components of the adjusted representation vector at each time step This indicates the function corresponding to the dimension embedding module; Indicates the first The principal component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step Represents a multilayer perceptron; and They represent the first The principal key vector and noise key vector at each time step; The query vector of the first time step, the primary key vector and the noise key vector are input into the gating weighting module, and the contrast log likelihood vector of the first time step is calculated according to the query vector of the first time step, the primary key vector and the noise key vector, as shown in the following formula: ； wherein, represents the contrast log-likelihood vector at the represents a temperature hyperparameter, represents the noise key vector at the , represents a similarity function;​​ According to the contrast log-likelihood vector of the first time step, the controllability score vector of the first time step is calculated as follows: ； wherein, represents the expected value at the th time step, represents the controllability score vector at the th time step; According to the controllability score vector of the first time step, the dimension-level gate vector of the first time step is calculated, and the main effective component of the representation vector of the first time step is weighted to obtain the final main effective representation of the first time step, as shown in the following formula: ； ； wherein, denotes a translation reference for the score, denotes a temperature coefficient for the gating function; denotes a dimension level gating vector for the time step; denotes a final principal effect representation for the time step.

4. The anti-noise reinforced world model based robotic arm reasoning and grasping method according to claim 3, characterized in that, The information filtering world model is constructed based on a cyclic state-space model, which includes a filtering system, an observation system, a dynamic system, and a reward system. In the filtering system, the information filtering system constructs the first... The first multimodal sensing vector at the nth time step, or the vector generated during training by the nth time step. The first time step is constructed from the masked RGB image, the masked depth map, and the target item availability map. The second multimodal sensing vector at the nth time step is sequentially passed through the encoder and the gated representation decomposition module to obtain the nth time step. The principal component and noise component of the representation vector at each time step are shown in the following equation: ； wherein, represents a first multi-modal perception vector or a second multi-modal perception vector at a time step, represents an encoder with a parameter set represents a gating representation decomposition module;​ The principal component of the representation vector of the first time step is input to the decoder to obtain the observation vector of the first time step, as shown in the following formula: ； wherein, represents a decoder with parameter set ; The main effect component and the noise component of the observation vector, the action, and the representation vector of the first time step are input into the contrastive causal gating module, and the final main effect representation of the first time step is obtained, as shown in the following formula: ； wherein, represents a contrast causal gating module; According to the first The implicit state, action and final main effect representation of the t-th time step are recursively updated by a state transition function, and the implicit state of the t+1-th time step is obtained As shown in the following formula:​ ； The hidden state and the final main effect representation of the first time step are input to the reward function in the training phase to obtain the reward of the first time step, as shown in the following formula: ； wherein, represents a parameter set for a reward function, represents a reward at the th time step.

5. The anti-noise reinforced world model based robotic arm reasoning and grasping method according to claim 4, characterized in that, The training process of the information filtering world model, the value network and the strategy network comprises two stages: In the first stage, the information filtering world model is trained based on a comprehensive loss function to obtain a trained information filtering world model; and the comprehensive loss function is constructed as follows: The mean squared error is used to measure the difference between the observation vector of the decoder output at the t h time step and the principal component of the representation vector, resulting in an observation reconstruction loss, as shown in the following equation: ； wherein, represents the observation reconstruction loss; The KL divergence between the posterior distribution of the hidden state of the encoder output at the first time step and a standard Gaussian prior distribution is computed, resulting in a KL divergence regularization loss, as follows: ； wherein, represents the posterior distribution of the representation vector of the encoder output at the represents a standard Gaussian prior distribution, represents the KL divergence between two probability distributions, represents the KL divergence regularized loss;​ According to the first The main component and the noise component of the feature vector of the first time step are used to calculate the decomposition loss of the gating feature decomposition module, as shown in the following formula: ； wherein, denotes a gating representation decomposition module decomposition loss; According to the first Contrastive loss of the contrastive causal gating module is calculated according to the contrastive log-likelihood vector of the second ； wherein, represents the contrast loss of the contrast causal gating module; The comprehensive loss function is constructed as follows: ； wherein, denotes the comprehensive loss function, are weight coefficients corresponding to the observation reconstruction loss, the KL divergence regularization loss, the reward prediction loss, the decomposition loss of the gating representation decomposition module, and the contrast loss of the contrastive causal gating module, respectively. In the second stage, the parameters of the trained information filtering world model are fixed, the value network and the strategy network are trained based on a total loss function by using information value constraint noise-resistant reinforcement learning to obtain a trained value network and a trained strategy network; and the total loss function is constructed as follows: Firstly, a target function of a lambda-Return multi-step return is constructed as follows: ； ； wherein, denotes a balancing coefficient of the λ-Return multi-step return, denotes a discount factor; denotes a target value of the λ-Return multi-step return at the time step, which is calculated by a value recursion of the reward at the time step and the value obtained by inputting the hidden state at the time step into the value network, denotes a target value of the λ-Return multi-step return at the time step; denotes a value network with the parameter set , denotes a value obtained by inputting the hidden state at the time step into the value network; H denotes a maximum time step of the λ-Return multi-step return, denotes a target value of the λ-Return multi-step return corresponding to the maximum time step; Then, a training loss of the value network is constructed as follows: ； wherein, denotes the training loss of the value network, denotes the Huber loss function, is a threshold hyperparameter, denotes the weight hyperparameter for Bellman consistency regularization, denotes the one-step TD residual square, denotes the value resulting from inputting the hidden state at the time step into the value network; Next, a training loss of the strategy network is constructed as follows: ； wherein, represents the training loss of the policy network; represents the hidden state of the policy network at the time step, the probability of the policy network selecting the action at the time step, represents the probability distribution of the policy network selecting any action given the hidden state of the policy network at the time step; represents the hyperbolic tangent function; represents the advantage function at the time step, represents the stop gradient operation, represents the advantage scaling hyperparameter, represents the entropy regularization coefficient; represents the entropy of the policy distribution; Subsequently, an information compression decoupling contrastive loss is constructed as follows: ； in, This indicates the information compression and decoupling contrast loss; Represents mutual information; The weight hyperparameters representing the information bottleneck term. This represents the balance coefficient hyperparameter. This represents the noise weighting hyperparameter. This represents the contrastive learning weight hyperparameters; Indicates the first The observation vectors at each time step are input into the parameter set as follows: Summary encoder The future summary obtained by compression; The InfoNCE loss under the denoising invariance task is expressed as follows: ； wherein, denotes the time step, denotes the total number of time steps of the training phase; Finally, a total loss function is constructed as follows: ； wherein, represents the total loss function; respectively represent the weight hyperparameters of the training loss of the value network, the training loss of the policy network, and the information compression decoupling contrast loss.

6. The anti-noise reinforcement world model based robotic arm reasoning and grasping method according to claim 4, characterized in that, The construction process of the first multi-modal perception vector or the second multi-modal perception vector of the current time step includes: The current time step's masked RGB image, masked depth image and target item availability map are preprocessed and channel dimension spliced to obtain an image vector, the preprocessing manner including spatial dimension alignment, mask consistency check and repair, feature standardization and dimension and channel adaptation, and the image vector is taken as the second multi-modal perception vector; The construction process of the first multi-modal perception vector of the current time step further includes: The joint activity data of the current time step's robot arm is feature standardized, dimension and channel adapted and channel dimension spliced to obtain a joint activity feature vector, the joint activity data including joint angle, angular velocity and / or torque; The image vector and the joint activity feature vector are spliced to obtain the first multi-modal perception vector.

7. A mechanical arm reasoning grabbing device based on an anti-noise reinforced world model, characterized in that, comprises: a model construction module configured to construct an information filtering world model, a value network and a policy network, the information filtering world model comprising an encoder, a gated representation decomposition module, a contrastive causal gating module and a decoder; the information filtering world model, the value network and the policy network are trained to obtain a trained information filtering world model, a trained value network and a trained policy network; an action prediction module configured to obtain a fuzzy instruction issued by a user at a current time step and input into a BidirPR-Grasp framework to obtain a current time step's masked RGB image, masked depth image and target item availability map, and combine joint activity data of a robot arm collected at the current time step to construct a first multi-modal perception vector of the current time step, input the first multi-modal perception vector of the current time step, action and hidden state into the trained information filtering world model, the first multi-modal perception vector of the current time step and the hidden state first pass through the encoder to obtain a representation vector of the current time step, the representation vector of the current time step is input into the gated representation decomposition module to filter to obtain a main effective component and a noise component of the representation vector of the current time step; the main effective component of the representation vector of the current time step is input into the decoder to obtain an observation vector of the current time step; the observation vector of the current time step, action and the main effective component and the noise component of the representation vector are input into the contrastive causal gating module to obtain a final main effective representation of the current time step, the hidden state of the next time step is obtained by recursive update according to the hidden state of the current time step, action and final main effective representation using a state transition function, the hidden state of the next time step is input into the trained policy network to obtain an action of the next time step; an execution module configured to control the robot arm to perform a grasping action of the next time step based on the action of the next time step. 8.An electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, which when executed by the processor implements the method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program, which when executed by the processor implements the method of any one of claims 1-6.

Images