Arm reasoning and grabbing method and device based on anti-noise reinforced world model

Abstract

The application discloses a mechanical arm reasoning and grabbing method and device based on an anti-noise reinforced world model, and relates to the field of robot intelligent control. Although the existing large language model assisted grabbing framework can reason fuzzy instructions, it is easily disturbed by noise in a complex scene. Therefore, first, an information filtering world model is constructed, a gating representation decomposition module and a contrastive causal gating module are introduced, a multi-modal perception vector is decomposed into a main effective component and a noise component, key dimensions related to the future are selected under the action condition, and multi-step prediction is performed in the cycle state space. Second, an information value constraint anti-noise reinforcement learning mechanism is designed, the mutual information of the future and useful information is maximized and the mutual information of the future and useless information is minimized through value policy optimization, and the disturbance of perception noise on decision optimization is inhibited. The method combines the imagination reasoning of the world model and the decision optimization of the reinforcement learning, inhibits noise disturbance through information value constraint, and enhances the robustness of the mechanical arm reasoning and grabbing process.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of robot intelligent control, and in particular to a mechanical arm reasoning grasping method and device based on a noise-resistant reinforced world model. BACKGROUND

[0002] In recent years, with the rapid development of intelligent manufacturing, warehouse logistics and home service robots, the robustness and adaptability of mechanical arm autonomous grasping technology in complex dynamic environments have become a key challenge. Although existing grasping methods based on large language models have certain instruction reasoning and generalization capabilities, in actual scenes with sensor noise, light changes, occlusions and dynamic disturbances, the perception and decision-making process is easily affected by irrelevant information noise, resulting in a significant decrease in grasping success rate. Traditional grasping systems rely on manual features or end-to-end reinforcement learning, and lack explicit modeling of causal structures and task-critical information in the scene, making it difficult to achieve accurate state prediction and action planning in the presence of noise. SUMMARY

[0003] The present application aims to propose a mechanical arm reasoning grasping method and device based on a noise-resistant reinforced world model to solve the above technical problems.

[0004] In a first aspect, the present application provides a mechanical arm reasoning grasping method based on a noise-resistant reinforced world model, comprising the following steps:

[0005] Constructing an information filtering world model, a value network and a policy network, the information filtering world model including an encoder, a gated representation decomposition module, a contrastive causal gating module and a decoder; training the information filtering world model, the value network and the policy network to obtain a trained information filtering world model, a trained value network and a trained policy network;

[0006] The system acquires the fuzzy commands issued by the user at the current time step and inputs them into the BidirPR-Grasp framework to obtain the masked RGB image, masked depth map, and target item availability map for the current time step. Combined with the joint activity data of the robotic arm collected at the current time step, it constructs the first multimodal perception vector for the current time step. The first multimodal perception vector, action, and latent state of the current time step are then input into a trained information filtering world model. The first multimodal perception vector and latent state of the current time step are first processed by an encoder to obtain the representation vector of the current time step. The representation vector of the current time step is then input into a gating table. The feature decomposition module filters out the principal component and noise component of the representation vector at the current time step; the principal component of the representation vector at the current time step is input into the decoder to obtain the observation vector at the current time step; the observation vector, action, and the principal component and noise component of the representation vector at the current time step are input into the contrastive causal gating module to obtain the final principal representation at the current time step; the hidden state, action, and final principal representation at the current time step are recursively updated using a state transition function to obtain the hidden state at the next time step; the hidden state at the next time step is input into the trained policy network to obtain the action at the next time step.

[0007] Based on the motion control of the next time step, the robotic arm executes the grasping action of the next time step.

[0008] As a preferred embodiment, the gated representation decomposition module includes a dimension-wise gated network and an element-wise multiplication operation, and its calculation process is as follows:

[0009] ;

[0010] ;

[0011] ;

[0012] in, Indicates the first A gated vector at each time step. Indicates the first output of the encoder The representation vector at each time step, Represents the space of real numbers. Indicates dimension; This represents the Sigmoid activation function. This represents element-wise multiplication; This represents the weight matrix of a dimensionally gated network. Represents the bias vector of a dimension-wise gated network; and They represent the first The representation vector of each time step The main effect component and the noise component.

[0013] As preferred, the contrastive causal gating module comprises an action sequence compressor, a dimension embedding module, a query-key pair construction module and a gating weighting module, and the calculation process is as follows:

[0014] According to the prediction action of the th time step, the prediction implicit state and the prediction representation vector, the prediction implicit state and the prediction representation vector of the th time step are recursively predicted by using the state transition function, and the prediction implicit state and the prediction representation vector of the th time step are input into the policy network to obtain the prediction action of the th time step, and the above steps are repeated until the prediction action of the th time step is obtained, and the prediction actions from the th time step to the th time step are constructed into the future th time step th time step action sequence , as shown in the following formula:

[0015] ;

[0016] ;

[0017] wherein, , and when , , , , represents the implicit state of the th time step, represents the action of the th time step; represents the policy network, and the parameter set is ; represents the prediction representation vector of the th time step, represents the prediction implicit state of the th time step, represents the prediction action of the th time step, represents the prediction representation vector of the th time step, represents the prediction implicit state of the th time step, represents the prediction action of the th time step, represents the state transition function, represents the parameter set of the state transition function;

[0018] the action sequence input of the first time step is input into an action sequence compressor to obtain an action condition vector of the first time step through compression; the principal component of the representation vector of the first time step is input into a dimension embedding module for dimension adjustment to obtain an adjusted principal component of the representation vector of the first time step;

[0019] the action condition vector of the first time step, the adjusted principal component of the representation vector, the observation vector and the noise component of the representation vector are input into a query-key pair construction module to obtain a query vector, a principal key vector and a noise key vector of the first time step, as shown in the following formula:

[0020] the query vector, the principal key vector and the noise key vector of the first time step are input into a gated weighting module, and a contrast log-likelihood vector of the first time step is calculated according to the query vector, the principal key vector and the noise key vector of the first time step, as shown in the following formula:

[0021] ;

[0022] ;

[0023] ;

[0024] wherein, represents the action condition vector of the first time step, represents a compression function corresponding to the action sequence compressor, represents splicing; represents the adjusted principal component of the representation vector of the first time step, represents a function corresponding to the dimension embedding module; represents the principal component of the representation vector of the first time step input into the decoder to obtain the observation vector of the first time step, represents a multi-layer perception; and respectively represent the principal key vector and the noise key vector of the first time step; the query vector, the principal key vector and the noise key vector of the first time step are input into a gated weighting module, and a contrast log-likelihood vector of the first time step is calculated according to the query vector, the principal key vector and the noise key vector of the first time step, as shown in the following formula:

[0025]

[0026] ​​​​​​​​​​​​​​ ;

[0027] in, Indicates the first The contrastive log-likelihood vector at each time step This indicates temperature hyperparameters. Indicates the first The noise key vector at each time step. , Represents the similarity function;

[0028] According to the The comparison log-likelihood vector at the nth time step is calculated to obtain the th... The controllability score vector for each time step is shown in the following formula:

[0029] ;

[0030] in, Indicates the first The expectation at each time step Indicates the first Controllability score vector for each time step;

[0031] According to the The controllability score vector at the nth time step is calculated to obtain the first... The dimension-level gated vector at the nth time step, and for the nth time step We weight the principal effective components of the representation vector at the nth time step to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation:

[0032] ;

[0033] ;

[0034] in, Indicates the benchmark for the shift in scoring. This represents the temperature coefficient of the gated function; Indicates the first A dimensional gated vector at each time step; Indicates the first The final main effect characterization at each time step.

[0035] As a preferred option, the information filtering world model is constructed based on a cyclic state-space model. The cyclic state-space model includes a filtering system, an observation system, a dynamic system, and a reward system. In the filtering system, the information filtering system incorporates the information generated during the reasoning phase. The first multimodal sensing vector at the nth time step, or the vector generated during training by the nth time step. the target item affordance map of the t th time step the second multimodal perception vector of the t th time step sequentially passes through the encoder and the gated representation decomposition module to obtain the main effect component and the noise component of the representation vector of the t th time step, as shown in the following formula:

[0036] ;

[0037] wherein, the first multimodal perception vector or the second multimodal perception vector of the t th time step, the encoder with a parameter set of ; the gated representation decomposition module; the main effect component of the representation vector of the t th time step is input into the decoder to obtain the observation vector of the t th time step, as shown in the following formula:

[0038]

[0039] ;

[0040] wherein, the decoder with a parameter set of ;

[0041] the observation vector of the t th time step, the action, and the main effect component and the noise component of the representation vector are input into the contrastive causal gating module to obtain the final main effect representation of the t th time step, as shown in the following formula:

[0042] ;

[0043] wherein, the contrastive causal gating module;

[0044] the hidden state of the t th time step, the action and the final main effect representation are recursively updated by using a state transition function to obtain the hidden state of the t th time step , as shown in the following formula:

[0045] ;

[0046] the hidden state of the t th time step and the final main effect representation are input into a reward function in the training stage to obtain the reward of the t th time step, as shown in the following formula: ​​​​​​​​​​

[0047] ;

[0048] wherein, represents a reward function of a parameter set represents the reward of the th time step.

[0049] As preferred, the training process of the information filtering world model, the value network and the policy network is divided into two stages:

[0050] In the first stage, the information filtering world model is trained individually based on a comprehensive loss function to obtain a trained information filtering world model; the construction process of the comprehensive loss function is as follows:

[0051] The mean square error is used to measure the difference between the observation vector of the th time step output by the decoder and the principal component of the representation vector, to obtain an observation reconstruction loss, as shown in the following formula:

[0052] ;

[0053] wherein, represents the observation reconstruction loss;

[0054] The KL divergence between the posterior distribution of the hidden state of the th time step output by the encoder and the standard Gaussian prior distribution is calculated to obtain a KL divergence regularization loss, as shown in the following formula:

[0055] ;

[0056] wherein, represents the posterior distribution of the representation vector of the th time step output by the encoder, represents the standard Gaussian prior distribution, represents the KL divergence between the two probability distributions, represents the KL divergence regularization loss;

[0057] The decomposition loss of the gated representation decomposition module is calculated according to the principal component and the noise component of the representation vector of the th time step, as shown in the following formula:

[0058] ;

[0059] wherein, represents the decomposition loss of the gated representation decomposition module;

[0060] The decomposition loss of the gated representation decomposition module is calculated according to the principal component and the noise component of the representation vector of the ​The contrast loss of the contrastive log-likelihood vector at each time step is calculated for the contrastive causal gating module, as shown in the following formula:

[0061] ;

[0062] in, This represents the contrast loss of the contrast-cause-gated module;

[0063] The comprehensive loss function is constructed as shown in the following equation:

[0064] ;

[0065] in, Represents the comprehensive loss function. These are the weight coefficients corresponding to the observation reconstruction loss, KL divergence regularization loss, reward prediction loss, decomposition loss of the gated representation decomposition module, and contrast loss of the contrastive causal gating module, respectively.

[0066] In the second stage, the parameters of the trained information filtering world model are fixed, and information value-constrained noise-resistant reinforcement learning is used to train the value network and policy network respectively based on the total loss function, resulting in trained value networks and trained policy networks; the construction process of the total loss function is as follows:

[0067] First, construct the objective function for the λ-Return multi-step return, as shown in the following equation:

[0068] ;

[0069] ;

[0070] in, This represents the balance coefficient of the λ-Return multi-step return. Indicates the discount factor; Indicates the first The target value of the λ-Return multistep return at the time step is determined by the λ-Return multistep return at the λ-Return time step. The reward at the first time step and the first The value is obtained by recursively calculating the value after the hidden state at each time step is input into the value network. Indicates the first The target value of the multi-step λ-Return return at each time step; The parameter set is The value network Indicates the first The value obtained by inputting the hidden state at each time step into the value network; H represents the maximum time step of the λ-Return multi-step reward. This represents the target value of the multi-step return corresponding to the maximum time step;

[0071] Then, the training loss of the value network is constructed as follows:

[0072] ;

[0073] wherein, denotes the training loss of the value network, denotes the Huber loss function, is a threshold hyperparameter, denotes a weight hyperparameter of Bellman consistency regularization, denotes one-step TD residual square, denotes the value obtained by inputting the hidden state at the time step into the value network;

[0074] Next, the training loss of the policy network is constructed as follows:

[0075] ;

[0076] wherein, denotes the training loss of the policy network; denotes the probability of the action selected by the policy network at the time step under the hidden state at the time step, denotes the probability distribution of selecting any action by the policy network at the time step under the hidden state at the time step; denotes the hyperbolic tangent function; denotes the advantage function at the time step, denotes the stop gradient operation, denotes an advantage scaling hyperparameter, denotes an entropy regularization coefficient; denotes the entropy of the policy distribution;

[0077]

[0078] ;

[0079] wherein, denotes the information compression decoupling contrast loss; denotes mutual information; denotes a weight hyperparameter of the information bottleneck term, denotes a balance coefficient hyperparameter, denotes a noise weight hyperparameter, denotes a contrast learning weight hyperparameter; denotes inputting the observation vector of the time step into the abstract encoder with the parameter set ; denotes the future abstract obtained by compressing the future abstract; denotes the InfoNCE loss under the denoising invariance task, and its expression is as follows:

[0080] ;

[0081] wherein, denotes the time step, denotes the total number of time steps in the training phase;

[0082] Finally, the total loss function is constructed, as shown in the following formula:

[0083] ;

[0084] wherein, denotes the total loss function; denote the weight hyperparameters of the training loss of the value network, the training loss of the policy network and the information compression decoupling contrast loss, respectively.

[0085] Preferably, the construction process of the first multi-modal perception vector or the second multi-modal perception vector of the current time step comprises:

[0086] The masked RGB image, the masked depth image and the target object affordance map of the current time step are preprocessed and channel-dimensionally spliced to obtain an image vector, the preprocessing manner comprises spatial dimension alignment, mask consistency check and repair, feature standardization and dimension and channel adaptation, and the image vector is taken as the second multi-modal perception vector;

[0087] The construction process of the first multi-modal perception vector of the current time step further comprises:

[0088] The joint activity data of the robot arm at the current time step are subjected to feature standardization, dimension and channel adaptation and channel-dimension splicing to obtain a joint activity feature vector, and the joint activity data comprises joint angle, angular velocity and / or torque;

[0089] The image vector and the joint activity feature vector are spliced to obtain the first multi-modal perception vector.

[0090] In a second aspect, the present application provides a robot arm reasoning grasping device based on an anti-noise reinforcement world model, comprising:

[0091] The model construction module is configured to construct an information filtering world model, a value network and a policy network, the information filtering world model comprising an encoder, a gated representation decomposition module, a contrastive causal gating module and a decoder; and the information filtering world model, the value network and the policy network are trained to obtain a trained information filtering world model, a trained value network and a trained policy network.

[0092] The action prediction module is configured to obtain a fuzzy instruction issued by the user at a current time step and input the fuzzy instruction into the BidirPR-Grasp framework to obtain a masked RGB image, a masked depth image and an object affordance map at the current time step, and construct a first multi-modal perception vector at the current time step in combination with joint activity data of the robot arm collected at the current time step, input the first multi-modal perception vector at the current time step, an action and a hidden state into the trained information filtering world model, and first pass the first multi-modal perception vector at the current time step and the hidden state through the encoder to obtain a representation vector at the current time step, input the representation vector at the current time step into the gated representation decomposition module to filter a main effective component and a noise component of the representation vector at the current time step, input the main effective component of the representation vector at the current time step into the decoder to obtain an observation vector at the current time step, and input the observation vector at the current time step, the action and the main effective component and the noise component of the representation vector at the current time step into the contrastive causal gating module to obtain a final main effective representation at the current time step, and recursively update the hidden state at the current time step, the action and the final main effective representation by using a state transition function to obtain a hidden state at a next time step, input the hidden state at the next time step into the trained policy network to obtain an action at the next time step.

[0093] The execution module is configured to control the robot arm to perform a grasping action at the next time step based on the action at the next time step.

[0094] In a third aspect, the present application provides an electronic device, comprising one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the implementation manners of the first aspect.

[0095] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, when the computer program is executed by a processor, the method described in any of the implementation manners of the first aspect is implemented.

[0096] In a fifth aspect, the present application provides a computer program product, comprising a computer program, when the computer program is executed by a processor, the method described in any of the implementation manners of the first aspect is implemented.

[0097] Compared with the prior art, the present application has the following beneficial effects:

[0098] (1) The mechanical arm reasoning grasping method based on the anti-noise reinforced world model constructs an information filtering world model, and realizes noise suppression and causal enhancement through the cooperation of a gating representation decomposition module and a comparative causal gating module. The gating representation decomposition module explicitly separates the main components related to the task and the noise components irrelevant to the task from the representation vector through the sparse gating mechanism and the dimension-by-dimension screening; the comparative causal gating module strengthens the key dimensions that have causal contributions to future prediction in the implicit state through cross-time domain comparative learning under the action condition, significantly improves the robust perception ability of the model under the conditions of occlusion, illumination change and sensor noise, and solves the problems of low grasping success rate of traditional grasping methods caused by insufficient perception redundancy and strategy coupling in a dynamic environment.

[0099] (2) The mechanical arm reasoning grasping method based on the anti-noise reinforced world model of the embodiment of the present application proposes an information value constraint anti-noise reinforcement learning mechanism, which breaks through the limitations of traditional reinforcement learning in a noisy environment, such as large value estimation fluctuations and unstable strategy convergence. The mechanism combines lambda-Return multi-step return estimation, Huber robust regression, Bellman consistency constraint and information theory noise suppression to realize smooth modeling of long-term return, time consistency of value function and bounded optimization of strategy gradient, and through mutual information minimization and maximization operations, the sensitivity to past redundant information and future noise in the strategy learning process is explicitly constrained, and the discrimination and utilization ability of the model to key task features is enhanced.

[0100] (3) The mechanical arm reasoning grasping method based on the anti-noise reinforced world model of the embodiment of the present application realizes the significant improvement of perception robustness in the mechanical arm reasoning grasping task through the collaborative design and deep integration of the information filtering world model and the information value constraint anti-noise reinforcement learning mechanism, and provides a novel and effective solution for the reliable operation of embodied intelligent systems in complex real-world scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0101] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0102] Figure 1 The flowchart of the mechanical arm reasoning grasping method based on the anti-noise reinforced world model of the embodiment of the present application;

[0103] Figure 2A schematic diagram of a calculation process of an information filtering world model of an anti-noise reinforcement world model-based mechanical arm reasoning grasping method of an embodiment of the present application;

[0104] Figure 3 A schematic diagram of a calculation process of a gating representation decomposition module of an anti-noise reinforcement world model-based mechanical arm reasoning grasping method of an embodiment of the present application;

[0105] Figure 4 A schematic diagram of a calculation process of a contrastive causal gating module of an anti-noise reinforcement world model-based mechanical arm reasoning grasping method of an embodiment of the present application;

[0106] Figure 5 A schematic diagram of a construction process of a total loss function of information value constraint anti-noise reinforcement learning of an anti-noise reinforcement world model-based mechanical arm reasoning grasping method of an embodiment of the present application;

[0107] Figure 6 A schematic diagram of an anti-noise reinforcement world model-based mechanical arm reasoning grasping device of an embodiment of the present application;

[0108] Figure 7 A schematic diagram of a hardware structure of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0109] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0110] Figure 1 An anti-noise reinforcement world model-based mechanical arm reasoning grasping method provided by an embodiment of the present application is shown, which includes the following steps:

[0111] S1, an information filtering world model, a value network and a policy network are constructed, the information filtering world model includes an encoder, a gating representation decomposition module, a contrastive causal gating module and a decoder; the information filtering world model, the value network and the policy network are trained to obtain a trained information filtering world model, a trained value network and a trained policy network.

[0112] Specifically, the hardware system corresponding to the anti-noise reinforcement world model-based mechanical arm reasoning grasping method proposed in the embodiments of the present application is constructed based on a mature mechanical arm grasping platform, and the mechanical arm grasping platform includes:

[0113] The execution unit includes a JAKA C7 six-axis collaborative robot arm and its matching two-finger planar gripper;

[0114] The perception unit includes an Orbbec Gemini 2 depth camera fixedly installed on a flange at the end of the robot arm, for real-time acquisition of 480x640 resolution RGB images and depth maps of the working area.

[0115] The interaction unit includes a programmable microphone array for receiving natural language instructions from the user.

[0116] The computing unit includes a local workstation configured as CPU, GPU and 32GB RAM, running a Windows operating system. All model inferences, world model predictions for information filtering, and decision processes for reinforcement learning are completed on this local workstation.

[0117] The embodiments of the present application can achieve seamless integration with the upstream BidirPR-Grasp framework. The output of the BidirPR-Grasp framework will serve as the input of the information filtering world model of the embodiments of the present application, and the specific interface is as follows:

[0118] The masked RGB image and the masked depth image are obtained from the visual positioning stage of the BidirPR-Grasp framework. The target object affordance map is generated with the optimal component key point as the center from the affordability generation stage; the BidirPR-Grasp framework proposed in the embodiments of the present application is prior art, based on the content mentioned in the paper entitled "Research on Robot Grasping Method Based on Large Language Model", which will not be repeated here.

[0119] The masked RGB image, the masked depth image and the target object affordance map are preprocessed and channel spliced to form a second multi-modal perception vector, or further combined with the joint activity data of the robot arm to form a first multi-modal perception vector, which is used as one of the inputs of the information filtering world model (IFWM).

[0120] In specific embodiments, the construction process of the first multi-modal perception vector or the second multi-modal perception vector at the current time step includes:

[0121] The masked RGB image, the masked depth image and the target object affordance map at the current time step are preprocessed and channel dimension spliced to obtain an image vector, the preprocessing methods include spatial dimension alignment, mask consistency verification and repair, feature standardization, and dimension and channel adaptation, and the image vector is used as the second multi-modal perception vector.

[0122] The construction process of the first multi-modal perception vector of the current time step further includes:

[0123] The joint activity data of the robot arm at the current time step is subjected to feature standardization, dimension and channel adaptation, and channel dimension splicing to obtain a joint activity feature vector, the joint activity data including joint angle, angular velocity and / or torque;

[0124] The image vector and the joint activity feature vector are spliced to obtain the first multi-modal perception vector.

[0125] Specifically, the specific steps of the pre-processing and channel splicing of the current time step of the masked RGB image, the masked depth image and the target object availability map in the embodiments of the present application are as follows:

[0126] The first step is spatial dimension alignment, and the specific process is as follows:

[0127] Resolution unification: taking the resolution (HxW) of the masked RGB image output by the visual positioning stage of the BidirPR-Grasp framework as the benchmark, the masked depth image and the target object availability map are resized and interpolated to ensure that the spatial dimensions of the three are completely consistent.

[0128] Coordinate benchmark alignment: based on the coordinates of the optimal component key points output by the BidirPR-Grasp, the target object availability map is subjected to fine-tuning operations such as translation and cropping to ensure that its key regions completely overlap with the mask regions in the masked RGB image and the masked depth image, avoiding spatial misalignment leading to feature failure.

[0129] The second step is mask consistency verification and repair, and the specific process is as follows:

[0130] Mask homology verification: the mask channels of the masked RGB image and the masked depth image are extracted, and the intersection rate of the two masks is verified, the intersection rate being used to quantify the degree of mask deviation, the lower the intersection rate, the greater the deviation; when the deviation is small, local inconsistency can be eliminated through morphological repair, and when the deviation is large, bottom-up processing is needed through mask rejection or re-acquisition mechanism; if there is a small deviation, the mask in the masked depth image is subjected to morphological repair based on the mask in the masked RGB image to ensure that the target regions of the two are completely consistent.

[0131] Mask application reinforcement: the unified mask channel is applied to the target object availability map to ensure that only the effective features within the target object mask are retained and background interference is eliminated.

[0132] The third step is feature standardization, and the specific process is as follows:

[0133] Pixel value normalization: Firstly, for the RGB image with mask: normalize the pixel values of the 3 channels to the interval [0, 1] to eliminate the influence of pixel magnitude differences on subsequent feature fusion; secondly, for the depth image with mask: the depth value is the actual physical distance, the range may fluctuate with the scene, use the effective range of the depth image based on the BidirPR-Grasp output, that is, the minimum value min and the maximum value max of the depth value in the mask area, map the depth value to the interval [0, 1] to preserve the relative difference in depth; finally, for the target item availability map: its pixel value is the availability score, which is usually normalized to [0, 1] by the BidirPR-Grasp framework, representing the probability of adapting the robot's action at this location, if not normalized, use the same normalization logic as pixel value normalization to ensure consistency with the magnitude of the previous two types of data.

[0134] Fourth step, dimension and channel adaptation, the specific process is as follows:

[0135] Tensor format conversion: convert the three types of image data from 2D space matrix (HxW) to 4D tensor format (BxCxHxW) compatible with deep learning models, where B represents batch size (to meet the input requirements of IFWM, usually set to 1, i.e. single frame input); C represents the number of channels (3 channels for the RGB image with mask, 1 channel for the depth image with mask, and 1 channel for the target item availability map);

[0136] Invalid channel removal: check the channel dimension of the three types of image data, remove the redundant mask channel in the RGB image with mask, and only keep the R / G / B three channels and the redundant information of the depth image with mask to ensure that each type of data channel is pure and avoid dimension redundancy after splicing.

[0137] After the above preprocessing, the format of the three types of image data is unified to a tensor with dimensions BxCxHxW:

[0138] The dimension of the RGB image with mask is Bx3xHxW;

[0139] The dimension of the depth image with mask is Bx1xHxW;

[0140] The dimension of the target item availability map is Bx1xHxW.

[0141] Through channel dimension splicing, a second multi-modal perception vector is finally generated, with a dimension of Bx(3+1+1)xHxW=Bx5xHxW. This second multi-modal perception vector can be directly used as the input of the IFWM in the training phase, realizing the integrated input of multi-modal features, and then the action and hidden state are input together into the information filtering world model to accurately predict the action at the next time step.

[0142] In specific embodiments, the gating representation decomposition module comprises a dimension-wise gating network and an element-wise multiplication operation, which is calculated as follows:

[0143] ;

[0144] ;

[0145] ;

[0146] wherein, denotes the gating vector at the t-th time step, denotes the representation vector of the t-th time step of the encoder output, denotes the real number space, denotes the dimension; denotes the Sigmoid activation function, denotes the element-wise multiplication; denotes the weight matrix of the dimension-wise gating network, denotes the bias vector of the dimension-wise gating network; and denote the principal component and the noise component of the representation vector at the t-th time step, respectively. Specifically, based on the above input, the parameters of each module in the information filtering world model proposed in the embodiments of the present application are built and initialized on a local workstation equipped with a GPU based on the PyTorch framework. Due to the limitation of 24GB of GPU memory in the local workstation, all large models need to be designed as lightweight versions, and the total parameter quantity needs to be optimized to meet the real-time inference requirements. The value network and the policy network mentioned in the embodiments of the present application adopt a medium-sized MLP structure.

[0147] The information filtering world model proposed in the embodiments of the present application is used to perform multi-step simulation prediction on the grasping action of the target object, and reference is made to

[0148] ,

[0149] The information filtering world model proposed in the embodiments of the present application is used to perform multi-step simulation prediction on the grasping action of the target object, and reference is made to Figure 2 , Figure 2 ​Taking t=1, 2, 3 as an example, the same logic applies in subsequent time steps. The embodiments of this application propose an information filtering world model that introduces a gated representation decomposition module and a contrastive causal gated module. Using the latent states extracted from the information filtering world model as the environment, it obtains information favorable for grasping the target object from three types of image data: masked RGB images, masked depth maps, and target object availability maps. It evaluates the information gain of candidate grasping actions and selects the optimal action. Furthermore, it designs an information value-constrained noise-resistant reinforcement learning mechanism, improving perceptual robustness by designing future and past policy optimizations. The embodiments of this application combine the perceptual reasoning capabilities of the world model with the environmental modeling advantages of reinforcement learning. Through innovative noise-resistant theory of information bottlenecks, it enhances the robustness of the robotic arm in grasping target parts, making it suitable for various grasping task scenarios such as home robots.

[0150] refer to Figure 3 The gated representation disentanglement module (GRD) mentioned in the embodiments of this application mainly consists of a dimension-wise gated network and element-wise multiplication operations. First, the representation disentanglement module (GRD) obtained by the encoder... The representation vector of each time step Inputting a dimension-wise gated network, the 1st generation is obtained through linear transformation and Sigmoid activation. The gating vector at the nth time step, and then the gating vector at the th time step. The representation vector of each time step Weighting is applied to separate useful and useless information. In one embodiment, the encoder in this application uses a Transformer-type encoder; in other embodiments, the encoder in this application may also use a convolutional encoder.

[0151] In a specific embodiment, the comparative causal gating module includes an action sequence compressor, a dimension embedding module, a query-key pair construction module, and a gating weighting module, and its calculation process is as follows:

[0152] According to the The predicted action, predicted latent state, and predicted representation vector at the nth time step are recursively predicted using the state transition function. The predicted latent state and predicted representation vector at the nth time step, and the nth time step... The predicted latent state and predicted representation vector at the i-th time step are input into the policy network to obtain the i-th time step. For the predicted action at the nth time step, repeat the above steps until the nth time step is obtained. Predict the action at the nth time step, and the nth time step. The time step to the 1 The predicted action at the nth time step is constructed into the nth time step. The future at a time step Action sequence at each time step As shown in the following formula:

[0153] ;

[0154] ;

[0155] in, , and when hour, , , , Indicates the first The implicit state at each time step Indicates the first The action at each time step; Represents the policy network, with the parameter set as follows: ; Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Represents the state transition function. The set of parameters representing the state transition function;

[0156] The first The future at a time step The action sequence at time step n is input into the action sequence compressor and compressed to obtain the action sequence at time step n. Action condition vectors for each time step;

[0157] The first The principal effective component of the representation vector at the nth time step is input into the dimension embedding module for dimension adjustment, resulting in the nth time step. The principal components of the adjusted representation vector at each time step;

[0158] The first The action condition vector at each time step, the principal component of the adjusted representation vector, the observation vector, and the noise component of the representation vector are input into the query-key pair construction module to obtain the first time step. The query vector, principal key vector, and noise key vector for each time step are shown in the following formula:

[0159] ;

[0160] ;

[0161] ;

[0162] in, Indicates the first Action condition vectors at each time step This represents the compression function corresponding to the action sequence compressor. Indicates splicing; Indicates the first The principal effective components of the adjusted representation vector at each time step This indicates the function corresponding to the dimension embedding module; Indicates the first The principal effective component of the representation vector at time step n is input into the decoder to obtain the nth time step. The observation vector at each time step This represents a multilayer perceptron; and They represent the first The principal key vector and noise key vector at each time step;

[0163] The first The query vector, principal key vector, and noise key vector at each time step are input into the gated weighting module. First, based on the query vector, principal key vector, and noise key vector at each time step... The query vector, principal key vector, and noise key vector at the nth time step are used to calculate the nth time step. The contrastive log-likelihood vector for each time step is shown in the following equation:

[0164] ;

[0165] in, Indicates the first The contrastive log-likelihood vector at each time step This indicates temperature hyperparameters. Indicates the first The noise key vector at each time step. , Represents the similarity function;

[0166] According to the The comparison log-likelihood vector at the nth time step is calculated to obtain the th... The controllability score vector for each time step is shown in the following formula:

[0167] ;

[0168] wherein, represents the expectation of the th time step, represents the controllability score vector of the th time step;

[0169] The dimension-level gating vector of the th time step is calculated according to the controllability score vector of the th time step, and the main effective component of the representation vector of the th time step is weighted to obtain the final main effective representation of the th time step, as shown in the following formula:

[0170] ;

[0171] ;

[0172] wherein, represents the translation reference of the score, represents the temperature coefficient of the gating function; represents the dimension-level gating vector of the th time step; represents the final main effective representation of the th time step.

[0173] Specifically, with reference to Figure 4 , the contrastive causal gate module (CCG) mentioned in the embodiments of the present application mainly consists of an action sequence compressor, a dimension embedding module, a query-key pair construction module and a gating weighting module. First, the main effective component of the representation vector of the current time step is input into the dimension embedding module, and the action sequence from the current time step to the future several steps is compressed into an action condition vector to capture the regulation effect of the action on the future state. Subsequently, the main effective component of the adjusted representation vector and the action condition vector are input into a multilayer perceptron to generate a query vector, which represents the predicted potential of the main effective component of the adjusted representation vector of each dimension under the given action condition. Then, the observation vector and the noise component of the representation vector are respectively mapped into a main effective key vector and an interference key vector, and the controllability score vector of each dimension is calculated by a contrastive loss, which reflects the contribution degree of the feature of the dimension to the future state. Subsequently, the controllability score vector is converted into a dimension-level gating vector by using a temperature coefficient to realize the weighted selection of each dimension. Then, the dimension-level gating vector and the main effective component are multiplied element by element to obtain the final main effective representation, which ensures that the information of the key dimension is retained and the noise dimension is suppressed.

[0174] The The implicit state and the representation vector of the time step are input into the policy network to generate an action and perform multi-step forward expansion in the information filtering world model, so as to construct a future action sequence. Specifically, the policy network generates an action of the first time step with the implicit state and the representation vector of the first time step as input , and inputs the action of the first time step into a state transition function to recursively predict the implicit state of the subsequent time step; on the basis of each predicted implicit state, the policy network further generates an action of the next time step, until an action sequence with a length of is formed. The action sequence obtained represents a future action trajectory predicted by the policy network in combination with the information filtering world model under the current state condition. In order to serve as the action condition input of the causal gating module, the action sequence needs to be compressed through splicing, weighted summation and a mapping network to form an action condition vector , which is used to depict the regulation information of the action on the state evolution in a future period of time under a given policy hypothesis, and participate in subsequent controllability evaluation and gating weight calculation. In specific embodiments, the information filtering world model is constructed based on a recurrent state space model, which includes a filtering system, an observation system, a dynamic system and a reward system. In the filtering system, the first multi-modal perception vector of the first time step constructed in the reasoning phase or the second multi-modal perception vector of the first time step constructed by the masked RGB image, the masked depth image and the target object affordance map of the first time step in the training process are sequentially input into an encoder and a gated representation decomposition module to obtain the main component and the noise component of the representation vector of the first time step, as shown in the following formula:

[0175]

[0176]

[0177]

[0178] ​​​​​​​​​​​​​​The principal effective component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step is shown in the following equation:

[0179] ;

[0180] in, The parameter set is The decoder;

[0181] The first The observation vector, action, and principal component and noise component of the representation vector at the nth time step are input into the contrastive causal gating module to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation:

[0182] ;

[0183] in, This indicates a comparison of causal gating modules;

[0184] According to the The implicit state, action, and final principal effect representation at the nth time step are recursively updated using a state transition function to obtain the nth time step. The hidden state at each time step As shown in the following formula:

[0185] ;

[0186] During the training phase, the first The latent state and final main effect representation at the nth time step are input into the reward function to obtain the nth time step. The reward for each time step is shown in the following formula:

[0187] ;

[0188] in, The parameter set is The reward function, Indicates the first Rewards for each time step.

[0189] Specifically, the information filtering world model mentioned in the embodiments of this application is based on the cyclic state-space model (RSSM), which consists of four parts, including:

[0190] Filtering system: on one hand, the first or second multi-modal perception vector and the hidden state of the current time step are input into the encoder to obtain the representation vector, which is sequentially input into the gated representation decomposition module and the contrastive causal gating module to further suppress irrelevant noise and enhance the causality of the state representation; on the other hand, the representation vector is adaptively dimensionally filtered by the gated representation decomposition module, and the noise component is decomposed for subsequent constraint training for noise suppression.

[0191] Observation system: the observation vector is constructed according to the principal component of the representation vector of the current time step.

[0192] Dynamic system: the hidden state of the current time step, the action and the final principal representation filtered by the CCG are input into the dynamic system, and the hidden state of the next time step is obtained by recursively updating the hidden state through the state transition function.

[0193] Reward system: the reward of the current time step is estimated according to the principal component and the hidden state of the current time step by the reward function.

[0194] It should be noted that the information filtering world model of the embodiments of the present application needs to be cyclically used in the filtering system, the observation system, the dynamic system and the reward system in the training stage, and only needs to be used in the filtering system, the observation system and the dynamic system in the inference stage.

[0195] In specific embodiments, the training process of the information filtering world model, the value network and the policy network is divided into two stages:

[0196] In the first stage, the information filtering world model is trained based on the comprehensive loss function to obtain the trained information filtering world model; the construction process of the comprehensive loss function is as follows:

[0197] The mean square error is used to measure the difference between the observation vector of the first time step output by the decoder and the principal component of the representation vector, to obtain the observation reconstruction loss, as shown in the following formula:

[0198] ;

[0199] Wherein, represents the observation reconstruction loss;

[0200] The KL divergence between the posterior distribution of the hidden state of the first time step output by the encoder and the standard Gaussian prior distribution is calculated to obtain the KL divergence regularization loss, as shown in the following formula:

[0201] ;

[0202] Wherein, the posterior distribution of the representation vector of the encoder output at the

[0203] the balance coefficient of the λ-Return multi-step return, the decomposition loss of the gated representation decomposition module is calculated according to the principal component and the noise component of the representation vector at the

[0204]

[0205]

[0206] the contrast loss of the contrast-causal gated module is calculated according to the contrast log-likelihood vector at the

[0207]

[0208] the comprehensive loss function is constructed as follows:

[0209]

[0210]

[0211]

[0212] In the second stage, the parameters of the trained information filtering world model are fixed, and the value network and the policy network are trained based on the total loss function with information value constraint anti-noise reinforcement learning to obtain the trained value network and the trained policy network; the construction process of the total loss function is as follows:

[0213] First, the target function of λ-Return multi-step return is constructed as follows:

[0214]

[0215]

[0216] the balance coefficient of the λ-Return multi-step return,​​​​​​​​​​​​​​​​​​ Indicates the discount factor; Indicates the first The target value of the λ-Return multistep return at the time step is determined by the λ-Return multistep return at the λ-Return time step. The reward at the first time step and the first The value is obtained by recursively calculating the value after the hidden state at each time step is input into the value network. Indicates the first The target value of the multi-step λ-Return return at each time step; The parameter set is The value network Indicates the first The value obtained by inputting the hidden state at each time step into the value network; H represents the maximum time step of the λ-Return multi-step reward. This represents the target value of the multi-step return corresponding to the maximum time step;

[0217] Then, the training loss of the value network is constructed as shown in the following formula:

[0218] ;

[0219] in, This represents the training loss of the value network. This represents the Huber loss function. For threshold hyperparameters, This represents the weight hyperparameter of the Bellman consistency regularization. This represents the squared residual of the first-step TD. Indicates the first The value obtained by inputting the hidden state of each time step into the value network;

[0220] Next, the training loss of the policy network is constructed as shown in the following formula:

[0221] ;

[0222] in, This represents the training loss of the policy network; Indicates the first The hidden state at each time step Next strategy network selection Actions at each time step The probability, Indicates the first The hidden state at each time step The probability distribution of the next policy network choosing any action; Represents the hyperbolic tangent function; Indicates the first the advantage function of the time step, denotes the stop-gradient operation, denotes the advantage scaling hyperparameter, denotes the entropy regularization coefficient; denotes the entropy of the policy distribution;

[0223] Subsequently, the information compression decoupled contrastive loss is constructed as follows:

[0224] ;

[0225] wherein, denotes the information compression decoupled contrastive loss; denotes the mutual information; denotes the weight hyperparameter of the information bottleneck term, denotes the balance coefficient hyperparameter, denotes the noise weight hyperparameter, denotes the contrastive learning weight hyperparameter; denotes inputting the observation vector of the time step to the summary encoder with the parameter set to compress the future summary; denotes the InfoNCE loss under the denoising invariance task, and the expression is as follows:

[0226] ;

[0227] wherein, denotes the time step , and denotes the total number of time steps in the training stage;

[0228] Finally, the total loss function is constructed as follows:

[0229] ;

[0230] wherein, denotes the total loss function; respectively denote the weight hyperparameters of the training loss of the value network, the training loss of the policy network and the information compression decoupled contrastive loss.

[0231] Specifically, refer to Figure 5The information value bounded robust reinforcement learning (IVBRL) used by the embodiments of the present application mainly consists of a multi-step return estimation unit, a value consistency fitting unit, a policy advantage optimization unit, and an information noise suppression consistency unit. First, the multi-step return estimation unit takes the hidden state of the current time as input, uses the value estimation of the next time step to guide the construction of the target function of the lambda-Return multi-step return in a backward recursive manner, combines the reward of the current time, and realizes the smooth depiction of long-term returns. Then, the value consistency fitting unit regresses the target value of the lambda-Return multi-step return of the current time step with the value, uses a threshold Huber loss to suppress abnormal errors, and introduces a one-step Bellman consistency regularization to constrain the temporal relationship between adjacent time points, thereby stabilizing the value curve in the presence of observation noise and contact transients, and constructing the training loss formula of the value network. Next, the policy advantage optimization unit constructs the advantage according to the difference between the target value of the lambda-Return multi-step return of the current time step and the value, and performs bounded transformation on the advantage through the stop gradient and hyperbolic compression to reduce the amplification effect of noise spikes on policy gradients; On this basis, the entropy regularization term of the policy distribution is superimposed to maintain moderate exploration and avoid premature convergence, and the training loss of the policy network is constructed. Finally, the information noise suppression consistency unit constructs an information compression decoupling contrast loss based on the useful and useless components obtained in the representation vector. The useful component implements information compression to reduce redundancy by minimizing the mutual information with the past feature representation, and only retains task-related information, and maximizes the mutual information between the future summary under the future action to strengthen the prediction ability of the controllable future result. For the useless component, decoupling suppression is achieved by minimizing the mutual information with the future summary to weaken the interference of noise information on prediction and control. At the same time, the stable expression is maintained through the contrast consistency constraint, thereby enhancing the discriminability of the future. Through the weighted synergistic effect of the above four units, the information value bounded robust reinforcement learning (IVBRL) integrates the long-term nature of multi-step return, the temporal consistency of value learning, the robustness of policy update, and the discriminability of information theory noise suppression into one, and finally realizes the stable learning and reliable control of the robot arm in the bidirectional perception task, and constructs a total loss function that balances the contributions of each training target in the total loss function using a weight hyperparameter. The total loss function is used to train the information filtering world model through reinforcement learning, while updating the parameters of the policy network and the value network, etc., to obtain a trained information filtering world model, a trained policy network, and a trained value network.In the training process, training is carried out in two stages, the first stage fixes the parameters of the policy network and the value network, and trains the information filtering world model alone; the second stage fixes the parameters of the trained information filtering world model, and trains the policy network and the value network.

[0232] S2, the fuzzy instruction issued by the user at the current time step is obtained and input into the BidirPR-Grasp framework to obtain a masked RGB image, a masked depth image and a target object affordance map at the current time step, and a first multi-modal perception vector at the current time step is constructed by combining the joint activity data of the robot arm collected at the current time step, the first multi-modal perception vector at the current time step, the action and the hidden state are input into the trained information filtering world model, the first multi-modal perception vector at the current time step and the hidden state are first input into the encoder to obtain a representation vector at the current time step, the representation vector at the current time step is input into the gated representation decomposition module to screen the main effective component and the noise component of the representation vector at the current time step; the main effective component of the representation vector at the current time step is input into the decoder to obtain an observation vector at the current time step; the observation vector at the current time step, the action and the main effective component and the noise component of the representation vector are input into the contrastive causal gating module to obtain a final main effective representation at the current time step, the hidden state at the next time step is obtained by recursively updating the hidden state at the current time step, the action and the final main effective representation according to the state transition function, and the hidden state at the next time step is input into the trained policy network to obtain the action at the next time step.

[0233] Specifically, the trained information filtering world model, the trained policy network and the trained value network are deployed, and the following process is run in the deployment stage:

[0234] Upper instruction reasoning: the BidirPR-Grasp framework normally operates, that is, receives the fuzzy instruction issued by the user, infers the target object through DeepSeek-R1 and Qwen2.5-VL, and performs positioning and segmentation through GroundingDINO and SAM, and generates a masked RGB image, a masked depth image and a target object affordance map.

[0235] Anti-noise reinforcement decision: the above generated masked RGB image, masked depth image and target object affordance map are combined with the joint activity data of the robot arm to form a first multi-modal perception vector , which is input into the trained information filtering world model. The trained information filtering world model performs anti-noise filtering and causal enhancement on the input representation vector through the gated representation decomposition module and the contrastive causal gating module in it, and the trained information filtering world model outputs the hidden state at the next time step which is more robust and rich in task key information.

[0236] Policy optimization and action generation: the policy network of the IVBRL receives the hidden state of the next time step generated by the trained information filtering world model, and generates a policy based on it, which is the action of the next time step . The policy can better resist noise interference in the input due to anti-noise reinforcement training, and output more stable and accurate grasping action parameters. An action of a time step contains the joint angle, angular velocity, torque and other grasping action parameters of the robot arm in the next time step.

[0237] S3, based on the action of the next time step, control the robot arm to execute the grasping action of the next time step.

[0238] Finally, the action of the next time step output by the policy network is converted into the real pose in the robot arm coordinate system through the pre-calibrated coordinate transformation matrix. The instruction to execute the real pose is issued through the robot arm control interface to drive the robot arm to execute the grasping action. Subsequently, enter the next perception-decision cycle to realize continuous and robust operation in a dynamic environment.

[0239] Further referring to Figure 6 , as an implementation of the method shown in the above figures, the present application provides an embodiment of a robot arm reasoning grasping device based on an anti-noise reinforcement world model. The device embodiment corresponds to the method embodiment shown in Figure 1 , and the device can be applied to various electronic devices.

[0240] The embodiment of the present application provides a robot arm reasoning grasping device based on an anti-noise reinforcement world model, comprising:

[0241] A model construction module 1 is configured to construct an information filtering world model, a value network and a policy network, the information filtering world model comprising an encoder, a gated representation decomposition module, a contrastive causal gating module and a decoder; training the information filtering world model, the value network and the policy network to obtain a trained information filtering world model, a trained value network and a trained policy network;

[0242] The action prediction module 2 is configured to obtain the ambiguous instruction issued by the user at the current time step and input into the BidirPR-Grasp framework to obtain the masked RGB image, the masked depth image and the target object availability map at the current time step, and construct a first multi-modal perception vector at the current time step in combination with the joint activity data of the robot arm collected at the current time step. The first multi-modal perception vector at the current time step, the action and the hidden state are input into the trained information filtering world model. The first multi-modal perception vector at the current time step and the hidden state are first input into the encoder to obtain a representation vector at the current time step. The representation vector at the current time step is input into the gated representation decomposition module to screen the main effective component and the noise component of the representation vector at the current time step. The main effective component of the representation vector at the current time step is input into the decoder to obtain an observation vector at the current time step. The observation vector at the current time step, the action, the main effective component and the noise component of the representation vector are input into the contrastive causal gating module to obtain a final main effective representation at the current time step. The hidden state at the next time step is obtained by recursively updating the hidden state at the current time step, the action and the final main effective representation according to a state transition function. The hidden state at the next time step is input into the trained strategy network to obtain an action at the next time step.

[0243] The execution module 3 is configured to control the robot arm to perform the grasping action at the next time step based on the action at the next time step.

[0244] Figure 7 A hardware structure schematic diagram of an electronic device provided by the embodiment of the present application is shown in FIG. 7. As shown in the figure, the electronic device of the embodiment includes a processor 701 and a memory 702; the memory 702 is used to store computer execution instructions; and the processor 701 is used to execute the computer execution instructions stored by the memory to realize each step performed by the electronic device in the above-mentioned embodiment. For details, please refer to the related description in the foregoing method embodiment. Figure 7

[0245] Optionally, the memory 702 can be independent or integrated with the processor 701.

[0246] When the memory 702 is independently arranged, the electronic device further includes a bus 703 for connecting the memory 702 and the processor 701.

[0247] The embodiment of the present application further provides a computer storage medium, and the computer storage medium stores computer execution instructions. When the processor 701 executes the computer execution instructions, the method as described above is realized.

[0248] The embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by the processor 701, the method as described above is realized.​

[0249] In the embodiments of the present application, it should be understood that the disclosed device and method can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, another division manner can be used. For example, a plurality of modules can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the modules shown or discussed can be indirect coupling or communication connection through some interfaces, devices or modules, and can be electrical, mechanical or other forms.

[0250] The modules illustrated as separate components can or can not be physically separate, and the components illustrated as modules can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Part or all of the modules can be selected according to actual needs to implement the embodiments of the present application.

[0251] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each module can be physically present separately, or two or more modules can be integrated in one unit. The unit formed by the above modules can be realized in the form of hardware, or in the form of hardware plus software functional units.

[0252] The integrated modules realized in the form of software functional modules can be stored in a computer readable storage medium. The software functional modules stored in the storage medium include a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or the processor 701 to execute part of the steps of the method of each embodiment of the present application.

[0253] It should be understood that the processor 701 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor 701 can also be any conventional processor 701, etc. The steps of the method disclosed in the present application can be directly embodied as the execution of the processor 701 in hardware, or the execution of the combination of hardware and software modules in the processor 701.

[0254] The memory 702 can include a high-speed RAM memory and can also include a non-volatile storage NVM, for example at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a magnetic or optical disk, etc.

[0255] The bus 703 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 703 can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, the bus 703 in the drawings of the present application does not limit to only one bus 703 or one type of bus 703.

[0256] The storage medium described above can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0257] An exemplary storage medium is coupled to the processor 701, so that the processor 701 can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be an integral part of the processor 701. The processor 701 and the storage medium can be located in an Application Specific Integrated Circuit (ASIC). Of course, the processor 701 and the storage medium can also exist as discrete components in an electronic device or a host device.

[0258] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction-related hardware. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the foregoing storage medium includes various storage media that can store program codes, such as ROM, RAM, magnetic disk or optical disk, etc.

[0259] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A robotic arm reasoning and grasping method based on a noise-resistant enhanced world model, characterized in that, Includes the following steps: Construct an information filtering world model, a value network, and a policy network. The information filtering world model includes an encoder, a gated representation decomposition module, a contrastive causal gating module, and a decoder. The information filtering world model, value network, and policy network are trained to obtain a trained information filtering world model, a trained value network, and a trained policy network. The fuzzy commands issued by the user at the current time step are acquired and input into the BidirPR-Grasp framework to obtain a masked RGB image, a masked depth map, and a target object availability map for the current time step. These are then combined with the joint activity data of the robotic arm collected at the current time step to construct the first multimodal perception vector for that time step. The first multimodal perception vector, action, and latent state of the current time step are input into a trained information filtering world model. The first multimodal perception vector and latent state of the current time step first pass through the encoder to obtain the representation vector of the current time step. This representation vector is then input into the gated representation. The decomposition module filters and obtains the principal component and noise component of the representation vector at the current time step; the principal component of the representation vector at the current time step is input into the decoder to obtain the observation vector at the current time step; the observation vector, action, and the principal component and noise component of the representation vector at the current time step are input into the contrastive causal gating module to obtain the final principal representation at the current time step; the hidden state, action, and final principal representation at the current time step are recursively updated using a state transition function to obtain the hidden state at the next time step; the hidden state at the next time step is input into the trained policy network to obtain the action at the next time step. The robotic arm is controlled to perform the grasping action of the next time step based on the action of the next time step.

2. The robotic arm reasoning and grasping method based on a noise-resistant enhanced world model according to claim 1, characterized in that, The gated representation decomposition module includes a dimension-wise gated network and an element-wise multiplication operation, and its calculation process is as follows: ； ； ； in, Indicates the first A gated vector at each time step. Indicates the first output of the encoder The representation vector at each time step, Represents the space of real numbers. Indicates dimension; This represents the Sigmoid activation function. This represents element-wise multiplication; This represents the weight matrix of a dimension-wise gated network. Represents the bias vector of a dimension-wise gated network; and They represent the first The representation vector of each time step The main effect component and the noise component.

3. The robotic arm reasoning and grasping method based on a noise-resistant enhanced world model according to claim 2, characterized in that, The comparative causal gating module includes an action sequence compressor, a dimension embedding module, a query-key pair construction module, and a gating weighting module. Its calculation process is as follows: According to the The predicted action, predicted latent state, and predicted representation vector at the nth time step are recursively predicted using the state transition function. The predicted latent state and predicted representation vector at the nth time step, and the nth time step... The predicted latent state and predicted representation vector at the nth time step are input into the policy network to obtain the nth time step. For the predicted action at the nth time step, repeat the above steps until the nth time step is obtained. Predict the action at the nth time step, and the nth time step. The time step to the 1 The predicted action at the nth time step is constructed into the nth time step. The future at a time step Action sequence at each time step As shown in the following formula: ； ； in, , and when hour, , , , Indicates the first The implicit state at each time step Indicates the first The action at each time step; Represents the policy network, with the parameter set as follows: ; Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Indicates the first The predicted representation vector at each time step, Indicates the first Predicted latent state at each time step Indicates the first Predicted actions at each time step, Represents the state transition function. The set of parameters representing the state transition function; The first The future at a time step The action sequence at time step n is input into the action sequence compressor and compressed to obtain the nth time step n. Action condition vectors for each time step; The first The principal effective component of the representation vector at the nth time step is input into the dimension embedding module for dimension adjustment to obtain the nth time step. The principal components of the adjusted representation vector at each time step; The first The action condition vector at each time step, the principal component of the adjusted representation vector, the observation vector, and the noise component of the representation vector are input into the query-key pair construction module to obtain the first time step. The query vector, principal key vector, and noise key vector for each time step are shown in the following formula: ； ； ； in, Indicates the first Action condition vectors at each time step This represents the compression function corresponding to the action sequence compressor. Indicates splicing; Indicates the first The principal effective components of the adjusted representation vector at each time step This indicates the function corresponding to the dimension embedding module; Indicates the first The principal component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step Represents a multilayer perceptron; and They represent the first The principal key vector and noise key vector at each time step; The first The query vector, principal key vector, and noise key vector at each time step are input into the gated weighting module. First, based on the query vector, principal key vector, and noise key vector at each time step... The query vector, principal key vector, and noise key vector at the nth time step are calculated to obtain the nth time step. The contrastive log-likelihood vector for each time step is shown in the following equation: ； in, Indicates the first The contrastive log-likelihood vector at each time step This indicates temperature hyperparameters. Indicates the first The noise key vector at each time step , Represents the similarity function; According to the first The comparison log-likelihood vector at the nth time step is calculated to obtain the th... The controllability score vector for each time step is shown in the following formula: ； in, Indicates the first The expectation at each time step Indicates the first Controllability score vector for each time step; According to the The controllability score vector at the nth time step is calculated to obtain the first... The dimension-level gated vector at the nth time step, and for the nth time step We weight the principal effective components of the representation vector at the nth time step to obtain the nth time step. The final principal effect representation at each time step is shown in the following equation: ； ； in, Indicates the benchmark for the shift in scoring. This represents the temperature coefficient of the gated function; Indicates the first A dimensional gated vector at each time step; Indicates the first The final main effect characterization at each time step.

4. The robotic arm reasoning and grasping method based on a noise-resistant enhanced world model according to claim 3, characterized in that, The information filtering world model is constructed based on a cyclic state-space model, which includes a filtering system, an observation system, a dynamic system, and a reward system. In the filtering system, the information filtering system constructs the first... The first multimodal sensing vector at the nth time step, or the vector generated during training by the nth time step. The first time step is constructed from the masked RGB image, the masked depth map, and the target item availability map. The second multimodal sensing vector at the nth time step is sequentially passed through the encoder and the gated representation decomposition module to obtain the nth time step. The principal component and noise component of the representation vector at each time step are shown in the following equation: ； in, Indicates the first The first or second multimodal sensing vector at each time step The parameter set is The encoder, This indicates the gating representation decomposition module; The first The principal effective component of the representation vector at the nth time step is input into the decoder to obtain the nth time step. The observation vector at each time step is shown in the following equation: ； in, The parameter set is The decoder; The first The observation vector, action, and principal component and noise component of the representation vector at each time step are input into the contrastive causal gating module to obtain the first time step. The final principal effect representation at each time step is shown in the following equation: ； in, This indicates a comparison of causal gating modules; According to the The implicit state, action, and final principal effect representation at the nth time step are recursively updated using a state transition function to obtain the nth time step. The hidden state at each time step As shown in the following formula: ； During the training phase, the first The latent state and final main effect representation at the nth time step are input into the reward function to obtain the nth time step. The reward for each time step is shown in the following formula: ； in, The parameter set is The reward function, Indicates the first Rewards for each time step.

5. The robotic arm reasoning and grasping method based on a noise-resistant enhanced world model according to claim 4, characterized in that, The training process for the information filtering world model, value network, and policy network is divided into two stages: In the first stage, the information filtering world model is trained separately based on the comprehensive loss function to obtain the trained information filtering world model; the construction process of the comprehensive loss function is as follows: The mean square error is used to measure the first step of the decoder output. The difference between the observation vector and the principal effective component of the representation vector at each time step is used to obtain the observation reconstruction loss, as shown in the following equation: ； in, Indicates the observation reconstruction loss; Calculate the encoder output of the first The KL divergence between the posterior distribution of the hidden states at each time step and the standard Gaussian prior distribution is used to obtain the KL divergence regularization loss, as shown in the following equation: ； in, Indicates the first output of the encoder The posterior distribution of the representation vectors at each time step. This represents the standard Gaussian prior distribution. The KL divergence represents the distance between two probability distributions. This represents the KL divergence regularization loss; According to the The decomposition loss of the gated representation decomposition module is calculated by combining the principal component and noise component of the representation vector at each time step, as shown in the following equation: ； in, This represents the decomposition loss of the gated representation decomposition module; According to the The contrast loss of the contrastive log-likelihood vector at each time step is calculated for the contrastive causal gating module, as shown in the following formula: ； in, This represents the contrast loss of the contrast-cause-gated module; The comprehensive loss function is constructed as shown in the following equation: ； in, Represents the comprehensive loss function. These are the weight coefficients corresponding to the observation reconstruction loss, KL divergence regularization loss, reward prediction loss, decomposition loss of the gated representation decomposition module, and contrast loss of the contrastive causal gating module, respectively. In the second stage, the parameters of the trained information filtering world model are fixed, and the value network and policy network are trained respectively using information value-constrained noise-resistant reinforcement learning based on the total loss function, resulting in a trained value network and a trained policy network; the construction process of the total loss function is as follows: First, construct the objective function for the λ-Return multi-step return, as shown in the following equation: ； ； in, This represents the balance coefficient of the λ-Return multi-step return. Indicates the discount factor; Indicates the first The target value of the λ-Return multistep return at the time step is determined by the λ-Return multistep return at the λ-Return time step. The reward at the first time step and the first The value is obtained by recursively calculating the value after the implicit state at each time step is input into the value network. Indicates the first The target value of the multi-step λ-Return return at each time step; The parameter set is The value network Indicates the first The value obtained by inputting the implicit state at each time step into the value network; H represents the maximum time step of the λ-Return multi-step reward. This represents the target value of the multi-step return corresponding to the maximum time step; Then, the training loss of the value network is constructed as shown in the following formula: ； in, This represents the training loss of the value network. This represents the Huber loss function. For threshold hyperparameters, This represents the weight hyperparameter of the Bellman consistency regularization. This represents the squared residual of the first-step TD. Indicates the first The value obtained by inputting the implicit state of each time step into the value network; Next, the training loss of the policy network is constructed as shown in the following formula: ； in, This represents the training loss of the policy network; Indicates the first The hidden state at each time step Next strategy network selection Actions at each time step The probability, Indicates the first The hidden state at each time step The probability distribution of the next policy network choosing any action; Represents the hyperbolic tangent function; Indicates the first The advantage function at each time step This indicates that the gradient operation is stopped. This indicates the scaling hyperparameters. Represents the entropy regularity coefficient; Entropy represents the policy distribution; Subsequently, the information compression decoupling contrast loss is constructed as shown in the following equation: ； in, This indicates the information compression and decoupling contrast loss; Represents mutual information; The weight hyperparameters representing the information bottleneck term. This represents the balance coefficient hyperparameter. This represents the noise weighting hyperparameter. This represents the contrastive learning weight hyperparameters; Indicates the first The observation vectors at each time step are input into the parameter set as follows: Summary encoder The future summary obtained by compression; The InfoNCE loss under the denoising invariance task is expressed as follows: ； in, Indicates the first Each time step This represents the total number of time steps in the training phase. Finally, the total loss function is constructed as shown in the following equation: ； in, Represents the total loss function; represents the weight hyperparameters of the training loss of the value network, the training loss of the policy network, and the information compression decoupling contrast loss, respectively.

6. The robotic arm reasoning and grasping method based on a noise-resistant enhanced world model according to claim 4, characterized in that, The process of constructing the first or second multimodal sensing vector at the current time step includes: The masked RGB image, masked depth image, and target item availability map at the current time step are preprocessed and concatenated along the channel dimensions to obtain an image vector. The preprocessing methods include spatial dimension alignment, mask consistency verification and repair, feature standardization, and dimension and channel adaptation. The image vector is then used as a second multimodal perception vector. The process of constructing the first multimodal sensing vector at the current time step also includes: The joint activity data of the robotic arm at the current time step is subjected to feature standardization, dimension and channel adaptation, and channel dimension concatenation to obtain a joint activity feature vector. The joint activity data includes joint angle, angular velocity, and / or torque. The image vector is concatenated with the joint activity feature vector to obtain the first multimodal perception vector.

7. A robotic arm reasoning and grasping device based on a noise-resistant enhanced world model, characterized in that, include: The model building module is configured to build an information filtering world model, a value network, and a policy network. The information filtering world model includes an encoder, a gating representation decomposition module, a contrastive causal gating module, and a decoder. The information filtering world model, value network, and policy network are trained to obtain a trained information filtering world model, a trained value network, and a trained policy network. The action prediction module is configured to acquire the fuzzy commands issued by the user at the current time step and input them into the BidirPR-Grasp framework to obtain a masked RGB image, a masked depth map, and a target object availability map for the current time step. Combined with the joint activity data of the robotic arm collected at the current time step, a first multimodal perception vector for the current time step is constructed. This first multimodal perception vector, action, and latent state are then input into a trained information filtering world model. The first multimodal perception vector and latent state for the current time step are first processed by the encoder to obtain a representation vector for the current time step. This representation vector is then input into... The gated representation decomposition module filters to obtain the principal component and noise component of the representation vector at the current time step; the principal component of the representation vector at the current time step is input into the decoder to obtain the observation vector at the current time step; the observation vector, action, and the principal component and noise component of the representation vector at the current time step are input into the contrastive causal gating module to obtain the final principal representation at the current time step; the hidden state, action, and final principal representation at the current time step are recursively updated using a state transition function to obtain the hidden state at the next time step; the hidden state at the next time step is input into the trained policy network to obtain the action at the next time step. The execution module is configured to control the robotic arm to perform the grasping action of the next time step based on the action of the next time step.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images