A visual-state-force multi-modal observation fusion method based on attention mechanism for diffusion strategy

Abstract

This invention discloses a vision-state-force multimodal observation fusion method based on an attention mechanism for diffusion strategies, belonging to the field of intelligent robotic arm operation technology. To provide richer observation features, this invention stacks point cloud features and state features, then uses a multi-head attention mechanism for feature interaction to obtain fused features of point cloud and state features. Force and torque information data are input into a projection module, and the projected force and torque information data is position-encoded before being input into a force and torque information encoder to obtain force and torque features. Cross-modal information is extracted using a multi-head cross-attention mechanism. The obtained cross-modal information and the fused features of point cloud and state features are then simply concatenated to obtain vision-state-force multimodal observation fusion features. These features are then input into a diffusion strategy network to obtain the final action sequence to be executed by the robotic arm.

Description

Technical Field

[0001] This invention belongs to the field of intelligent operation technology for robotic arms, specifically relating to a multimodal observation fusion method based on an attention mechanism for diffusion strategies, which integrates vision, state, and force perception. Background Technology

[0002] In the intelligent operation of robotic arms, the multimodal fusion of visual and force information mainly includes the following methods:

[0003] Using optical pixel-based sensors as force signals for fusion, since these sensors transmit image information, this type of multimodal fusion is mostly divided into two stages. The first stage involves applying different image processing techniques to the two types of image information, followed by modal fusion of the two image features in the second stage. While optical pixel-based sensors provide more accurate force feedback, they are relatively expensive and require separate installation, which is somewhat inconvenient.

[0004] When training an agent using reinforcement learning, an encoder converts images into feature vectors, and a decoder predicts force / torque information. The difference between the predicted value and the actual force / torque information is used as an exploration reward to incentivize the agent to explore autonomously in the absence of external environments. However, this reward-based exploration approach is not suitable for behavioral cloning.

[0005] Current multimodal fusion models do not fully utilize force / torque information from force perception. Some studies employ crude feature extraction methods, such as only performing normalization without deep modeling or effective feature extraction mechanisms. Furthermore, feature extraction using causal convolution requires a long number of steps, potentially leading to latency. Regarding fusion methods, many studies directly concatenate the fusion data into feature vectors from other modalities. Although some studies have introduced fusion strategies such as PoE, the interaction between different modalities during the multimodal fusion process remains somewhat lacking. Summary of the Invention

[0006] The problem this invention aims to solve is to provide observation features with richer information. It proposes a multimodal observation fusion method based on an attention mechanism for diffusion strategies, which integrates visual, state, and force perception.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A multimodal observation fusion method based on an attention mechanism for diffusion strategies, comprising the following steps:

[0009] S1. Collect point cloud data, status information, force and torque information data;

[0010] S2. Input the point cloud data obtained in step S1 into the visual encoder for encoding processing to obtain point cloud features. Input the state information obtained in step S1 into the state encoder to obtain state features. After stacking the point cloud features and state features, use a multi-head attention mechanism to perform feature interaction to obtain the fused features of point cloud features and state features.

[0011] S3. Input the force and torque information data obtained in step S1 into the projection module, and after the projected force and torque information data is position-encoded, it is input into the force and torque information encoder to obtain the force and torque features.

[0012] S4. Use the force and torque features obtained in step S3 as query vectors, and the fusion features of point cloud features and state features obtained in step S2 as key vectors and value vectors, and input them into the multi-head cross-attention mechanism to extract cross-modal information.

[0013] S5. Perform a simple stitching operation on the cross-modal information obtained in step S4 and the fusion features of point cloud features and state features obtained in step S2 to obtain the multimodal observation fusion features of vision-state-force perception.

[0014] S6. Input the multimodal observation fusion features of vision-state-force obtained in step S5 into the diffusion strategy network to obtain the action sequence to be executed by the robotic arm.

[0015] Furthermore, in step S1, the point cloud data is acquired by a depth camera, the state information is the joint pose of the robotic arm, and the force and torque information data is acquired by a force / torque sensor located above the end effector. ,in The force is in the x-axis direction. The force is in the y-axis direction. The force is in the z-axis direction. The torque is in the x-axis direction. The torque is in the y-axis direction. The torque is in the z-axis direction.

[0016] Furthermore, the specific implementation method of step S2 includes the following steps:

[0017] S2.1. Input the point cloud data obtained in step S1 into the DP3 Encoder visual encoder for encoding processing to obtain 64-dimensional point cloud features. ;

[0018] S2.2. Input the state information obtained in step S1 into the state encoder based on a lightweight multilayer perceptron (MLP) to obtain 64-dimensional state features. ;

[0019] S2.3. The point cloud features obtained in step S2.1 and the state features obtained in step S2.2 Feature interaction is performed using a multi-head attention mechanism, which includes first performing a stacking operation, then using a multi-head self-attention function to interact with features to obtain a fused representation, and finally adding the obtained fused representation to the original feature representation after the stacking operation and performing a residual connection to obtain a fused feature of point cloud features and state features. The calculation expression is:

[0020]

[0021] in, For stacking operations, It is a multi-head self-attention function, calculated using 8 attention heads.

[0022] Furthermore, the specific implementation method of step S3 includes the following steps:

[0023] S3.1. The projection module consists of a Linear layer connected to a LayerNorm layer connected to a GELU activation function. The Linear layer maps the 6-dimensional force and torque information data at each time step to 64 dimensions. Then, LayerNorm is used for layer normalization, and finally, the GELU activation function is used for nonlinear mapping to obtain the projected force and torque information data. The expression is:

[0024]

[0025] in, , These are all learnable parameters of the Linear layer. For normalization layer, For activation function, This provides force and torque information for five time steps within the sliding window;

[0026] S3.2. Construct a force and torque information encoder based on the Transformer Encoder architecture, which consists of multiple stacked encoder layers. Each encoder layer includes two core sub-modules: a multi-head self-attention module and a feedforward neural network module. Set the number of self-attention heads in the encoder to 8 and the dimension of the feedforward network layer to 256.

[0027] S3.3. The projected force and torque information data obtained in step S3.1 is encoded with absolute position, and then input into the force and torque information encoder constructed in step S3.2 for deeper feature extraction, resulting in 64-dimensional force and torque features. .

[0028] Furthermore, in step S3, before being input to the encoder, the force and torque information data of the current time step are grouped into a set containing the previously continuous data. For a time series window of historical data at each time step, with n set to 5, the expression is:

[0029] … , ]

[0030] in, A sliding window with a size of 5 is used to input force / torque information for the current time step.

[0031] After inputting into the encoder, Remodeling In order to restore the original time step structure.

[0032] Furthermore, the specific implementation method of step S4 is as follows:

[0033] by As a query vector, and As key and value vectors, cross-modal information is further extracted through a multi-head cross-attention mechanism, enabling the query vector to... From key vector and value vector Selectively aggregate related information, and then use residual connections to combine the results of the multi-head cross-attention mechanism with... Perform residual connections to obtain cross-modal information. The expression is:

[0034]

[0035] in, The multi-head cross-attention function is calculated using eight attention heads. It is a 64-dimensional vector.

[0036] Furthermore, the specific implementation method of step S5 is as follows:

[0037] Will With fusion features A simple concatenation operation is performed to obtain the visual-state-force multimodal observation fusion feature with a dimension of 128 at the t-th time step. The expression is:

[0038] .

[0039] Furthermore, the specific implementation method of step S6 includes the following steps:

[0040] S6.1. Construct a diffusion policy network, abstracting the environment as a Markov decision process, using quintuples. It means that, among them A is the state space, and A is the action space. From state to state The state transition probability, That is, the reward function. It is a discount factor used to measure the importance of future rewards relative to immediate rewards;

[0041] The goal of imitation learning is to enable an agent to utilize the state space provided by an expert. and action space The data, the mapping function of the learning strategy, and all sequences of the expert's states and actions in the environment are called the trajectory set. Each of them has a length of The trajectory itself is represented using a list of state-action pairs, i.e. , ;

[0042] The target loss function of the diffusion strategy during training Defined as:

[0043]

[0044] in, Represents a noise prediction network. Represents the observation features at time step t. This represents the denoised output action sequence. This represents the scheduling step size during the diffusion process. Gaussian noise is added during the k-th step diffusion strategy. Mean squared error is used to measure the difference between predicted noise and actual noise.

[0045] S6.2. Input the multimodal observation fusion features of vision, state, and force obtained in step S5 into the diffusion policy network to obtain the output action sequence:

[0046] .

[0047] The beneficial effects of this invention are:

[0048] This invention discloses a multimodal observation fusion method based on an attention mechanism for diffusion strategies, which enhances the extraction of feature information from multi-step force / torque data within a temporal window through deep modeling. Subsequently, the feature vectors encoded by each modality are input into the designed multimodal fusion module, utilizing the attention mechanism for cross-modal interaction to achieve information complementarity and integration, and generating unified fused observation features for subsequent diffusion strategies to learn. Attached Figure Description

[0049] Figure 1 This is a flowchart of a multimodal observation fusion method based on an attention mechanism for diffusion strategies, as described in this invention;

[0050] Figure 2 This is a flowchart of a multimodal observation fusion method based on an attention mechanism for diffusion strategies, which integrates visual, state, and force perception.

[0051] Figure 3 This is a top-view diagram of some of the tasks selected in the simulation scenario of this invention, where (a) is the "closebox" task, (b) is the "water plants" task, and (c) is the "unplug charger" task.

[0052] Figure 4 This is a comparison chart showing the success rates of the multimodal fusion model proposed in this invention and the baseline model in simulation tasks. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0054] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0055] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 -Appendix Figure 4 Detailed explanation is as follows:

[0056] Example 1:

[0057] A multimodal observation fusion method based on an attention mechanism for diffusion strategies, comprising the following steps:

[0058] S1. Collect point cloud data, status information, force and torque information data;

[0059] Furthermore, in step S1, the point cloud data is acquired by a depth camera, the state information is the joint pose of the robotic arm, and the force and torque information data is acquired by a force / torque sensor located above the end effector. ,in The force is in the x-axis direction. The force is in the y-axis direction. The force is in the z-axis direction. The torque is in the x-axis direction. The torque is in the y-axis direction. The torque is in the z-axis direction;

[0060] S2. Input the point cloud data obtained in step S1 into the visual encoder for encoding processing to obtain point cloud features. Input the state information obtained in step S1 into the state encoder to obtain state features. After stacking the point cloud features and state features, use a multi-head attention mechanism to perform feature interaction to obtain the fused features of point cloud features and state features.

[0061] Furthermore, the specific implementation method of step S2 includes the following steps:

[0062] S2.1. Input the point cloud data obtained in step S1 into the DP3 Encoder visual encoder for encoding processing to obtain 64-dimensional point cloud features. ;

[0063] S2.2. Input the state information obtained in step S1 into the state encoder based on a lightweight multilayer perceptron (MLP) to obtain 64-dimensional state features. ;

[0064] S2.3. The point cloud features obtained in step S2.1 and the state features obtained in step S2.2 Feature interaction is performed using a multi-head attention mechanism, which includes first performing a stacking operation, then using a multi-head self-attention function to interact with features to obtain a fused representation, and finally adding the obtained fused representation to the original feature representation after the stacking operation and performing a residual connection to obtain a fused feature of point cloud features and state features. The calculation expression is:

[0065]

[0066] in, For stacking operations, It is a multi-head self-attention function, calculated using 8 attention heads.

[0067] S3. Input the force and torque information data obtained in step S1 into the projection module, and after the projected force and torque information data is position-encoded, it is input into the force and torque information encoder to obtain the force and torque features.

[0068] Furthermore, the specific implementation method of step S3 includes the following steps:

[0069] S3.1. The projection module consists of a Linear layer connected to a LayerNorm layer connected to a GELU activation function. The Linear layer maps the 6-dimensional force and torque information data at each time step to 64 dimensions. Then, LayerNorm is used for layer normalization, and finally, the GELU activation function is used for nonlinear mapping to obtain the projected force and torque information data. The expression is:

[0070]

[0071] in, , These are all learnable parameters of the Linear layer. For normalization layer, For activation function, This provides force and torque information for five time steps within the sliding window;

[0072] S3.2. Construct a force and torque information encoder based on the Transformer Encoder architecture, which consists of multiple stacked encoder layers. Each encoder layer includes two core sub-modules: a multi-head self-attention module and a feedforward neural network module. Set the number of self-attention heads in the encoder to 8 and the dimension of the feedforward network layer to 256.

[0073] S3.3. The projected force and torque information data obtained in step S3.1 is encoded with absolute position, and then input into the force and torque information encoder constructed in step S3.2 for deeper feature extraction, resulting in 64-dimensional force and torque features. .

[0074] Furthermore, in step S3, before being input to the encoder, the force and torque information data of the current time step are grouped into a set containing the previously continuous data. For a time series window of historical data at each time step, with n set to 5, the expression is:

[0075] … , ]

[0076] in, A sliding window with a size of 5 is used to input force / torque information for the current time step.

[0077] After inputting into the encoder, Remodeling In order to restore the original time step structure;

[0078] Furthermore, if there are fewer than 5 sampling steps before time step t (e.g., in the initial stage of the sequence), the data from the current time step is used to pad the data, ensuring that a complete 30-dimensional representation is generated at each time step. During preprocessing, Robust normalization is used to process the force / torque data. This is a normalization method based on the median and interquartile range, effectively suppressing the impact of extreme values ​​on the overall data distribution. Then, the Tanh activation function is introduced to map its feature space to the (-1,1) interval to maintain feature consistency among the data when input into the model. After being input into the encoder, it is first reshaped into a tensor of shape (5,6). This is done to restore the original time-step structure. To better capture the 6-dimensional data interactions at each time step, a projection module was constructed to process them.

[0079] Furthermore, absolute position encoding assigns a position-related vector to each time step in the input sequence and superimposes it onto the corresponding feature representation to model temporal dependencies.

[0080] S4. Use the force and torque features obtained in step S3 as query vectors, and the fusion features of point cloud features and state features obtained in step S2 as key vectors and value vectors, and input them into the multi-head cross-attention mechanism to extract cross-modal information.

[0081] Furthermore, the specific implementation method of step S4 is as follows:

[0082] by As a query vector, and As key and value vectors, cross-modal information is further extracted through a multi-head cross-attention mechanism, enabling the query vector to... From key vector and value vector Selectively aggregate related information, and then use residual connections to combine the results of the multi-head cross-attention mechanism with... Perform residual connections to obtain cross-modal information. The expression is:

[0083]

[0084] in, The multi-head cross-attention function is calculated using eight attention heads. It is a 64-dimensional vector.

[0085] Furthermore, multimodal fusion is based on an attention mechanism. The attention mechanism is a computational mechanism that simulates the selective attention ability in human visual and auditory cognition. It allows the model to focus on key parts of the input data when processing complex information, thereby improving processing performance. The core of the attention mechanism lies in its attention function, which is essentially a learnable mapping that accepts a query-key-value triple structure and outputs aggregated information. Generally, the calculation formula for the attention mechanism is as follows:

[0086]

[0087] in, , , These represent the query matrix, key matrix, and value matrix, respectively. The embedding dimension is represented by , and softmax is the normalization function. In practical applications of attention mechanisms, a multi-head attention (MHA) structure is typically used to enhance expressive power. Specifically, the input is projected into different subspaces through multiple independent linear transformations, each subspace corresponding to an attention head, each with independent learnable parameters. The aforementioned attention operation is then performed on each attention head. Subsequently, the outputs of all attention heads are concatenated and linearly projected through another set of learnable parameters to obtain a consistent output vector.

[0088] S5. Perform a simple stitching operation on the cross-modal information obtained in step S4 and the fusion features of point cloud features and state features obtained in step S2 to obtain the multimodal observation fusion features of vision-state-force perception.

[0089] Furthermore, the specific implementation method of step S5 is as follows:

[0090] Will With fusion features A simple concatenation operation is performed to obtain the visual-state-force multimodal observation fusion feature with a dimension of 128 at the t-th time step. The expression is:

[0091] .

[0092] S6. Input the multimodal observation fusion features of vision-state-force obtained in step S5 into the diffusion strategy network to obtain the action sequence to be executed by the robotic arm;

[0093] Furthermore, the specific implementation method of step S6 includes the following steps:

[0094] S6.1. Construct a diffusion policy network, abstracting the environment as a Markov decision process, using quintuples. It means that, among them A is the state space, and A is the action space. From state to state The state transition probability, That is, the reward function. It is a discount factor used to measure the importance of future rewards relative to immediate rewards;

[0095] The goal of imitation learning is to enable an agent to utilize the state space provided by an expert. and action space The data, the mapping function of the learning strategy, and all sequences of the expert's states and actions in the environment are called the trajectory set. Each of them has a length of The trajectory itself is represented using a list of state-action pairs, i.e. , ;

[0096] Furthermore, the imitation learning strategy used is a diffusion strategy, the core idea of ​​which is to model the action sequence as a process of progressively denoising from Gaussian noise. During the training phase, this method adds noise of varying intensities to real actions and trains the neural network to predict this noise. During the inference phase, the diffusion strategy starts from pure noise and progressively generates the action sequence through multiple denoising steps.

[0097] Furthermore, in practical applications, agents learn from demonstrations, which are typically obtained by sampling from one or more expert trajectories. Diffusion strategies, a type of behavior cloning, do not rely on reward signals or exploration of environmental interactions. Instead, they replicate expert behavior by learning directly from state-action pairs in demonstrations, aiming to make the model's policy behavior as similar as possible to the expert's policy behavior.

[0098] The target loss function of the diffusion strategy during training Defined as:

[0099]

[0100] in, Represents a noise prediction network. Represents the observation features at time step t. This represents the denoised output action sequence. This represents the scheduling step size during the diffusion process. Gaussian noise is added during the k-th step diffusion strategy. Mean squared error is used to measure the difference between predicted noise and actual noise.

[0101] S6.2. Input the multimodal observation fusion features of vision, state, and force obtained in step S5 into the diffusion policy network to obtain the output action sequence:

[0102] .

[0103] Furthermore, the overall architecture for observation feature fusion is as follows:

[0104]

[0105] in, For point cloud visual encoders, For status information encoder, Force / torque information encoder, This is a feature fusion encoder obtained by a multimodal feature fusion method. For point cloud data, For robot status information, This provides force / torque data from the sensor above the end effector.

[0106] Furthermore, in the area of ​​intelligent robotic arm operation, when the agent uses a diffusion strategy for imitation learning, the force / torque information extraction method and multi-modal feature fusion method provided by this invention can be utilized to provide a better state observation representation for the convolution-based diffusion strategy. In the diffusion strategy used for robots, the initial action... It is usually sampled from Gaussian noise, and then... After each iteration of denoising, a series of intermediate actions are generated until a noise-free action is finally generated. In this process, it is necessary to use an approximate conditional probability distribution. To accelerate the diffusion process and improve the accuracy of the generated action sequences, among which That means The agent's observation features at each moment. Since this method predicts actions based on observation information, the quality of the action sequence is directly affected by the information contained in the observation features. Currently, most modal feature inputs for diffusion strategies are mainly visual information.

[0107] The vision-force / torque fusion method proposed in this invention provides a novel multi-modal feature fusion approach, which can provide observational features with richer information for subsequent policy learning. For a robotic arm agent, during the execution of a target task, all observable or accessible state information in the current environment can be defined as follows: ,from A series of observational information can be obtained from this, such as point cloud data. Robot status information The force / torque data f / t from the sensor above the end effector. This is used when employing an existing point cloud visual encoder. Status information encoder The force / torque information encoder proposed in this patent and the feature fusion encoder obtained by the multimodal feature fusion method. Then the multi-mode observation features can be obtained as follows: This observation feature extracts complex interaction relationships of observation information at multiple scales. Compared with features dominated by visual modalities, it has richer information and can effectively provide richer representations for agents in the process of policy learning, helping agents to learn policies from expert data faster and perform better on target tasks.

[0108] As shown in Table 1, the experimental part of this invention selects multiple simulation tasks from RLBench, each of which includes a robotic arm and a series of task objects.

[0109] Table 1

[0110] Task Name Task Description close box Close the open box water plants Water the flowers with a watering can. unplug charger Unplug the charger Put rubbish in bin Put the trash in the trash can. close laptop lid Turn off the laptop Sweep to dustpan Sweep the trash into the dustpan with a broom. Slide block to target Push the block to the target position.

[0111] The "close box" task is relatively simple. In this task, the Franka robotic arm needs to close an open box randomly placed on a table. Due to the large size of the target object, occlusion is unlikely. The "water plants" task is a slightly more difficult task in the RLBench benchmark set. In this task, the robotic arm needs to pick up a watering can on the table, move it over a flowerpot, and pour water to simulate watering plants. It is challenging because robotic arm parts may obstruct the handle during the execution. The "unplug charger" task is more challenging. In this task, the robotic arm needs to unplug the charger from a power outlet on a wooden board and place it stably on the table. Due to the small size of the target object and the limited field of view, the observability of the object is low during the task, making it more challenging.

[0112] To verify the effectiveness of the multimodal fusion model in this patent, 50 expert demonstration data points were collected for each task in the table. If a task has multiple variations, only the first variation's demonstration data was collected, and testing was conducted on that variation. This patent uses the remaining skeleton parts without force / torque information and the multimodal fusion layer as the baseline model for comparison. For each simulation task, both models were trained using 50 expert demonstration data points. The hyperparameters, including the random seed number, were set identically during training. Model performance was evaluated every 400 epochs during training, for a total training period of 2800 epochs. Each evaluation involved 20 independent tests on the task, and the task success rate was used as the performance metric.

[0113] The final comparison results across all tasks are as follows Figure 4 As shown, each data point represents the best performance of the corresponding model at each checkpoint throughout the entire training cycle. This demonstrates that, under the same training conditions, the multimodal fusion model exhibits a higher task success rate than the baseline model, reflecting the superiority of multimodal information.

[0114] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for diffusion strategy based on attention mechanism visual-state-force multi-modal observation fusion, characterized in that, Includes the following steps: S1. Collect point cloud data, status information, force and torque information data; In step S1, the point cloud data is acquired by a depth camera, the state information is the joint pose of the robotic arm, and the force and torque information data is acquired by a force / torque sensor located above the end effector. ,in The force is in the x-axis direction. The force is in the y-axis direction. The force is in the z-axis direction. The torque is in the x-axis direction. The torque is in the y-axis direction. The torque is in the z-axis direction; S2. Input the point cloud data obtained in step S1 into the visual encoder for encoding processing to obtain point cloud features. Input the state information obtained in step S1 into the state encoder to obtain state features. After stacking the point cloud features and state features, use a multi-head attention mechanism to perform feature interaction to obtain the fused features of point cloud features and state features. The specific implementation method of step S2 includes the following steps: S2.

1. Input the point cloud data obtained in step S1 into the DP3 Encoder visual encoder for encoding processing to obtain 64-dimensional point cloud features. ; S2.

2. Input the state information obtained in step S1 into the state encoder based on a lightweight multilayer perceptron (MLP) to obtain 64-dimensional state features. ; S2.

3. The point cloud features obtained in step S2.1 and the state features obtained in step S2.2 Feature interaction is performed using a multi-head attention mechanism, which includes first performing a stacking operation, then using a multi-head self-attention function to interact with features to obtain a fused representation, and finally adding the obtained fused representation to the original feature representation after the stacking operation and performing a residual connection to obtain a fused feature of point cloud features and state features. The calculation expression is: ； in, For stacking operations, The function is a multi-head self-attention function, calculated using 8 attention heads; S3. Input the force and torque information data obtained in step S1 into the projection module, and after the projected force and torque information data is position-encoded, it is input into the force and torque information encoder to obtain the force and torque features. S4. Use the force and torque features obtained in step S3 as query vectors, and the fusion features of point cloud features and state features obtained in step S2 as key vectors and value vectors, and input them into the multi-head cross-attention mechanism to extract cross-modal information. S5. Perform a simple stitching operation on the cross-modal information obtained in step S4 and the fusion features of point cloud features and state features obtained in step S2 to obtain the multimodal observation fusion features of vision-state-force perception. S6. Input the multimodal observation fusion features of vision-state-force obtained in step S5 into the diffusion strategy network to obtain the action sequence to be executed by the robotic arm.

2. The multimodal observation fusion method based on an attention mechanism for diffusion strategies according to claim 1, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. The projection module consists of a Linear layer connected to a LayerNorm layer connected to a GELU activation function. The Linear layer maps the 6-dimensional force and torque information data at each time step to 64 dimensions. Then, LayerNorm is used for layer normalization, and finally, the GELU activation function is used for nonlinear mapping to obtain the projected force and torque information data. The expression is: ； in, , These are all learnable parameters of a linear layer. For normalization layer, For activation function, This provides force and torque information for five time steps within the sliding window; S3.

2. Construct a force and torque information encoder based on the Transformer Encoder architecture, which consists of multiple stacked encoder layers. Each encoder layer includes two core sub-modules: a multi-head self-attention module and a feedforward neural network module. Set the number of self-attention heads in the encoder to 8 and the dimension of the feedforward network layer to 256. S3.

3. The projected force and torque information data obtained in step S3.1 is encoded with absolute position, and then input into the force and torque information encoder constructed in step S3.2 for deeper feature extraction, resulting in 64-dimensional force and torque features. .

3. A multimodal observation fusion method based on an attention mechanism for diffusion strategies according to claim 2, characterized in that, Step S3 involves inputting the force and torque information data of the current time step into a data set containing the previously continuous data before it is input to the encoder. For a time series window of historical data at each time step, with n set to 5, the expression is: … , ]； in, A sliding window with a size of 5 is used to input force / torque information for the current time step. After inputting into the encoder, Remodeling This is to restore the original time step structure.

4. A multimodal observation fusion method based on an attention mechanism for diffusion strategies according to claim 3, characterized in that, The specific implementation method of step S4 is as follows: by As a query vector, and As key and value vectors, cross-modal information is further extracted through a multi-head cross-attention mechanism, enabling the query vector to... From key vector and value vector Selectively aggregate related information, and then use residual connections to combine the results of the multi-head cross-attention mechanism with... Perform residual connections to obtain cross-modal information. The expression is: ； in, The multi-head cross-attention function is calculated using eight attention heads. It is a 64-dimensional vector.

5. A multimodal observation fusion method based on an attention mechanism for diffusion strategies according to claim 4, characterized in that, The specific implementation method of step S5 is as follows: With fusion features A simple concatenation operation is performed to obtain the visual-state-force multimodal observation fusion feature with a dimension of 128 at the t-th time step. The expression is: 。 6. A multimodal observation fusion method based on an attention mechanism for diffusion strategies according to claim 5, characterized in that, The specific implementation method of step S6 includes the following steps: S6.

1. Construct a diffusion policy network, abstracting the environment as a Markov decision process, using quintuples. It means that among them A is the state space, and A is the action space. From state to state The state transition probability, That is, the reward function. It is a discount factor used to measure the importance of future rewards relative to immediate rewards; The goal of imitation learning is to enable an agent to utilize the state space provided by an expert. and action space The data, the mapping function of the learning strategy, and all sequences of the expert's states and actions in the environment are called the trajectory set. Each of them has a length of The trajectory itself is represented using a list of state-action pairs, i.e. , ; The target loss function of the diffusion strategy during training Defined as: ； in, Represents a noise prediction network. Represents the observation features at time step t. This represents the denoised output action sequence. This represents the scheduling step size during the diffusion process. Gaussian noise is added during the k-th step diffusion strategy. Mean squared error is used to measure the difference between predicted noise and actual noise. S6.

2. Input the multimodal observation fusion features of vision, state, and force obtained in step S5 into the diffusion policy network to obtain the output action sequence: 。

Images