Double-arm robot imitation learning method and system based on action blocking and force sensing

Abstract

The invention discloses a double-arm robot imitation learning method and system based on action blocking and force perception, and the method comprises the steps: setting the pose of the tail end of a mechanical arm as an action, carrying out the action blocking, and collecting the action blocking and multi-mode observation information in a process that an expert operates robot teaching to complete a specified task; inputting the action blocks and the multi-modal observation information into a pre-established imitation learning model, constructing a mapping relationship between the action blocks and the multi-modal observation information, generating a predicted action sequence, and completing imitation learning model training; and deploying the trained imitation learning model to a real environment, obtaining real-time multi-modal observation information, dynamically generating corresponding action blocks according to the multi-modal observation information, outputting a robot action sequence according to the action blocks, and completing a specified task. According to the method, the accuracy of the position and force in the operation process can be ensured, fine and smooth operation skills can be learned only through a small amount of demonstration, and the task requirements of an unstructured environment and diversity of operation objects are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of dual-arm service robots, and particularly relates to a method and system for dual-arm robot imitation learning based on action chunking and force perception, which is particularly applicable to complex operation scenarios that require the integration of high-precision trajectory control and real-time force perception, and realizes precise and compliant dual-arm collaborative operation through imitation learning. Background Art

[0002] Dual-arm robots usually consist of two robotic arms and achieve coordinated movement of the two arms through a controller. They can imitate the collaborative manner of human arms and complete various complex operation tasks. Dual-arm robots have high flexibility and adaptability and can achieve efficient and precise operation within a limited working space. Although dual-arm robots have shown broad application prospects in multiple fields, they also face challenges in aspects such as operation accuracy, coordination ability, and compliant operation.

[0003] Dual-arm robots need to achieve high precision during operation to meet the requirements of complex tasks. However, since the operation errors of the robotic arms themselves will be doubled and amplified during dual-arm collaborative operation, it is necessary to improve the accuracy of the robotic arms. At the same time, dual-arm robots need to achieve precise coordination between the left and right arms to complete complex operation tasks. However, in practical applications, the collaborative working ability between the two arms is often restricted by various factors. In addition, when operating fragile or flexible items, dual-arm robots need to be able to sense and adjust the operation force to avoid damaging the items. However, traditional imitation learning methods only rely on position and visual information and do not integrate force perception information, so they cannot operate on such items.

[0004] In recent years, the application of dual-arm robots in home service scenarios has gradually expanded. In home scenarios, the operation objects are diverse, and dual-arm robots need to cope with various unstructured environments and diverse operation objects, which pose extremely high requirements for the dual-arm coordination ability and compliant operation of the robots. For example, there are various items of different shapes, sizes, and materials in the home environment, from fragile glass cups to soft fabrics, and the robot needs to be able to adapt to and accurately operate these items. At the same time, the home environment is often complex and changeable, with various obstacles and uncertain factors, and the robot needs to have high flexibility and adaptability. Dual-arm robots need to achieve precise coordination between the left and right arms to complete complex operation tasks such as grasping, carrying, and assembling. When operating fragile or flexible items, the robot needs to be able to sense and adjust the operation force to avoid damaging the items.

[0005] The traditional imitation learning method has the following problems in practical applications: First, the generalization ability is insufficient: when facing inputs not seen in the training set, it will give predicted values with certain errors, resulting in entering states that are even less seen, and generating larger and larger errors. This limitation makes it difficult for the robot to maintain stable performance when facing complex and changeable tasks in the home environment, and may even lead to task failure due to error accumulation. Second, the information fusion is insufficient: existing methods only rely on position and visual information and do not fuse force perception information. When operating fragile or flexible items, the lack of force perception information makes it impossible for the robot to accurately judge the operating force, and thus unable to perform safe and effective operations on such items. Summary of the Invention

[0006] The purpose of the present invention is to address the above problems in the existing technology, and provide a dual-arm robot imitation learning method and system based on action chunking and force perception, which can ensure the accuracy of position and force during the operation process, and can learn fine and compliant operation skills with only a small amount of demonstrations, meeting the task requirements of unstructured environments and diverse operation objects in home scenarios.

[0007] To achieve the above purpose, the present invention has the following technical solutions:

[0008] In the first aspect, a dual-arm robot imitation learning method based on action chunking and force perception is provided, including:

[0009] Set the end pose of the robotic arm as an action, and perform action chunking. During the process of an expert operating the robot to teach and complete a specified task, collect action chunks and multi-modal observation information.

[0010] Input the action chunks and multi-modal observation information into a pre-established imitation learning model, construct a mapping relationship between the action chunks and multi-modal observation information, generate a predicted action sequence, and complete the training of the imitation learning model.

[0011] Deploy the trained imitation learning model to the real environment, obtain real-time multi-modal observation information, dynamically generate corresponding action chunks according to the multi-modal observation information, and output the robot action sequence according to the action chunks to complete the specified task.

[0012] As a preferred solution, the multi-modal observation information includes any one or a combination of images from cameras with different perspectives, six-dimensional force at the end, robot joint angles, and end pose.

[0013] As a preferred solution, the imitation learning model is established using the generative model conditional variational autoencoder framework to generate an action sequence according to the observation.

[0014] As a preferred solution, the imitation learning model is divided into an encoder and a decoder, and both the encoder and the decoder are designed based on the Transformer structure.

[0015] As a preferred solution, the input of the encoder is the current observation information, the target action chunks from the expert-operated robot teaching, and a vector to be learned. The multi-modal features are extracted through the self-attention mechanism to generate a style vector that fuses the observation and action information.

[0016] As a preferred solution, the decoder integrates a visual feature extraction module and a Transformer encoder-decoder. The visual feature extraction module uses a deep convolutional network to extract feature maps from the red, green, and blue (RGB) images, retains the position information, performs feature concatenation with other observations and the style vector, and inputs them into the Transformer encoder-decoder. The loss function is used to model the action sequence to generate a predicted action sequence.

[0017] As a preferred solution, the visual feature extraction module retains the position information by adding sine position embeddings.

[0018] As a preferred solution, the loss function uses the L1 loss function.

[0019] As a preferred solution, in the step of deploying the trained imitation learning model to the real environment, obtaining real-time multi-modal observation information, and dynamically generating corresponding action chunks according to the multi-modal observation information, the actions predicted at different times for the same time are weighted and added, and after compliant processing, the corresponding action chunks are generated.

[0020] In a second aspect, a dual-arm robot imitation learning system based on action chunking and force perception is provided, including:

[0021] An action chunking and multi-modal observation information acquisition module, which is used to set the pose of the end of the robotic arm as an action and perform action chunking, and collect action chunking and multi-modal observation information during the process of the expert operating the robot to complete the specified task;

[0022] An imitation learning model training module, which is used to input the action chunking and multi-modal observation information into a pre-established imitation learning model, construct the mapping relationship between the action chunking and multi-modal observation information, generate a predicted action sequence, and complete the training of the imitation learning model;

[0023] A robot action sequence output module, which is used to deploy the trained imitation learning model to the real environment, obtain real-time multi-modal observation information, dynamically generate corresponding action chunks according to the multi-modal observation information, output the robot action sequence according to the action chunks, and complete the specified task.

[0024] Compared with the prior art, the first aspect of the present invention has at least the following beneficial effects:

[0025] Driven by expert teaching data, the present invention combines action chunking and multi-modal observation information to enable the dual-arm robot to learn high-precision and compliant dual-arm collaborative operation skills from expert demonstrations. By inputting action chunking and multi-modal observation information into the imitation learning model, a mapping relationship between action chunking and multi-modal observation information is constructed to generate a predicted action sequence. Force perception is added as observation information to ensure the accuracy of position and force during the operation process. In order to reduce the cumulative error during the long-time operation process and reduce the time-related noise during data acquisition, the end pose of the robotic arm is set as an action and action chunking is performed. The method of action chunking reduces the effective horizon of high-frequency data, and only a small number of demonstrations are required to learn fine and compliant operation skills, meeting the task requirements of unstructured environments and diverse operation objects in home scenarios, and having broad application prospects.

[0026] It can be understood that the beneficial effects of the above second aspect can refer to the relevant descriptions in the above first aspect and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 Schematic diagram of data processing for an expert to operate a robot to complete a specified task in an embodiment of the present invention;

[0029] Figure 2 Schematic diagram of the encoder structure of the imitation learning model in an embodiment of the present invention;

[0030] Figure 3 Schematic diagram of the decoder structure of the imitation learning model in an embodiment of the present invention;

[0031] Figure 4 Schematic diagram of action compliance processing during the deployment stage of the imitation learning model in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0033] Please refer to Figure 1 , an embodiment of the present invention provides a method for imitation learning of a dual-arm robot based on action chunking and force perception. Driven by expert teaching data and combined with action chunking and multi-modal observation information, precise and compliant operation of the dual-arm robot in a home scenario is achieved. The specific steps are as follows:

[0034] S1. Set the end pose of the robotic arm as an action and perform action chunking. During the process of an expert operating the robot to teach and complete a specified task, collect action chunking and multi-modal observation information.

[0035] S2. Input the action chunking and multi-modal observation information into a pre-established imitation learning model, construct a mapping relationship between the action chunking and the multi-modal observation information, generate a predicted action sequence, and complete the training of the imitation learning model.

[0036] S3. Deploy the trained imitation learning model to a real environment, obtain real-time multi-modal observation information, dynamically generate corresponding action chunks according to the multi-modal observation information, and output a robot action sequence according to the action chunks to complete the specified task.

[0037] In a possible implementation manner, in the data collection stage of step S1, the multi-modal observation information includes any one or a combination of images from different perspective cameras, end six-axis force, robot joint angles, and end pose. Set the images, end six-axis force, and robot joint angles as environmental observations, set the end pose of the robotic arm as an action, and perform action chunking.

[0038] In a possible implementation manner, the pre-established imitation learning model in step S2 is established using the framework of a generative model conditional variational autoencoder (CVAE) to generate an action sequence according to the observations.

[0039] The conditional variational autoencoder (CVAE) framework of the generative model is developed based on the variational autoencoder (VAE). By introducing conditional information to guide the data generation process, the network structure of CVAE mainly includes an encoder, a decoder, and a conditional input part. Among them, the encoder is used to receive the input data and conditional information, and compress the input data into two parameters in the latent representation space, namely, the mean and variance of the multi-dimensional Gaussian distribution. Then, a latent variable is obtained by sampling from the obtained distribution. The decoder is used to receive the latent variable and conditional information as inputs and decode them back to an approximation of the original input data. In this way, the decoder can generate data that meets the requirements according to the given conditions. The conditional input can be a class label, a text description, etc., and is used to guide the data generation process. The combination of the conditional information with the inputs of the encoder and decoder enables the model to generate data that meets specific conditions.

[0040] The loss function of CVAE usually includes two parts: the reconstruction loss and the KL divergence loss. The reconstruction loss is used to measure the difference between the generated data and the real data, and usually uses the mean squared error (MSE) or cross-entropy as the loss function. The KL divergence loss is used to measure the difference between the latent variable distribution and the prior distribution (usually the standard Gaussian distribution). By minimizing the KL divergence loss, the latent variable distribution learned by CVAE will approach the standard normal distribution.

[0041] The training process of CVAE is similar to that of VAE, and both are optimized by minimizing the loss function. The specific steps include:

[0042] Input the input data and conditional information into the encoder to obtain the mean and variance of the latent variable;

[0043] Sample a latent variable from the distribution of the latent variable;

[0044] Input the latent variable and conditional information into the decoder to generate the reconstructed data;

[0045] Calculate the reconstruction loss and the KL divergence loss, and update the parameters of the model through the backpropagation algorithm.

[0046] In a possible implementation, the imitation learning model is divided into an encoder and a decoder, both of which are designed based on the Transformer architecture. Transformer is a deep learning model architecture based on the self-attention mechanism (Self-Attention Mechanism). Transformer mainly consists of an encoder (Encoder) and a decoder (Decoder), both of which are stacked by multiple identical layers. Each layer contains two main sub-layers: the multi-head self-attention mechanism (Multi-Head Self-Attention) and the fully connected feed-forward network (Feed-Forward Network). The purpose of the self-attention mechanism (Self-Attention) is to capture the dependencies between different positions in the sequence and solve the long-distance dependency problem. The purpose of multi-head attention (Multi-Head Attention) is to allow the model to simultaneously focus on information from different representation subspaces at different positions, enhancing the model's expressive power. The query, key, and value matrices are linearly projected into multiple different subspaces respectively, and self-attention calculations are performed in parallel on each subspace. The outputs of all subspaces are concatenated and then linearly transformed once to obtain the final output. Since the self-attention mechanism itself does not contain position information, position encoding needs to be introduced to capture the positional relationship of elements in the sequence. The implementation method is to generate position encoding vectors using sine and cosine functions and add them to the input embedding vectors. After each sub-layer, there is a fully connected feed-forward network, which contains two linear transformations and a ReLU activation function. Residual connection, which adds the input of the sub-layer to the output, helps to alleviate the vanishing gradient problem. Normalizing the hidden layer output of each sample can accelerate training and improve stability.

[0047] The main working process of the encoder (Encoder) is divided into:

[0048] Input processing: The input sequence first passes through the embedding layer to obtain an embedding vector sequence.

[0049] Position encoding: Add the position encoding vector to the embedding vector to obtain the input representation containing position information.

[0050] Stacked layer processing: Each encoder layer processes the input in turn, captures the dependencies in the sequence through the self-attention mechanism, and performs a non-linear transformation through the feed-forward neural network.

[0051] The main working process of the decoder (Decoder) is divided into:

[0052] Masked Self-Attention: The self-attention mechanism in the decoder adopts a masking mechanism to ensure that when predicting the current position, only previous positions can be seen, avoiding information leakage.

[0053] Encoder-Decoder Attention: The decoder focuses on the output of the encoder through the attention mechanism to obtain context information.

[0054] Output Generation: The final output of the decoder undergoes a linear transformation and the Softmax function to obtain a probability distribution over the vocabulary, and the word with the highest probability is selected as the current prediction result.

[0055] Please refer to Figure 2 , in the embodiment of the present invention, the input of the imitation learning model encoder is the current observation information (without inputting images to speed up training), the target action chunks from the expert-operated robot teaching, and a vector to be learned. The self-attention mechanism is used to extract multi-modal features and generate a style vector that fuses the observation and action information.

[0056] Please refer to Figure 3 , in the embodiment of the present invention, the decoder of the imitation learning model integrates a visual feature extraction module and a Transformer encoder-decoder. Among them, the visual feature extraction module uses a deep convolutional network to extract feature maps from the red, green, and blue (RGB) images, retains the position information, performs feature concatenation with other observations and the style vector, and inputs them into the Transformer encoder-decoder. The loss function is used to model the action sequence to generate a predicted action sequence.

[0057] RGB (Red, Green, Blue) is an additive color model, and all colors in the visible spectrum are generated by mixing different intensities of the three primary colors of red, green, and blue. Each pixel consists of three channels (R, G, B), and the value of each channel is usually an integer from 0 to 255 (8 bits) or a floating point number from 0.0 to 1.0 (normalized), representing the intensity of the corresponding color. A pixel is the smallest unit of an image, and each pixel stores the values of the three channels, and the channels independently represent the intensities of red, green, and blue.

[0058] Furthermore, the visual feature extraction module of the embodiment of the present invention preserves position information by adding Sinusoidal Positional Encoding. Since the core of Transformer is the self-attention mechanism, it captures the dependencies between features by calculating the similarity between queries (Q), keys (K), and values (V). However, the self-attention mechanism itself does not contain position information, that is, it cannot distinguish elements at different positions in the input sequence. In visual tasks, the position information of features is crucial. For example, in image classification, the position of an object may affect its category; in object detection, the position of an object is the key information in the output. Therefore, in the visual feature extraction module, position information must be introduced to preserve the spatial structure. Sinusoidal positional encoding encodes position information into features by generating a unique embedding vector for each position using sine and cosine functions with different frequencies.

[0059] Furthermore, the loss function of the embodiment of the present invention adopts the L1 loss function. L1 Loss, also known as Mean Absolute Error (MAE), is used to measure the difference between the predicted value and the true value of the model. The L1 loss calculates the average of the absolute differences between the predicted value and the true value. Since the L1 loss uses absolute values instead of squares, outliers have less impact on the loss. In scenarios where feature selection is required, the L1 loss can help the model automatically select important features.

[0060] Please refer to Figure 4 , when the imitation learning model trained in step S3 of the embodiment of the present invention is deployed to the real environment, real-time multi-modal observation information is obtained, and when corresponding action chunks are dynamically generated according to the multi-modal observation information, the actions predicted at different times for the same time are weighted and added together. After smoothing processing, the corresponding action chunks are generated, which can cope with the real-time changes of observations without increasing the extra training cost. In the action generation task, the predictions at different time steps may jitter or mutate due to the uncertainty or noise of the model. By weighted adding the prediction results at the same time point, the information of multiple time steps can be fused to improve the stability and consistency of the prediction. Smoothing is to reduce the high-frequency noise and mutations in the prediction results to make the generated action sequence smoother and more natural. Smoothing methods include moving average, Gaussian filtering, exponential smoothing, etc.

[0061] The present invention proposes an imitation learning method that integrates action chunking and multi-modal observation. By synchronously collecting the six-dimensional force information at the end-effector during expert demonstrations, the force data is used as observation information to input into the imitation learning model, constructing the mapping relationship between observation and action, and performing chunk modeling on the action sequence to reduce the cumulative error of long-time operation tasks and improve the operation accuracy and compliance in the home scenario. The present invention designs an imitation learning method that maps multi-modal observation to action, using the policy as a generative model to model the expert demonstration actions, thus focusing on the parts that require fine operation during the operation for learning. The imitation learning method of the present invention adds force perception as an observation and generates an action sequence according to the observation to ensure the accuracy of position and force during the operation. In order to reduce the cumulative error during long-time operation and reduce the time-related noise during data acquisition, the method of action chunking is used to reduce the effective horizon of high-frequency data. Only a small number of demonstrations are required to learn fine and compliant operation skills, meeting the task requirements of unstructured environments and diverse operation objects in the home scenario, and having broad application prospects.

[0062] Another embodiment of the present invention also proposes a dual-arm robot imitation learning system based on action chunking and force perception, including:

[0063] An action chunking and multi-modal observation information acquisition module, configured to set the end-effector pose of the robotic arm as an action and perform action chunking, and collect action chunking and multi-modal observation information during the process of an expert operating the robot to teach and complete a specified task;

[0064] An imitation learning model training module, configured to input the action chunking and multi-modal observation information into a pre-established imitation learning model, construct the mapping relationship between the action chunking and multi-modal observation information, generate a predicted action sequence, and complete the training of the imitation learning model;

[0065] A robot action sequence output module, configured to deploy the trained imitation learning model to the real environment, obtain real-time multi-modal observation information, dynamically generate corresponding action chunks according to the multi-modal observation information, and output the robot action sequence according to the action chunks to complete the specified task.

[0066] In a possible implementation manner, the multi-modal observation information collected by the action chunking and multi-modal observation information acquisition module includes camera images from different perspectives, six-dimensional force at the end, robot joint angles, end-effector pose, etc.

[0067] In a possible implementation, the imitation learning model trained by the imitation learning model training module is established using the framework of the generative model Conditional Variational Autoencoder (CVAE), and an action sequence is generated according to the observation. The CVAE framework of the generative model guides the data generation process by introducing conditional information. The network structure of CVAE mainly includes an encoder, a decoder, and a conditional input part. The encoder is used to receive the input data and conditional information, and compress the input data into two parameters in the latent representation space, namely, the mean and variance of the multi-dimensional Gaussian distribution. Then, sampling is performed in the obtained distribution to obtain a latent variable. The decoder is used to receive the latent variable and conditional information as inputs and decode them back to an approximation of the original input data. In this way, the decoder can generate data that meets the requirements according to the given conditions. The conditional input can be a class label, a text description, etc., and is used to guide the data generation process. The combination of the conditional information with the inputs of the encoder and decoder enables the model to generate data that meets specific conditions.

[0068] Furthermore, the imitation learning model is divided into an encoder and a decoder, and both the encoder and the decoder are designed based on the Transformer structure. Transformer is a deep learning model architecture based on the Self-Attention Mechanism. Transformer mainly consists of an encoder and a decoder, both of which are stacked by multiple identical layers. Each layer contains two main sub-layers: the Multi-Head Self-Attention and the Feed-Forward Network. In the embodiment of the present invention, the input of the encoder is the current observation information, the target action chunks from the expert's teaching of the robot operation, and a vector to be learned. The multi-modal features are extracted through the self-attention mechanism to generate a style vector that fuses the observation and action information. The decoder integrates the visual feature extraction module and the Transformer encoder-decoder. The visual feature extraction module uses a deep convolutional network to extract feature maps from the red, green, and blue (RGB) images, retains the position information by adding sinusoidal position embeddings, performs feature concatenation with other observations and the style vector, and inputs them into the Transformer encoder-decoder. The loss function is used to model the action sequence to generate a predicted action sequence.

[0069] Furthermore, the loss function in the embodiments of the present invention adopts the L1 loss function. The L1 loss, also known as the Mean Absolute Error (MAE), is used to measure the difference between the predicted value of the model and the true value. The L1 loss calculates the average of the absolute differences between the predicted value and the true value. Since the L1 loss uses absolute values instead of squares, outliers have less impact on the loss. In scenarios where feature selection is required, the L1 loss can help the model automatically select important features.

[0070] In a possible implementation manner, when the robot action sequence output module deploys the trained imitation learning model to the real environment, obtains real-time multi-modal observation information, and dynamically generates corresponding action chunks according to the multi-modal observation information, it adds the actions predicted at different times for the same time step after weighted summation and performs smoothing processing to generate the corresponding action chunks, which can cope with the real-time changes of observations without increasing additional training costs. In the action generation task, the predictions at different time steps may jitter or mutate due to the uncertainty or noise of the model. By performing weighted summation on the prediction results at the same time point, the information of multiple time steps can be fused to improve the stability and consistency of the prediction. Smoothing processing is to reduce the high-frequency noise and mutations in the prediction results and make the generated action sequence smoother and more natural. Smoothing processing methods include moving average, Gaussian filtering, exponential smoothing, etc.

[0071] Another embodiment of the present invention also proposes an electronic device, including a processor and a memory. The processor is configured to execute a computer program stored in the memory to implement the dual-arm robot imitation learning method based on action chunking and force perception.

[0072] Another embodiment of the present invention also proposes a computer-readable storage medium. The computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, it implements the dual-arm robot imitation learning method based on action chunking and force perception.

[0073] The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable storage medium can include: any entity or device, medium, USB flash drive, removable hard disk, magnetic disk, optical disc, computer memory, read-only memory, random access memory, electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals. For the sake of convenience of description, only the parts related to the embodiments of the present invention are shown above. For the specific technical details not disclosed, please refer to the method part of the embodiments of the present invention. This computer-readable storage medium is non-transitory and can be stored in a storage device formed by various electronic devices, and can implement the execution process recorded in the method of the embodiments of the present invention.

[0074] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or a combination of multiple flows and / or blocks

[0076] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one or more of the flows Figure 1 or a combination of multiple flows and / or blocks

[0077] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions for implementing the functions specified in one process or multiple processes and / or one block or multiple blocks in the flow Figure 1 one process or multiple processes and / or blocks Figure 1 steps of the function specified in one block or multiple blocks.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention shall be covered by the protection scope of the claims of the present invention.

Claims

1. A method for imitation learning of a dual-arm robot based on action chunking and force perception, characterized in that, Including: Set the pose of the end effector of the robotic arm as an action and perform action chunking. During the process of an expert operating the robot to complete a specified task through teaching, collect the action chunks and multi-modal observation information; Input the action chunks and multi-modal observation information into a pre-established imitation learning model, construct the mapping relationship between the action chunks and multi-modal observation information, generate a predicted action sequence, and complete the training of the imitation learning model; Deploy the trained imitation learning model to the real environment, obtain real-time multi-modal observation information, and dynamically generate corresponding action chunks according to the multi-modal observation information, and output the robot action sequence according to the action chunks to complete the specified task.

2. The method for imitation learning of a two-armed robot based on action chunking and force perception according to claim 1, wherein The multi-modal observation information includes any one or a combination of images from cameras with different perspectives, six-dimensional force at the end, robot joint angles, and end effector pose.

3. The method for imitation learning of a dual-arm robot based on action chunking and force perception according to claim 1, wherein The imitation learning model is established using the generative model conditional variational autoencoder framework to generate an action sequence based on the observation.

4. The method for imitation learning of a two-armed robot based on action chunking and force perception according to claim 3, wherein The imitation learning model is divided into an encoder and a decoder, and both the encoder and the decoder are designed based on the Transformer structure.

5. The method for imitation learning of a two-armed robot based on action chunking and force perception according to claim 4, wherein The input of the encoder is the current observation information, the target action chunks from the expert operating the robot through teaching, and a vector to be learned. It extracts multi-modal features through the self-attention mechanism to generate a style vector that fuses the observation and action information.

6. The method for imitation learning of a two-armed robot based on action chunking and force perception according to claim 4, wherein The decoder integrates a visual feature extraction module and a Transformer encoder-decoder. The visual feature extraction module uses a deep convolutional network to extract feature maps from RGB images of red, green, and blue primary colors, retains the position information, performs feature concatenation with other observations and the style vector, inputs it into the Transformer encoder-decoder, and models the action sequence using a loss function to generate a predicted action sequence.

7. The method for imitation learning of a two-armed robot based on action chunking and force perception according to claim 6, wherein The visual feature extraction module retains the position information by adding sinusoidal position embeddings.

8. The method for imitation learning of a two-armed robot based on action chunking and force perception according to claim 6, characterized in that, The loss function uses the L1 loss function.

9. The method for imitation learning of a dual-arm robot based on action chunking and force perception according to claim 1, wherein In the step of deploying the trained imitation learning model to the real environment, obtaining real-time multi-modal observation information, and dynamically generating corresponding action chunks according to the multi-modal observation information, the actions at the same time predicted at different times are weighted and added together, and after compliant processing, the corresponding action chunks are generated.

10. A dual-arm robot imitation learning system based on action chunking and force perception, characterized in that, Including: An action chunk and multi-modal observation information acquisition module, which is used to set the pose of the end effector of the robotic arm as an action and perform action chunking. During the process of an expert operating the robot to complete a specified task through teaching, collect the action chunks and multi-modal observation information; An imitation learning model training module, which is used to input the action chunks and multi-modal observation information into a pre-established imitation learning model, construct the mapping relationship between the action chunks and multi-modal observation information, generate a predicted action sequence, and complete the training of the imitation learning model; A robot action sequence output module, which is used to deploy the trained imitation learning model to the real environment, obtain real-time multi-modal observation information, and dynamically generate corresponding action chunks according to the multi-modal observation information, and output the robot action sequence according to the action chunks to complete the specified task.

Images