A method for dexterous operation using two hands

Abstract

The present invention discloses the field of imitation learning in embodied intelligence, and in particular, a method for collaborative dexterous operation of both hands. This method proposes an innovative control framework, including a motion model and a classifier model, to address the high-freedom control and data collection problems of collaborative dexterous operation of both hands. The motion model includes a dual-arm model, a left-hand model, and a right-hand model, which respectively predict the motion instructions of the corresponding modules, decompose the high-freedom control process, and reduce the difficulty of model prediction; the classifier model outputs the category score of each module, and calls the motion model according to the classification result. In addition, for the training data of the four models, this paper proposes a modular collection strategy based on the existing remote control data collection method, which can effectively meet the training data requirements. The present invention decouples the two-hand collaborative control process and uses a classifier model and three motion models to achieve modular control, thereby effectively reducing the difficulty of model control and improving the success rate of two-hand collaborative tasks.

Description

Technical Field

[0001] The present invention relates to the field of imitation learning in embodied intelligence, and in particular to a method for dexterous bimanual collaborative operation. Background Art

[0002] In the field of dexterous manipulation, bimanual collaborative manipulation is becoming a research hotspot. Bimanual manipulation refers to a robot simulating human bimanual collaboration, using both hands to collaboratively complete tasks. This method of manipulation enables more complex and delicate tasks. Compared to traditional one-handed manipulation, bimanual manipulation can perform more complex tasks while exhibiting higher efficiency, stability, and precision, as well as a wider range of applications. By operating with both hands simultaneously, the robot can significantly improve task efficiency and shorten completion time. At the same time, sharing the load with both hands significantly improves operational stability and control accuracy, making it particularly suitable for scenarios requiring delicate manipulation. In addition, bimanual collaboration, with its coordination and flexibility, can better adapt to changing environments, support diverse task requirements, and further expand its application scenarios. These advantages give bimanual manipulation great potential in industrial manufacturing, medical surgery, home services, and other fields, making it an important direction for future robotics development.

[0003] While bimanual collaborative manipulation offers significant advantages, practical applications still face numerous challenges, primarily in high-degree-of-freedom control, feature extraction, and demonstration data collection. Robotic bimanual manipulation involves increased degrees of freedom, requiring simultaneous control of the position, velocity, and posture of both dexterous hands in three-dimensional space and dynamic matching. This places extremely high demands on the robot's real-time perception and adjustment capabilities. Regarding feature extraction, bimanual manipulation requires simultaneous attention to multiple regions of interest. As task complexity and steps increase, regions of interest may overlap or switch in time and space, becoming more diverse and dynamic. Image preprocessing modules must be able to capture these subtle changes and features from multiple regions. However, this reduces the ability to extract features from a single region, resulting in reduced accuracy or incomplete information, which in turn affects the accuracy of the overall manipulation. Demonstration data collection becomes more complex, as the operator must teleoperate a robot with increased degrees of freedom, increasing the operational complexity. Furthermore, the increased dimensionality of the action space requires more data for model training. These factors result in a lack of sufficient samples during model learning, making it difficult to learn effective bimanual collaborative strategies.

[0004] Therefore, it is necessary to develop a two-handed collaboration strategy to improve the high-degree-of-freedom control and complex visual feature extraction problems of two-handed collaboration, overcome the complexity of high-dimensional motion space and the challenges of feature extraction; at the same time, improve the data collection method, reduce the difficulty of remote operation, and improve data collection efficiency. Summary of the Invention

[0005] The present invention discloses a method for two-handed collaborative dexterous operation. This method addresses the high-degree-of-freedom control and data collection difficulties of two-handed collaborative dexterous operation, and proposes an innovative control framework, including a motion model and a classifier model. The motion model includes a dual-arm model, a left-hand model, and a right-hand model, which respectively predict the motion instructions of the corresponding modules, decomposing the high-degree-of-freedom control problem into three low-degree-of-freedom control problems, reducing the difficulty of model training and prediction; the classifier model predicts the possibility of calling each module in the next time step based on the observed image and the current position of the robot, and stipulates that only the module with the greatest possibility is called at each time step to predict the action and execute it, so as to avoid interference between different modules. In addition, for the training data of the four models, this paper makes adjustments based on the existing remote control data collection method and proposes a modular collection strategy that can effectively meet the training data requirements. The present invention reduces the difficulty of two-handed collaborative control through multi-dimensional technological innovation and a modular control strategy. It can adapt to different imitation learning models and be effectively applied to the field of two-handed collaborative dexterous operation.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] Step 1: Build a modular telecontrol data collection framework, use modular control to obtain a demonstration dataset, and build a training dataset based on the demonstration dataset, including a baseline training dataset for non-modular strategies, as well as a dual-arm training dataset, a left-hand training dataset, a right-hand training dataset, and a classifier training dataset for modular strategies.

[0008] Step 2: Use the imitation learning model as the action prediction model to process the input image and position and predict future actions;

[0009] Step 3: Create a classifier model based on the transformer architecture, process the input image and position, and predict the category probability;

[0010] Step 4: Optimize the loss function based on the dataset preparation method;

[0011] Step 5: Design a verification framework and specify how each model is called.

[0012] The present invention is also characterized in that:

[0013] Furthermore, the specific process of step 1 is:

[0014] During the remote control process, only one module is controlled to move at a time. The modules include the dual-arm module, the left-hand module and the right-hand module. The switching of the control modules is achieved through keyboard buttons.

[0015] At each time step t, save the image O t, all joint positions J t and target action a t , as a benchmark demonstration dataset , which can be used for training non-modular strategies.

[0016] Based on the benchmark dataset, the target action a t Replaced with module class c t , and the data of the first time step saved after each switching control module is repeatedly saved a certain number of times to improve the probability of successful switching of the classifier prediction result at the switching position. The obtained data set is used as the classifier demonstration data set .

[0017] Based on the benchmark demonstration dataset, all joint positions J t and target action a t Replaced with double arm joint position and double-arm target movements , and save the data only in the dual-arm control stage. In order to avoid the interference caused by the discontinuous dual-arm control time when using the input image / position block strategy or the output action block strategy, each time the dual-arm control is switched, the image O with the value of None is saved K times repeatedly. None 、Current position of both arms and a target action with a value of None , K should be greater than the maximum time step length involved in the input image / position block and the output action block, and the resulting dataset is used as the dual-arm demonstration dataset B arm , similarly get the left-hand demonstration data set B left and right-hand demonstration dataset B right :

[0018] The image data of the classifier demonstration dataset and the action model demonstration dataset both use the index representation of the benchmark dataset to reduce the dataset storage cost.

[0019] For the benchmark demonstration dataset, after specifying the length L of the benchmark training dataset, in order to avoid interference from benchmark datasets of different demonstrations when using the input image / position block strategy or the output action block strategy, K images with the value of None are added at the beginning of each benchmark demonstration dataset. None , the joint position J0 at the initial time step, and the action a with a value of None None , add K times of image O with value None at the end None ,,The joint position J at the last time step end , action a with value None None, k should be greater than the maximum time step length involved in the input image / position block and the output action block, and then all benchmark demonstration datasets from different demonstration data are spliced ​​to obtain the spliced ​​dataset , where N is the number of all benchmark datasets, and To avoid interference between different data sets, the head and tail padding is performed. , , and then perform Bconcat pruning based on the length L of the standard benchmark dataset, and finally obtain a benchmark dataset with the same length .

[0020] The processing methods of other classifier demonstration datasets and action model demonstration datasets are consistent with the benchmark demonstration dataset.

[0021] Furthermore, the imitation learning strategy described in step 2 is a behavior cloning model based on the transformer architecture.

[0022] Furthermore, the specific process of implementing the classifier model described in step 3 is as follows:

[0023] The classifier model uses a transformer encoder architecture, takes an image and the robot’s position as input, and outputs a category score.

[0024] First, the input image is divided into fixed-size image patches. Feature extraction is then performed on each image patch. The patches are then processed into fixed-length image feature tokens through a fully connected layer. The robot position is directly processed into a robot position token through the fully connected layer.

[0025] The category CLS is concatenated with the robot position token and image feature token and then input into the transformer encoder. The final output is the category score, and the loss function is the cross entropy loss.

[0026] Furthermore, the loss function adjustment method described in step 4 is:

[0027] Based on the dataset creation method, a test is performed before calculating the loss. If the value of the image corresponding to the current time step in the input image is None, the action prediction results or category prediction results of the training data to which the image belongs are not included in the loss calculation. If the action label or category label contains an action with a value of None, the action prediction results or category prediction results corresponding to some labels with a value of None are not included in the loss calculation.

[0028] Furthermore, the verification phase model calling method described in step 5 is:

[0029] During inference verification, each time step first collects the current camera image and all joint positions, then calls the classifier model, inputs the image and all joint positions, outputs the category score, and then calls the action model corresponding to the category with the highest score, inputs the image and specific joint positions, outputs the specific joint action and executes it. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is the overall framework of the steps of the present invention.

[0031] Figure 2 A design framework for classifier models. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present disclosure.

[0033] In order to solve the shortcomings of the prior art, the present invention discloses a method for dexterous operation of two hands. Figure 1 As shown, the method includes:

[0034] Step 1: Build a modular telecontrol data collection framework, use modular control to obtain a demonstration dataset, and build a training dataset based on the demonstration dataset;

[0035] Step 2: Use the imitation learning model as the action prediction model to process the input image and position and predict future actions;

[0036] Step 3: Create a classifier model based on the transformer architecture, process the input image and position, and predict the category probability;

[0037] Step 4: Adjust the loss function based on the dataset creation method;

[0038] Step 5: Design a verification framework and specify how each model is called.

[0039] In a specific example of the present disclosure, for step 2, the specific operations are as follows: using the ACT model as the action prediction model, setting the model action space dimensions to 12, 16, and 16, respectively, corresponding to the two-arm model, the left-hand model, and the right-hand model;

[0040] For step 3, if Figure 2As shown in the figure, the specific operation is as follows: a classifier model is designed based on the transformer encoder, which takes the camera image and robot position as input and outputs the category score. The input image is divided into patches of equal size and then passes through the CNN feature extractor and the fully connected layer to obtain the image feature token. The input robot position is directly passed through the fully connected layer to obtain the robot position token. The image feature token, robot position token and category CLS are spliced ​​and input into the transformer encoder. The output of the corresponding position of CLS is then processed by the MLP layer to obtain the category score. The model will output the category scores of multiple time steps in the future each time. At each time step, the past category scores of the time step will be weighted and summed as the final category score to smooth the switching between action models.

[0041] For step 4, the specific operation is as follows: before calculating the loss, a test is performed. For the ACT action model and the classifier model, if the value of the image corresponding to the current time step in the input image is None, the action prediction result or category prediction result of the training data to which the image belongs is not included in the loss calculation. If the action label or category label contains an action or category with a value of None, the action prediction result or category prediction result corresponding to some labels with a value of None is not included in the loss calculation. The calculated loss will be multiplied by a compensation coefficient n as the final loss. The compensation coefficient n is calculated as follows;

[0042]

[0043] Where N is the number of predicted actions or categories, B is the training batch size, C is the length of the action block or category block, and N real The number of predicted actions or categories for which actual participation loss is calculated;

[0044] For step 5, the specific operation is as follows: during inference verification, the current camera image and all joint positions are first collected at each time step, and then the classifier model is called, the image and all joint positions are input, and the category score is output. The current and past category scores of the time step are weighted and summed to obtain the final category score of the time step. Subsequently, the action model corresponding to the category with the highest score is called, the image and specific joint positions are input, and the specific joint action is output and executed.

[0045] Evaluation Metrics:

[0046] FPS: The number of frames processed per second by the model during inference, also known as the model's real-time performance indicator. A higher FPS indicates better algorithm performance.

[0047] Success rate: The ratio of the number of successfully executed tasks to the total number of executed tasks when deploying the strategy. A higher success rate indicates a more effective modular two-handed collaboration strategy.

[0048] Generalization: The success rate of performing tasks in scenes with target objects that do not appear in the dataset. Higher generalization indicates more effective modular bimanual collaboration strategies.

Claims

1. A method for coordinated dexterous operation of both hands, characterized in that: The specific steps of this method are as follows: Step 1: Build a modular telecontrol data collection framework, use modular control to obtain a demonstration dataset, and build a training dataset based on the demonstration dataset, including: Step 1.1 stipulates that during the remote control process, only one module is controlled at a time. The modules include the dual-arm module, the left-hand module, and the right-hand module. The switching of the control modules is realized by key input. The image is saved at each time step t. , all joint positions and target action a t , as a benchmark demonstration dataset ; Based on the benchmark dataset, the target action a t Replaced with module class c t , the obtained dataset is used as the classifier demonstration dataset ; Based on the benchmark dataset, all joint positions and target action a t Replaced with double arm joint position and double-arm target movements , and save the data only during the dual-arm control phase, and the resulting dataset is used as the dual-arm demonstration dataset , and similarly get the left-hand demonstration dataset and the right-hand demonstration dataset ; After the demonstration data is collected in step 1.2, due to the different lengths of different demonstration data sets, it is necessary to construct a training data set based on the demonstration data set, including a benchmark training data set for non-modular strategies, and a two-arm training data set for modular strategies, a left-hand training data set, a right-hand training data set, and a classifier training data set. For the benchmark demonstration data set B, after specifying the length L of the benchmark training data set, all benchmark data sets from different demonstration data are spliced ​​to obtain the spliced ​​data set. , where N is the number of all benchmark datasets, and To avoid interference between different data sets, the head and tail padding is performed, and then the length L of the standard benchmark data set is used. Crop and finally get a benchmark dataset of the same length ,The processing methods for other classifier demonstration datasets and action prediction model demonstration datasets are consistent; Step 2: Use the imitation learning behavior cloning model as the action prediction model to process the input image and position and predict future actions; Step 3: Create a classifier model based on the transformer architecture, process the input image and position, and predict the category probability; Step 4: Adjust the loss function based on the dataset creation method; Step 5: Design a verification framework and specify how each model is called.

2. The method for two-handed collaborative dexterous operation according to claim 1, characterized in that: The classifier model design includes: Step 3: The classifier model uses the transformer encoder architecture, takes an image and robot position as input, and outputs a category score. The input image and robot position are processed into image tokens and position tokens, which are then concatenated with the category CLS and processed by the transformer encoder to obtain the category score.

3. The method for two-handed collaborative dexterous operation according to claim 1, characterized in that: The calling methods of the action prediction model and the classifier model include: Step 5. During inference verification, at each time step, the current camera image and all joint positions are first collected. Then, the classifier model is called, the image and all joint positions are input, and the category score is output. Then, the action prediction model corresponding to the category with the highest score is called, the image and specific joint positions are input, and the specific joint action is output and executed.

Images