Two-hand collaborative flexible operation method

Abstract

The invention relates to the field of imitation learning in intelligent intelligence, in particular to a two-hand collaborative flexible operation method. According to the method, an innovative control framework is provided for solving the problems of high-degree-of-freedom control and data collection of two-hand collaborative flexible operation, and the framework comprises an action model and a classifier model. The motion model comprises a two-arm model, a left hand model and a right hand model, motion instructions of corresponding modules are predicted respectively, the high-degree-of-freedom control process is decomposed, and the model prediction difficulty is reduced; and the classifier model outputs the category score of each module, and calls the action model according to the classification result. Besides, for training data of the four models, a modular collection strategy is provided based on an existing remote control data collection mode, and the training data requirement can be effectively met. According to the method, the two-hand cooperative control process is decoupled, and modular control is realized by using the classifier model and the three action models, so that the model control difficulty is effectively reduced, and the two-hand cooperative task success rate is improved.

Description

Technical Field

[0001] The present invention relates to the field of imitation learning in embodied intelligence, and particularly to a method for dexterous operation with two hands coordinated. Background Art

[0002] In the field of dexterous operation, two - hand coordinated operation is gradually becoming a research hotspot. Two - hand operation means that the robot simulates the cooperation of human hands and uses two hands to complete tasks simultaneously. This operation mode can perform more complex and delicate tasks. Compared with traditional single - hand operation, two - hand operation can perform more complex tasks, and at the same time shows higher efficiency, stability, precision and a wider application range. By operating with two hands simultaneously, the robot can significantly improve the task efficiency and shorten the completion time. At the same time, by sharing the load with two hands, the stability and control precision of the operation are significantly improved, making it particularly suitable for scenarios that require fine operation. In addition, due to its coordination and flexibility, two - hand coordination can better adapt to changing environments, support diverse task requirements, and further expand the application scenarios. These advantages make two - hand operation show great potential in fields such as industrial manufacturing, medical surgery, and home service, and become an important direction for the future development of robots.

[0003] Although two - hand coordinated operation shows significant advantages, it still faces many challenges in practical applications, mainly reflected in three aspects: high - degree - of - freedom control, feature extraction, and demonstration data collection. Robot two - hand operation involves more degrees of freedom and requires simultaneously controlling the positions, speeds, and postures of two dexterous hands in three - dimensional space and achieving dynamic matching, which poses extremely high requirements for the robot's real - time perception and adjustment capabilities. In terms of feature extraction, two - hand operation needs to simultaneously focus on multiple regions of interest. As the task complexity and steps increase, the regions of interest may overlap or switch in time and space, becoming more diverse and dynamic. The image pre - processing module needs to be able to capture these subtle changes and the features of multiple regions, but this will lead to a decrease in the feature extraction ability for a single region, resulting in a reduction in the accuracy of the features extracted in a single region or incomplete information, thus affecting the overall operation accuracy. In terms of demonstration data collection, data collection becomes more complex. The operator needs to remotely operate a robot with more degrees of freedom, increasing the operation difficulty. In addition, the increase in the dimension of the action space also requires more data for model training. These factors lead to a lack of sufficient samples in the learning process of the model, making it difficult to learn effective two - hand coordination strategies.

[0004] Therefore, it is necessary to develop a two - hand coordination strategy to improve the high - degree - of - freedom control and complex visual feature extraction problems of two - hand coordination, overcome the complexity of the high - dimensional action space and the challenges of feature extraction; at the same time, improve the data collection method, reduce the remote - operation difficulty, and improve the data collection efficiency. Summary of the Invention

[0005] The present invention discloses a method for dexterous two - hand collaborative operation. Aiming at the problems of high - degree - of - freedom control and data collection in two - hand collaborative dexterous operation, an innovative control framework is proposed, which includes an action model and a classifier model. The action model includes an arm model, a left - hand model, and a right - hand model, which respectively predict the motion instructions of the corresponding modules, decomposing the high - degree - of - freedom control problem into three low - degree - of - freedom control problems, reducing the difficulty of model training and prediction; the classifier model predicts the possibility of calling each module at the next time step based on the observed image and the current position of the robot, and it is stipulated that only the module with the highest possibility is called to predict the action and execute at each time step to avoid interference between different modules. In addition, for the training data of the four models, based on the existing method of collecting tele - operation data, this paper makes adjustments and proposes a modular collection strategy, which can effectively meet the training data requirements. Through multi - dimensional technological innovation, the present invention reduces the difficulty of two - hand collaborative control through a modular control strategy, can adapt to different imitation learning models, and is effectively applied to the field of two - hand collaborative dexterous operation.

[0006] To achieve the above object, the present invention adopts the following technical solutions: Step 1: Construct a modular tele - operation data collection framework, use a modular control method to obtain a demonstration data set, and construct a training data set based on the demonstration data set, including a benchmark training data set for non - modular strategies, and an arm training data set, a left - hand training data set, a right - hand training data set, and a classifier training data set for modular strategies; Step 2: Adopt an imitation learning model as an action prediction model to process the input image and position and predict future actions; Step 3: Create a classifier model based on the transformer architecture to process the input image and position and predict class probabilities; Step 4: Optimize the loss function according to the data set production method; Step 5: Design a verification framework and stipulate the calling method of each model.

[0007] The features of the present invention also lie in: Furthermore, the specific process of Step 1 is as follows: During the tele - operation process, only one module is manipulated to move each time. The modules include an arm module, a left - hand module, and a right - hand module, and the switching of control modules is achieved through keyboard keys.

[0008] At each time step t, the image O t , all joint positions J t and the target action a t are saved as a benchmark demonstration data set , which can be used for the training of non - modular strategies.

[0009] Based on the benchmark dataset, replace the target action a t with the module category c t , and repeat the data of the first time step saved after each control module switch a certain number of times to increase the probability of successful switching of the classifier prediction result at the switching position. The obtained dataset is used as the classifier demonstration dataset .

[0010] Based on the benchmark demonstration dataset, replace all joint positions J t and the target action a t with the double-arm joint positions and the double-arm target action , and only save data during the double-arm control phase. To avoid interference caused by discontinuous double-arm control time when using the input image / position block strategy or the output action block strategy, each time switching to double-arm control, repeat and save the image O with the value of None K times None , the current double-arm position and the target action with the value of None . K should be greater than the maximum time step length involved in the input image / position block and the output action block. The obtained dataset is used as the double-arm demonstration dataset B arm . Similarly, the left-hand demonstration dataset B left and the right-hand demonstration dataset B right are obtained : The image data of the classifier demonstration dataset and the action model demonstration dataset are both represented by the indices of the benchmark dataset to reduce the dataset storage cost

[0011] For the benchmark demonstration dataset, after specifying the length L of the benchmark training dataset, to avoid interference from the benchmark datasets from different demonstrations when using the input image / position block strategy or the output action block strategy, add the image O with the value of None K times None , the joint position J0 at the initial time step, and the action a with the value of None None at the beginning of each benchmark demonstration dataset, and add the image O with the value of None K times None , the joint position J end at the last time step, and the action a with the value of None None at the end. K should be greater than the maximum time step length involved in the input image / position block and the output action block. Then, splice all the benchmark demonstration datasets from different demonstration data to obtain the spliced dataset , where N is the number of all benchmark datasets and are the head and tail padding to avoid interference between different datasets . , and then perform Bconcat cropping according to the length L of the standard benchmark dataset, and finally obtain a benchmark dataset with consistent lengths. .

[0012] The processing methods of other classifier demonstration datasets and action model demonstration datasets are the same as those of the benchmark demonstration dataset.

[0013] Furthermore, the imitation learning strategy described in step 2 is a behavior cloning model based on the transformer architecture.

[0014] Furthermore, the specific implementation process of the classifier model described in step 3 is as follows: The classifier model uses the transformer encoder architecture, takes the input image and the robot position, and outputs the class scores; First, divide the input image into image patches of a fixed size, then extract features from each image patch, and then process them through a fully connected layer into image feature tokens of a fixed length. The robot position will be directly processed through a fully connected layer into robot position tokens; The class CLS will be concatenated with the robot position token and the image feature tokens and then input into the transformer encoder together, and finally output the class scores. The loss function is the cross-entropy loss; Furthermore, the way to adjust the loss function described in step 4 is as follows: Based on the dataset production method, a detection will be performed before calculating the loss. If the value of the image corresponding to the current time step in the input image is None, it is stipulated that the action prediction result or class prediction result of the training data to which the image belongs does not participate in the loss calculation. If the action label or class label contains an action with a value of None, it is stipulated that the action prediction result or class prediction result corresponding to the partial label with a value of None for the action or class does not participate in the loss calculation; Furthermore, the way to call the model in the verification stage described in step 5 is as follows: During inference verification, at each time step, first collect the current camera image and all joint positions, then call the classifier model, input the image and all joint positions, output the class scores, and then call the action model corresponding to the class with the highest score, input the image and specific joint positions, and output and execute the specific joint actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is the overall framework of the steps of the present invention.

[0016] Figure 2 is the design framework of the classifier model. DETAILED DESCRIPTION OF THE INVENTION

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present disclosure.

[0018] To solve the deficiencies of the prior art, the present invention discloses a method for coordinated and dexterous operation of both hands, as Figure 1 shown, the method includes: Step 1: Construct a modular remote control data collection framework, use a modular control method to obtain a demonstration data set, and construct a training data set based on the demonstration data set; Step 2: Adopt an imitation learning model as an action prediction model, process the input image and position, and predict future actions; Step 3: Create a classifier model based on the transformer architecture, process the input image and position, and predict the class probability; Step 4: Adjust the loss function according to the data set production method; Step 5: Design a verification framework and specify the calling method of each model.

[0019] In a specific example of the present disclosure, for Step 2, the specific operation is: Adopt the ACT model as the action prediction model, and set the dimensions of the model action space to 12, 16, and 16 respectively, corresponding to the double-arm model, the left-hand model, and the right-hand model; For Step 3, as Figure 2 shown, the specific operation is: Design a classifier model based on the transformer encoder, use the camera image and the robot position as inputs, output the class score. After the input image is divided into patches of equal size, it passes through a CNN feature extractor and a fully connected layer to obtain image feature tokens. The input robot position directly passes through a fully connected layer to obtain robot position tokens. The image feature tokens, robot position tokens, and the class CLS are concatenated and then input into the transformer encoder. The output at the corresponding position of CLS is further processed by an MLP layer to obtain the class score. The model will output the class scores for multiple future time steps each time. At each time step, the weighted sum of the past class scores at that time step will be used as the final class score to smooth the switching between action models; For step 4, the specific operation is as follows: Before calculating the loss, perform a detection. For the ACT action model and the classifier model, if the value of the image corresponding to the current time step in the input image is None, it is stipulated that the action prediction result or class prediction result of the training data to which the image belongs does not participate in the loss calculation. If the action label or class label contains an action or class with a value of None, it is stipulated that the action prediction result or class prediction result corresponding to the partial label with a value of None for the action or class does not participate in the loss calculation. The calculated loss will be multiplied by a compensation coefficient n as the final loss. The calculation method of the compensation coefficient n is as follows;

[0020] where N is the number of predicted actions or classes, B is the training batch size, C is the length of the action block or class block, and N real is the number of predicted actions or classes actually participating in the loss calculation; For step 5, the specific operation is as follows: During inference and verification, at each time step, first collect the current camera image and all joint positions, then call the classifier model, input the image and all joint positions, and output the class score. Then, perform a weighted sum of the current and past class scores at this time step to obtain the final class score at this time step. Subsequently, call the action model corresponding to the class with the highest score, input the image and specific joint positions, and output and execute the specific joint action.

[0021] Evaluation metrics: FPS: The number of frames processed per second by the model during the inference phase, which is the real-time metric of the model. The higher the FPS, the better the performance of the algorithm.

[0022] Success rate: The ratio of the number of successfully executed tasks to the total number of executed tasks when deploying the strategy. The higher the success rate, the more effective the modular two-handed collaborative strategy.

[0023] Generalization: The success rate of executing tasks in the target object scenarios not present in the dataset. The higher the generalization, the more effective the modular two-handed collaborative strategy.

Claims

1. A method for coordinated dexterous operation of both hands, characterized in that: The specific steps of this method are as follows: Step 1: Build a modular telecontrol data collection framework, use modular control to obtain a demonstration data set, build a training data set based on the demonstration data set, including a benchmark training data set for non-modular strategies, and a dual-arm training data set, a left-hand training data set, a right-hand training data set, and a classifier training data set for modular strategies; Step 2: Use the imitation learning behavior cloning model as the action prediction model to process the input image and position and predict future actions; Step 3: Create a classifier model based on the transformer architecture, process the input image and position, and predict the category probability; Step 4: Adjust the loss function based on the dataset production method; Step 5: Design a verification framework and specify how each model is called.

2. A two-handed collaborative dexterous operation method according to claim 1, characterized in that: The modular remote control data collection framework specifically includes: Step 1.1, collect demonstration dataset; It is stipulated that only one module is controlled at a time during the remote control process. The modules include the double-arm module, the left-hand module, and the right-hand module. The switching of the control modules is realized by key input. The image O is saved at each time step t. t 、All joint positions J t and target action a t , as a benchmark demonstration dataset ; Based on the benchmark dataset, the target action a t Replaced with module class c t The obtained dataset is used as the classifier demonstration dataset ; Based on the benchmark dataset, all joint positions J t and target action a t Replaced with double arm joint position and double arm target action , and save the data only in the dual-arm control phase, and the obtained data set is used as the dual-arm demonstration data set B arm , and similarly get the left-hand demonstration data set B left and right hand demonstration dataset B right; Step 1.2: Create a training dataset based on the demonstration dataset. After the data is collected, it needs to be processed again because the lengths of different demonstration data sets are different. For the benchmark demonstration data set B, after specifying the length of the benchmark training data set L, all benchmark data sets from different demonstration data are spliced ​​to obtain the spliced ​​data set Where N is the number of all benchmark datasets, and To avoid interference between different data sets, the head and tail padding is performed, and then B is performed according to the length L of the standard benchmark data set. concat Crop to get a benchmark dataset of the same length The processing methods of other classifier demonstration datasets and action model demonstration datasets are the same.

3. The method for two-handed coordinated dexterous operation according to claim 1, characterized in that: The classifier model design includes: Step 3.

1. The classifier model uses the transformer encoder architecture, inputs the image and the robot position, and outputs the category score. The input image and the robot position will be processed into image tokens and position tokens, which are then concatenated with the category CLS and processed by the transformer encoder to obtain the category score.

4. The method for two-handed coordinated dexterous operation according to claim 1, characterized in that: The calling methods of the action model and the classifier model include: Step 5.1: During inference verification, each time step first collects the current camera image and all joint positions, then calls the classifier model, inputs the image and all joint positions, outputs the category score, and then calls the action model corresponding to the category with the highest score, inputs the image and specific joint positions, outputs the specific joint action and executes it.

Images