A multi-task dexterous hand operation method based on adaptive bidirectional distillation

Abstract

The application discloses a kind of multi-task dexterous hand operation methods based on adaptive bidirectional distillation, comprising: constructing multiple single-task expert strategies based on visual-haptic pre-training, the state-action trajectory of high degree of freedom dexterous manipulator on multiple operation tasks is collected;Using online distillation, the multi-task expert data is distilled into the unified diffusion strategy of shared perception representation and task coding, and the action multimodal distribution is modeled by diffusion action generation;Adaptive routing network is introduced, and each expert supervision weight is dynamically allocated according to the current observation state, to realize the weighted distillation of mixed experts;After the performance of unified strategy is better than expert, action-reward joint constraint is used, and the unified diffusion strategy is reversely distilled into each task expert strategy, and bidirectional iterative distillation cycle is carried out.Through the forward and reverse knowledge transfer between unified strategy and task expert, the observation distribution deviation and task imbalance problem in multi-task dexterous operation can be effectively alleviated.

Description

Technical Field

[0001] This invention relates to the fields of robot control and machine learning, specifically to a multi-task dexterous hand operation method based on adaptive bidirectional distillation. Background Technology

[0002] High-degree-of-freedom (DoF) anthropomorphic multi-fingered robotic hands hold significant promise for performing dexterous maneuvers similar to human hands, such as grasping, rotating, and redirecting. However, the multi-DoF action space and complex hand-object contact dynamics significantly increase the difficulty of control, rendering traditional model predictive control based on precise dynamic modeling and reinforcement learning methods that directly explore the high-dimensional action space inadequate in terms of sample efficiency and stability. Existing research indicates that combining visual and tactile pre-training with reinforcement learning can improve policy performance and sample efficiency in single-task dexterous operations, but these methods typically train independent expert policies for a single task, lacking the ability to simultaneously perform multi-task dexterous operations under a unified policy.

[0003] In multi-task reinforcement learning scenarios, directly adopting a unified strategy to learn multiple operational tasks often faces the following problems: First, the action subspaces corresponding to different tasks differ significantly, requiring the learning process to switch between multiple subspaces; second, the reward scales for each task are inconsistent, easily leading to inter-task competition, negative transfer, and imbalance, thus significantly degrading the performance of some tasks. To alleviate data acquisition costs and training difficulties, some works employ behavior cloning methods based on offline expert demonstrations, obtaining a unified strategy through supervised learning of multi-task demonstration data. However, in behavior cloning, the observed distribution of training data differs from the state distribution accessed by the strategy in real interactions, causing an observation distribution bias. The strategy is prone to error accumulation in states outside the training data, a problem particularly prominent in dexterity operations with rich contact and high finger coordination requirements.

[0004] Online distillation methods such as DAgger mitigate the observation distribution bias problem to some extent by continuously querying experts for actions during policy execution and incorporating newly accessed states into the training set. Related work shows that, under the premise of shared perceptual representations, student policies can potentially outperform teacher policies, indicating that in multi-task scenarios, unified policies have the potential to learn richer skills than single-task experts through cross-task experience sharing. However, most existing methods only consider unidirectional "expert-to-student" distillation, failing to fully utilize the cross-task skills inherent in the unified policy to further enhance the policies of individual single-task experts.

[0005] On the other hand, diffusion models have gained increasing attention in policy representation in recent years. Compared with traditional discriminative policy networks that assume actions follow a unimodal Gaussian distribution, diffusion-based generative policies can better fit complex multimodal action distributions through a multi-step denoising process, making them particularly suitable for describing dexterous manipulation scenarios with dense contact and diverse action patterns. However, a systematic solution is still lacking for how to leverage the generative advantages of diffusion policies in multi-task settings and form an efficient bidirectional distillation mechanism with expert policies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a multi-task dexterous hand manipulation method based on adaptive bidirectional distillation. The goal is to address the problems of observation distribution bias, task imbalance, and difficulty in leveraging a unified strategy to enhance expert policies in existing multi-task dexterous hand manipulation techniques. This method constructs a multi-task expert set and a unified diffusion strategy, introduces an adaptive routing network and an online DAgger distillation mechanism, and designs a back-distillation process from the unified strategy to the expert strategy, forming a bidirectional iterative improvement framework of "expert → unified strategy → expert," thereby achieving cross-task skill sharing and continuous performance improvement in multi-task dexterous hand manipulation.

[0007] The objective of this invention is achieved through the following technical solution: a multi-task dexterous hand operation method based on adaptive bidirectional distillation, comprising the following steps:

[0008] The construction of a multi-task dexterity operation expert set includes: training several dexterity operation tasks separately based on a pre-trained vision-tactile perception model to obtain multiple single-task expert policies, and collecting state-action trajectories containing multimodal observation spaces as a multi-task expert demonstration dataset.

[0009] Construct a unified diffusion strategy model, including:

[0010] The environmental state and task identifiers of the multi-task expert demonstration dataset are feature-encoded and concatenated to form the conditional input of a unified diffusion strategy;

[0011] Based on expert demonstration data and conditional input, forward diffusion and denoising learning are performed to model the multimodal distribution of the action space and obtain the action generation strategy given the current environmental state and task identifier.

[0012] Bidirectional distillation iteration: The loss weight vector of each expert policy is obtained using an adaptive routing network. Online knowledge distillation is performed on the unified diffusion policy model based on the loss weight vector. When the unified diffusion policy model reaches a preset threshold, an action reward term is introduced to perform back distillation to obtain a performance-enhanced expert policy set. The online knowledge distillation and back distillation are iteratively repeated until the multi-task success rate converges or reaches a preset threshold, resulting in the final multi-task unified policy and improved expert policy set.

[0013] Furthermore, the process of acquiring the multiple single-task expert strategies specifically includes:

[0014] The robot system uses egocentric RGB cameras, tactile sensors deployed on the fingertips and palms, and joint encoder readings to form a multimodal observation space, which includes visual images, tactile readings, and proprioceptive states.

[0015] The vision-tactile pre-trained encoder is used to map RGB images and tactile signals to low-dimensional latent representations, which are then concatenated with robot proprioceptive features and input into a task-specific policy network to output high-dimensional joint space motion vectors.

[0016] Furthermore, the acquisition of state-action trajectories in the multimodal observation space specifically involves: when the success rate of a single-task expert on their respective task reaches a preset threshold, their parameters are frozen, and a large number of state-action trajectories are acquired in a simulation environment using these expert strategies as an expert demonstration dataset for subsequent unified diffusion strategy distillation; each sample in the state-action trajectory includes the current visual-tactile observation, proprioceptive state, task identifier, and the corresponding expert output action vector.

[0017] Furthermore, the feature encoding and concatenation of the environmental state and task identifiers in the multi-task expert demonstration dataset specifically involves: employing a multi-branch feature encoding structure; the first branch encodes the visual image using a convolutional neural network to output visual features; the second branch encodes the tactile signal temporally or spatially to obtain tactile features; the third branch encodes the proprioceptive information using a multilayer perceptron to obtain proprioceptive features; the visual and tactile features are fused along the channel dimension to form a visual-tactile joint feature, which is then concatenated with the proprioceptive features along the feature dimension to obtain a unified environmental state feature representation; discrete task IDs are mapped to low-dimensional task embedding vectors, which are then concatenated with the environmental state feature representation as conditional inputs to the diffusion strategy model.

[0018] Furthermore, the forward diffusion of the unified diffusion strategy model specifically involves: extracting action sequences from expert demonstration data. Treating the action sequence as a multidimensional continuous random variable, sampling the action sequence at a spread time step in each training iteration. and Gaussian noise Calculate noisy actions based on a preset noise scheduler. , making In the steps Noise injection results at:

[0019]

[0020] in, This indicates the transition from step 1 to step 2. The cumulative product of all noise attenuation factors is defined as: , This represents the noise attenuation factor.

[0021] Furthermore, the denoising learning of the unified diffusion strategy model specifically involves sampling initial actions from a standard Gaussian distribution. At each diffusion time step By using the current noisy action Time step A denoising network based on environmental state characteristics predicts noise or reconstructs actions, and the actions of the previous time step are inferred from the noise scheduler. The final motion output is obtained through multiple iterations of noise reduction. Used to control the dexterous robotic arm to perform the current task:

[0022]

[0023] in It is Gaussian noise. and Determined by the noise scheduler The function executes the above steps. After several iterations, the denoised motion output is finally obtained. .

[0024] Furthermore, the method of obtaining the loss weight vector of each expert policy using the adaptive routing network is as follows: the input environment state includes visual and tactile features, proprioceptive features, and task ID features, and is passed sequentially through a multilayer perceptron, a ReLU activation function, a second MLP, and a softmax layer for normalization. The output is a probability distribution vector with a dimension equal to the number of experts. Each component of the probability distribution vector is used as the loss weight of the corresponding expert. The action matching loss between the unified diffusion policy and each expert policy is weighted and summed to form a weighted distillation loss function.

[0025] Furthermore, the online knowledge distillation of the unified diffusion strategy model based on the loss weight vector specifically involves:

[0026] In the simulation environment, the current unified diffusion strategy is used for interaction. The states accessed under this strategy are collected, and at each access state, a multi-task expert strategy is invoked to generate supervised actions. The access state and expert action pairs are added to the distillation dataset. The unified diffusion strategy is trained using the updated distillation dataset, thereby enabling the unified diffusion strategy to approximate the expert actions under its own induced state distribution. In the weighted loss function of the multi-task expert distillation trained by Dagger, the mean squared error loss is used to minimize the L2 distance between the actions generated by the diffusion-based unified strategy and the actions generated by the multi-task hybrid expert. Specifically:

[0027]

[0028] in This represents the loss weights of different experts. This represents the sequence of actions in the expert demonstration. Indicates the discrete diffusion time step. The first term generated by closed reparameterization of forward diffusion The noise action of each diffusion step, This represents a denoising network. Indicates the denoising network in the environmental state The actions generated below; For visual and tactile features, For the robot's proprioceptive features, This is a task ID feature.

[0029] Furthermore, the backdistillation includes: when the success rate of the unified diffusion strategy on each task exceeds the average level of the initial single-task expert strategy, the unified diffusion strategy is regarded as a new teacher, and backdistillation is performed on the expert strategies for each task. During the backdistillation process, an action reward is introduced to encourage the expert strategies to output actions that are close to those generated by the unified strategy. The action reward is the negative mean square error between the action output by the unified diffusion strategy and the action output by the expert strategy. The task reward is the environmental reward for completing the operation objective under the corresponding task. The total reward is the sum of the action reward and the task reward.

[0030] According to another aspect of the specification, a multi-task dexterous hand operating device based on adaptive bidirectional distillation is also provided, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, it implements the aforementioned multi-task dexterous hand operating method based on adaptive bidirectional distillation.

[0031] The beneficial effects of this invention are:

[0032] 1. By combining a unified diffusion strategy with an adaptive routing network, the experience from multiple single-task experts is effectively integrated, and the overall performance of multi-task agile operation is improved by leveraging cross-task skill sharing.

[0033] 2. By using online DAgger distillation, the unified policy learns expert actions on its own induced state distribution, significantly mitigating the performance degradation caused by observation distribution shift.

[0034] 3. By introducing a back-distillation process from a unified strategy to an expert strategy, the multi-task collaborative capabilities learned from the unified strategy are fed back to the individual task experts, realizing a two-way knowledge transfer of "students improving teachers".

[0035] 4. By leveraging the diffusion model to model the multimodal distribution of actions, the unified strategy can generate diverse and high-quality action sequences in complex contact scenarios, improving the generalization ability for unseen objects and deformable tasks.

[0036] 5. In multiple practical dexterity tasks, the average success rate of the method of this invention is more than 10% higher than that of the single-task expert baseline pre-trained by vision-tactile perception, while exhibiting lower performance variance and stronger stability in both seen and unseen object scenarios.

[0037] 6. This invention can be used for autonomous operation in real-world scenarios and has been successfully deployed. Attached Figure Description

[0038] Figure 1 This is an architecture diagram of a multi-task dexterous hand operation method based on adaptive bidirectional distillation in an embodiment of the present invention;

[0039] Figure 2 This is a comparison diagram of simulation operations of different methods on five tasks in the embodiments of the present invention;

[0040] Figure 3 This is a diagram illustrating the deployment effect of real-world scenarios across five tasks in an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram of a multi-tasking dexterous hand operating device based on adaptive bidirectional distillation, according to an embodiment of the present invention. Detailed Implementation

[0042] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0043] It should be understood that the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0045] like Figure 1 As shown, in this embodiment, a multi-task dexterous hand manipulation method based on adaptive bidirectional distillation is provided. This method is implemented on a dexterous robot hand platform with multiple degrees of freedom and a multi-finger structure, and provides unified control over multiple types of operation tasks, including bottle cap rotation, faucet tightening, lever sliding, table reorientation, and hand reorientation. The implementation of this method includes the following steps:

[0046] S1: Construction of a multi-task expert set, specifically:

[0047] First, a multi-task dexterity operation expert set is constructed based on a pre-trained vision-tactile perception model. Specifically, an egocentric RGB camera, tactile sensors deployed on the dexterous fingertips and palm, and joint encoder readings are used to form the multimodal observation space of the robot system, which includes visual images, tactile readings, and proprioceptive states.

[0048] In the simulation environment, high-fidelity physical simulation scenarios are built for five types of tasks: bottle cap rotation, faucet tightening, lever sliding, table reorientation, and hand reorientation. Object instances with different geometric and material properties are initialized. Multiple single-task expert policies are trained on the basis of a shared vision-touch pre-trained encoder using a reinforcement learning algorithm (based on the PPO method), so that each task corresponds to four expert control policies that converge to a high success rate.

[0049] In the training process described above, a vision-tactile pre-trained encoder maps RGB images and tactile signals to low-dimensional latent representations. These representations are then concatenated with the robot's proprioceptive features and input into a task-specific policy network, outputting high-dimensional joint space motion vectors to drive the dexterous hand to perform operations such as grasping, rotating, or redirecting for the corresponding task. Once the success rate of a single-task expert on their respective task reaches a preset threshold, their parameters are frozen. A large number of state-action trajectories are then collected in a simulation environment using these expert policies, serving as an expert demonstration dataset for subsequent unified diffusion policy distillation. Each sample in the state-action trajectory includes the current vision-tactile observation, proprioceptive state, task identifier, and the corresponding expert's output motion vector.

[0050] S2: Construction of the unified diffusion strategy model, specifically:

[0051] Based on the expert demonstration data mentioned above, a unified diffusion strategy model is constructed. This model consists of a perception module, a task encoding module, and a decision module based on the diffusion model, which is used to generate multimodal action sequences given the current simulation state.

[0052] Specifically, such as Figure 1 As shown in the perception module in (a), the perception module adopts a multi-branch feature encoding structure: one branch encodes the egocentric RGB image using a convolutional neural network to output visual features; another branch encodes the tactile array or force sensor signal temporally or spatially to obtain tactile features; and a third branch encodes proprioceptive information such as joint angles and joint velocities using a multilayer perceptron to obtain proprioceptive features. Visual and tactile features are fused in the channel dimension to form a visual-tactile joint feature, which is then concatenated with the proprioceptive features in the feature dimension to obtain a unified representation of environmental state features.

[0053] The task encoding module maps discrete task IDs to low-dimensional task embedding vectors, which can be achieved through a lookup table or a trainable embedding layer. The resulting task embeddings are concatenated with the aforementioned environmental state features and used as conditional inputs to the diffusion policy model. These inputs participate in the modeling and generation of action distributions during both the training and inference phases, thereby achieving multi-task control based on task conditions.

[0054] In the decision-making module, the action sequence in the expert demonstration is treated as a continuous high-dimensional random variable, such as... Figure 1 As shown in (a), Gaussian noise is gradually injected into the action through a forward diffusion process, causing it to transition from the true action distribution to an isotropic Gaussian noise distribution within a preset time step. In the denoising learning stage, a conditional diffusion network is used with noisy actions, time steps, and environmental state features as inputs to predict noise or reconstruct actions, thereby learning the denoising mapping at different diffusion steps. By minimizing the noise prediction error or reconstruction error at each time step, the unified diffusion strategy can approximate the multimodal distribution of expert demonstration actions, and thus generate a high-quality action sequence from Gaussian noise through multiple denoising steps during inference.

[0055] The forward diffusion process of the unified diffusion strategy model in step S2 is as follows:

[0056] Extracting action sequences from expert demonstration data The action sequence is treated as a multidimensional continuous random variable, and the action sequence is sampled and diffused at time steps in each training iteration. and Gaussian noise Calculate noisy actions based on a preset noise scheduler. , making In the steps The noise injection results at the location. Specifically:

[0057]

[0058] in, This indicates the transition from step 1 to step 2. The cumulative product of all noise attenuation factors is defined as: . This represents the noise attenuation factor.

[0059] The denoising process of the unified diffusion strategy model in step S2 is as follows:

[0060] Sample initial actions from a standard Gaussian distribution. At each diffusion time step By using the current noisy action Time step The denoising network, based on the environmental state features obtained after the task encoding module is concatenated, predicts noise or reconstructs actions, and the actions of the previous time step are inferred from the noise scheduler. The final motion output is obtained through multiple iterations of noise reduction. It is used to control a dexterous robotic arm to perform the current task. Specifically:

[0061]

[0062] in It is Gaussian noise. This represents a denoising network. It is a characteristic of proprioception. It is a visual and tactile feature. It is a task identification feature. and Determined by the noise scheduler The functions are the scaling factor, the "noise gradient" scheduling factor, and the standard deviation of the Gaussian noise re-injected in each iteration. We perform the above steps. After several iterations, the denoised motion output is finally obtained. .

[0063] S3: Adaptive route weighted distillation, specifically:

[0064] To fully utilize the complementarity of multi-task experts during the distillation process, while avoiding task conflicts caused by simple averaging, this invention introduces an adaptive routing network, such as... Figure 1 As shown in the adaptive routing network module in (a), weighted distillation is performed on multi-task hybrid experts. The adaptive routing network takes the current environmental state features as input, passes through a multilayer perceptron and nonlinear activation, and outputs a weight vector with a dimension equal to the number of experts. After softmax normalization, the supervision weights of each expert are obtained.

[0065] When training the unified diffusion strategy, for each training sample, the output actions of each single-task expert in that state are first used as the supervision target. The matching loss between the action generated by the unified diffusion strategy and the corresponding expert action is calculated. Then, the weights given by the adaptive routing network are used to sum the losses of each expert, forming a weighted distillation loss function. By backpropagating this loss and updating the parameters, the unified diffusion strategy can adaptively favor experts who are more proficient in the current scenario under different states, thereby improving the efficiency of multi-task data fusion and mitigating the imbalance and negative transfer problems between tasks.

[0066] The adaptive routing network in step S3 includes:

[0067] Based on input environment status (Including visual and tactile features) Proprioceptive characteristics and task ID features Dynamically adjust the loss supervision weights for each hybrid expert. Specifically, the input state... The output is processed sequentially through a multilayer perceptron (MLP) (384 → 128), a ReLU activation function, a second MLP (128 → 4), and a softmax layer for normalization. The output dimension is equal to the probability distribution vector of the number of experts. ,in Each component of the probability distribution vector serves as the loss weight for the corresponding expert. The weighted summation of the action matching losses between the unified diffusion strategy and each expert's strategy forms a weighted distillation loss function.

[0068] S4: Online DAGger distillation, specifically:

[0069] After the initial offline distillation is completed, the unified diffusion strategy will be deployed separately in the simulation environment to perform multi-task operations and collect online interactive data to mitigate the observation distribution shift problem common in behavioral cloning. For example... Figure 1 As shown in (a), specifically, during each simulation rollout process, a unified strategy generates actions based on the current visual-tactile observations and task encoding, driving the dexterous hand to interact with the environment and generate a series of trajectory states.

[0070] At each access state of the aforementioned trajectory, the single-task expert policy under the corresponding task is invoked to obtain expert actions as supervision signals, and the "unified policy access state-expert action pair" is added to the distillation dataset. As online interaction progresses, new online data is continuously incorporated into the original offline expert demonstration data. The unified diffusion policy is retrained or incrementally updated based on the updated hybrid dataset, making it approximate expert behavior in its own induced real state distribution, effectively reducing the error accumulation caused by state distribution mismatch.

[0071] The online DAGger distillation process in step S4 includes:

[0072] In a simulation environment, the current unified diffusion policy is used for interaction. The states accessed under this policy are collected, and at each access state, a multi-task expert policy is invoked to generate a supervised action. The access state and expert action pairs are added to the distillation dataset. The unified diffusion policy is then trained using the updated distillation dataset, allowing it to approximate the expert action under its own induced state distribution. This is done within the weighted loss function for training multi-task expert distillation using Dagger. We use mean squared error (MSE) loss to minimize the L2 distance between actions generated by a diffusion-based unified policy and actions generated by multi-task hybrid experts. Specifically:

[0073]

[0074] in This represents the loss weights of different experts. This represents the sequence of actions in the expert demonstration. Indicates the discrete diffusion time step. The first term generated by closed reparameterization of forward diffusion The noise action of each diffusion step, This represents a denoising network, while Indicates the denoising network in the environmental state Actions generated under (including visual-tactile) conditions. Visual-tactile features. Robot proprioceptive characteristics and task ID features .

[0075] S5: Backdistillation from unified strategy to expert strategy, specifically:

[0076] Once the success rate of the unified diffusion strategy on all tasks exceeds the average level of the initial single-task expert strategies, the unified diffusion strategy is considered a new teacher, and back-distillation is performed on the expert strategies for each task to achieve knowledge transfer through "student-led teacher improvement." For example... Figure 1As shown in (b), specifically, under each task, the current unified policy parameters are fixed, and the unified policy is used to generate actions in the task scenario as the learning target of the expert policy. At the same time, a hybrid loss or reward signal is constructed by combining environmental task rewards.

[0077] In the backdistillation process, an action reward term is introduced to encourage expert policies to output actions that approximate those generated by the unified policy, while the original task reward term is retained to measure the degree of task completion. By maximizing the weighted sum or equivalent design of the action reward and task reward loss function, the parameters of the single-task expert policy network are updated, enabling it to maintain or improve the original task success rate while absorbing diverse skill patterns shared across tasks in the unified policy, thereby obtaining a performance-enhanced expert policy set.

[0078] The loss function of the reverse distillation process in step S5 includes an action reward term. and task rewards ,in:

[0079] The action reward is the negative mean square error between the output action of the uniform diffusion policy and the output action of the expert policy; specifically:

[0080]

[0081] in, Rewards are actions output by a unified strategy. Actions related to expert strategy output The negative mean square error (MSE) between them.

[0082] Task Rewards Environmental rewards for completing operational goals under corresponding tasks, such as: bottle cap rotation is the angle of bottle cap rotation, faucet tightening is the angle of faucet tightening, lever sliding is the sliding distance, table reorientation is the object rotation angle, and hand reorientation is the object rotation angle.

[0083] The total reward is an action reward. With task rewards The sum is used to balance approaching a unified strategy action with maintaining mission success rate.

[0084] S6: Bidirectional distillation iterative loop and convergence determination, specifically:

[0085] The online DAgger distillation stage and the backdistillation stage are combined into a bidirectional distillation iterative process of "expert strategy set → unified diffusion strategy → expert strategy set", and multiple rounds are performed until the multi-task success rate converges or reaches a preset threshold.

[0086] like Figure 1As shown in (c), specifically, in each iteration, the current expert policy set is first fixed, and online DAgger distillation is performed to update the unified diffusion policy, making it approximate the multi-task hybrid expert behavior in the latest state distribution; then, the updated unified policy is fixed, and back-distillation is performed to update the expert policy for each task. After each iteration, the success rate of the unified policy and the experts for each task is evaluated on the test sets of seen and unseen objects. When the average success rate improvement is lower than a preset incremental threshold (e.g., 1%) or the target performance is reached, the iteration stops, and the final multi-task unified policy and improved expert policy set are obtained.

[0087] S7: Platform Implementation and Application Scenarios Description, specifically:

[0088] The method of this invention is deployed on a dexterous robotic hand platform with multiple degrees of freedom and a multi-finger structure, such as a three- or five-finger anthropomorphic manipulator, mounted on a fixed base or at the end of a robotic arm, and equipped with an egocentric RGB camera and hand force / tactile sensors. In experiments, through the aforementioned multi-task training and bidirectional distillation framework, the method of this invention achieved a higher success rate than the baseline of a vision-tactile pre-trained single-task expert in five tasks: bottle cap rotation, faucet tightening, lever sliding, table reorientation, and hand reorientation. It also exhibited better generalization stability for unseen object instances, such as... Figure 2 and Figure 3 The simulation and real-world results are shown below.

[0089] Corresponding to the aforementioned embodiment of a multi-task dexterous hand operation method based on adaptive bidirectional distillation, the present invention also provides an embodiment of a multi-task dexterous hand operation device based on adaptive bidirectional distillation.

[0090] See Figure 4 The present invention provides a multi-task dexterous hand operation device based on adaptive bidirectional distillation, comprising a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement a multi-task dexterous hand operation method based on adaptive bidirectional distillation as described in the above embodiment.

[0091] The embodiment of the multi-task dexterous hand manipulation device based on adaptive bidirectional distillation provided by this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiment can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 4The diagram shown is a hardware structure diagram of any device with data processing capabilities, including the multi-task dexterous hand operating device based on adaptive bidirectional distillation provided by the present invention. (Except for...) Figure 4 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0092] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0093] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0094] This invention also provides a computer-readable storage medium storing a program that, when executed by a processor, implements a multi-tasking dexterous hand operation method based on adaptive bidirectional distillation as described in the above embodiments.

[0095] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device of any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of any data processing device. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0096] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned multi-task dexterous hand operation method based on adaptive bidirectional distillation.

[0097] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0098] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. This application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A multi-task dexterous hand operation method based on adaptive bidirectional distillation, characterized in that, Includes the following steps: The construction of a multi-task dexterity manipulation expert set includes: training several dexterity manipulation tasks separately based on a pre-trained vision-tactile perception model to obtain multiple single-task expert policies, and collecting state-action trajectories containing multimodal observation spaces as a multi-task expert demonstration dataset. Construct a unified diffusion strategy model, including: The environmental state and task identifiers of the multi-task expert demonstration dataset are feature-encoded and concatenated to form the conditional input of a unified diffusion strategy; Based on expert demonstration data and conditional input, forward diffusion and denoising learning are performed to model the multimodal distribution of the action space and obtain the action generation strategy given the current environmental state and task identifier. Bidirectional distillation iteration: The loss weight vector of each expert policy is obtained using an adaptive routing network. Online knowledge distillation is performed on the unified diffusion policy model based on the loss weight vector. When the unified diffusion policy model reaches a preset threshold, an action reward term is introduced to perform back distillation to obtain a performance-enhanced expert policy set. The online knowledge distillation and back distillation are iteratively repeated until the multi-task success rate converges or reaches a preset threshold, resulting in the final multi-task unified policy and improved expert policy set.

2. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The process of acquiring the multiple single-task expert strategies specifically includes: The robot system uses an egocentric RGB camera, tactile sensors deployed on the fingertips and palms, and joint encoder readings to form a multimodal observation space, which includes visual images, tactile readings, and proprioceptive states. The vision-tactile pre-trained encoder is used to map RGB images and tactile signals to low-dimensional latent representations, which are then concatenated with robot proprioceptive features and input into a task-specific policy network to output high-dimensional joint space motion vectors.

3. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The acquisition of state-action trajectories in the multimodal observation space specifically involves: when the success rate of a single-task expert on their respective task reaches a preset threshold, their parameters are frozen, and a large number of state-action trajectories are collected in the simulation environment using these expert strategies as an expert demonstration dataset for subsequent unified diffusion strategy distillation; each sample in the state-action trajectory includes the current visual-tactile observation, proprioceptive state, task identifier, and corresponding expert output action vector.

4. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The specific steps for feature encoding and splicing of the environmental state and task identifiers of the multi-task expert demonstration dataset are as follows: a multi-branch feature encoding structure is adopted, the first branch encodes the visual image using a convolutional neural network to output visual features, and the second branch encodes the tactile signal in time or space to obtain tactile features. The third branch encodes the proprioceptive information using a multilayer perceptron to obtain proprioceptive features; visual and tactile features are fused along the channel dimension to form a visual-tactile joint feature, which is then concatenated with the proprioceptive features along the feature dimension to obtain a unified environmental state feature representation; discrete task IDs are mapped to low-dimensional task embedding vectors, which are then concatenated with the environmental state feature representation as the conditional input to the diffusion strategy model.

5. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The forward diffusion of the unified diffusion strategy model is as follows: extract action sequences from expert demonstration data, treat the action sequences as multidimensional continuous random variables, sample the diffusion time steps and Gaussian noise in each training iteration, calculate the noisy action according to the preset noise scheduler and the cumulative product of all noise attenuation factors from the initial to the current time step, and inject noise into the result of this step.

6. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The denoising learning of the unified diffusion strategy model is as follows: initial actions are sampled from a standard Gaussian distribution. At each diffusion time step, noise or reconstructed actions are predicted by a denoising network based on the current noisy action, time step, and environmental state characteristics. The action of the previous time step is back-inferred based on the noise scheduler. After multiple iterations of denoising, the final action output is obtained, which is used to control the dexterous manipulator to perform the current task.

7. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The method of obtaining the loss weight vector of each expert policy using an adaptive routing network is as follows: the input environment state includes visual and tactile features, proprioceptive features and task ID features, and passes through a multilayer perceptron, a ReLU activation function, a second MLP and a softmax layer for normalization in sequence. The output is a probability distribution vector with a dimension equal to the number of experts. Each component of the probability distribution vector is used as the loss weight of the corresponding expert. The action matching loss between the unified diffusion policy and each expert policy is weighted and summed to form a weighted distillation loss function.

8. The multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The online knowledge distillation of the unified diffusion strategy model based on the loss weight vector specifically involves: Interacting with the current unified diffusion strategy in the simulation environment, collecting the states accessed under this strategy, and calling a multi-task expert strategy to generate supervised actions at each access state, adding the access state and expert action pairs to the distillation dataset, and training the unified diffusion strategy using the updated distillation dataset, so that the unified diffusion strategy approximates the expert actions under its own induced state distribution; in the weighted loss function of training multi-task expert distillation in Dagger, the mean squared error loss is used to minimize the L2 distance between the actions generated by the diffusion-based unified strategy and the actions generated by the multi-task hybrid expert.

9. A multi-task dexterous hand operation method based on adaptive bidirectional distillation according to claim 1, characterized in that, The backdistillation includes: when the success rate of the unified diffusion strategy on each task exceeds the average level of the initial single-task expert strategy, the unified diffusion strategy is regarded as a new teacher, and backdistillation is performed on the expert strategies for each task. During the backdistillation process, an action reward is introduced to encourage the expert strategies to output actions that are close to those generated by the unified strategy. The action reward is the negative mean square error between the action output by the unified diffusion strategy and the action output by the expert strategy. The task reward is the environmental reward for completing the operation objective under the corresponding task. The total reward is the sum of the action reward and the task reward.

10. A multi-tasking dexterous hand manipulation device based on adaptive bidirectional distillation, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that... When the processor executes the executable code, it implements a multi-task dexterous hand operation method based on adaptive bidirectional distillation as described in any one of claims 1-9.

Images