Dexterous hand repositioning method, apparatus, and storage medium

Abstract

The application belongs to the field of robot control, and particularly relates to a dexterous hand redirection method, equipment and a storage medium. The dexterous hand redirection method comprises the following steps: parameterizing joint angle trajectory of a target dexterous hand into a first parameter tensor to be optimized; constructing a differentiable loss function for constraining consistency of a source hand motion trajectory and a target dexterous hand motion trajectory; iteratively updating the first parameter tensor based on the motion trajectory parameter of the source hand using an optimizer to generate an optimized first parameter tensor; and redirecting the motion trajectory of the target dexterous hand based on the base pose trajectory of the source hand and the optimized first parameter tensor. The method of the application changes from traditional frame-by-frame optimization to global optimization of the entire motion trajectory, the generated motion trajectory is smoother and more coherent, and the system has high integration and good flexibility.

Description

Technical Field

[0001] This application belongs to the field of robot control, and in particular relates to a method, device and storage medium for repositioning a dexterous hand. Background Technology

[0002] With the development of humanoid dexterous hands, a key challenge has become how to efficiently and accurately transfer the natural and flexible motor skills of the human hand (source hand) to the dexterous hand (target hand). Existing retargeting techniques treat retargeting as a mathematical optimization problem, solving an objective function independently at each time step. Methods based on traditional optimization libraries such as dex-retargeting typically use solvers like nlopt for frame-by-frame independent optimization. This approach has several inherent drawbacks: (1) Poor temporal consistency: Frame-by-frame optimization ignores the temporal continuity of motion trajectory, which easily produces jitter and unsmooth motion results, requiring complex post-processing.

[0003] (2) Low system integration: Traditional optimizers are separated from the deep learning ecosystem, making it difficult to take advantage of GPU parallel computing and also unable to be integrated end-to-end with learning-based perception models.

[0004] (3) Poor flexibility: Adding new constraints or loss terms is usually complicated and difficult to use gradient information for efficient optimization.

[0005] In addition, most existing redirection techniques treat the bases of the source hand and the target hand as strictly fixed, ignoring the differences between them in size, configuration and workspace, which leads to optimization failure or unnatural posture when the differences are large.

[0006] Therefore, there is an urgent need in the field for a dexterous hand repositioning technique that can produce smooth, natural, physically reliable, and adaptive relative motion of the base. Summary of the Invention

[0007] The embodiments of this application aim to provide a dexterous hand redirection method, device and storage medium to solve the problems of poor timing consistency, low system integration and poor flexibility caused by frame-by-frame independent optimization in the prior art.

[0008] To address the aforementioned technical problems, this application provides the following technical solutions: In a first aspect, embodiments of this application provide a dexterous hand redirection method, the method comprising: The joint angle trajectory of the target dexterous hand is parameterized into the first parameter tensor to be optimized; Construct a differentiable loss function to constrain the consistency between the source hand's motion trajectory and the target dexterous hand's motion trajectory; Based on the motion trajectory parameters of the source hand, the optimizer iteratively updates the first parameter tensor to generate an optimized first parameter tensor; The motion trajectory of the target dexterous hand is redirected based on the base pose trajectory of the source hand and the optimized first parameter tensor.

[0009] Optionally, the motion trajectory parameters of the source hand include the base position posture trajectory of the source hand, the joint angle trajectory of the source hand, and the contact point posture trajectory of the source hand.

[0010] Optionally, the differentiable loss function includes gesture similarity loss, motion smoothness loss, touch point position loss, joint tendency loss, and joint limitation loss.

[0011] Optionally, the differentiable loss function may further include penetration loss.

[0012] Optionally, the gesture similarity constraint is achieved by calculating the similarity between the task vectors defined by the source hand and the dexterous hand on the corresponding finger chain, wherein the task vectors are composed of the direction vectors between key links on the finger chain.

[0013] Optionally, the step of iteratively updating the first parameter tensor using an optimizer based on the motion trajectory parameters of the source hand includes: Based on the motion trajectory parameters of the source hand, the gradient of the differentiable loss function with respect to the parameter tensor is calculated using an automatic differentiation framework. The first parameter tensor is iteratively updated using a gradient descent-based optimizer.

[0014] Optionally, the motion trajectory parameters based on the source hand are used to iteratively update the first parameter tensor using an optimizer to generate an optimized first parameter tensor, including: The motion trajectory parameters of the source hand are transformed into a local coordinate system with the base of the source hand as the origin; The first parameter tensor is iteratively updated using an optimizer in the local coordinate system to generate an optimized first parameter tensor.

[0015] Optionally, the method further includes: The relative pose trajectory of the target dexterous hand's base and the source hand's base is parameterized into a second parameter tensor to be optimized; An optimizer is used in the local coordinate system to perform a cooperative iterative update on the first parameter tensor and the second parameter tensor to generate an optimized first parameter tensor and an optimized second parameter tensor; The motion trajectory of the dexterous hand is redirected based on the base pose trajectory of the source hand, the optimized first parameter tensor, and the optimized second parameter tensor.

[0016] Secondly, embodiments of this application provide an apparatus, including: at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described above.

[0017] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor performs the steps of any of the methods described above.

[0018] The beneficial effects of this application's embodiments are as follows: Unlike existing technologies, this application provides a dexterous hand retargeting method that treats the parameters of an entire motion trajectory as an optimizable parameter tensor. By defining a differentiable loss function, a gradient descent optimizer is used to find the target dexterous hand trajectory parameters that best match the source hand's motion trajectory. This method can use an optimizer based on a deep learning framework to iteratively optimize the entire trajectory, supports GPU parallelism, and boasts high system integration and processing efficiency. It utilizes temporal context information to optimize the entire trajectory, making it smoother and more coherent. Furthermore, it facilitates the addition of new differentiable constraints, offering good flexibility. Attached Figure Description

[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0020] Figure 1 An exemplary flowchart of a dexterous hand redirection method is shown; Figure 2 An exemplary schematic diagram of the data glove structure is shown; Figure 3 An exemplary flowchart is shown for iteratively updating the first parameter tensor; Figure 4 An exemplary flowchart of another dexterous hand redirection method is shown; Figure 5 An exemplary schematic diagram of the hardware structure of a device is shown. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0023] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0024] This application provides a dexterous hand retargeting method based on differentiable optimization. This method transforms the retargeting problem from traditional frame-by-frame optimization to global optimization of the entire motion trajectory. By parameterizing the joint angle trajectory of the target dexterous hand (and optionally the base relative pose trajectory) into a parametric tensor and constructing a differentiable physical consistency loss function, the global optimal, smooth, and physically reliable motion trajectory is solved in one step using the automatic differentiation and optimizer (e.g., Adam) of modern deep learning frameworks (such as PyTorch).

[0025] Please refer to Figure 1 , Figure 1 A dexterous hand redirection method is shown, which mainly includes the following steps: Step S101: Parameterize the joint angle trajectory of the target dexterous hand into the first parameter tensor to be optimized.

[0026] A dexterous hand is a robotic end effector that possesses multiple fingers (three or more) and multiple joint degrees of freedom, capable of performing complex and precise maneuvers, and designed to mimic some or all of the functions of a human hand. In the embodiments of this application, the target dexterous hand includes a dexterous hand based on the URDF model and a dexterous hand based on the Mano model.

[0027] The dexterity hand retargeting method of this application redirects the motion trajectory of the target dexterity hand based on the motion trajectory of the source hand. In one embodiment, when retargeting the target dexterity hand based on an optimizer, the base poses of the source hand and the target dexterity hand are considered fixed. In this way, a complex, time-varying global motion trajectory retargeting problem can be simplified into the following two problems: (1) the problem of “finger shape imitation” in a unified, relatively static reference frame; (2) the problem of mapping the dexterity hand after “finger shape imitation” back to the real world coordinate system based on the base pose of the source hand.

[0028] To address the first problem, the joint angle trajectory of the target dexterous hand is parameterized as a first parameter tensor to be optimized. Specifically, this first parameter tensor is a PyTorch tensor of shape (T, N), where T is the total number of time steps (i.e., trajectory length), and N is the number of joints in the target dexterous hand. The initial value of this first parameter tensor can be set as the reference joint angle vector when the dexterous hand is in a neutral posture, or as the value mapped from the joint angle trajectory of the source hand using a simple mapping function. Here, a neutral posture of the dexterous hand refers to the state when the dexterous hand is not performing any task, is naturally relaxed, and is free from external forces.

[0029] Step S102: Construct a differentiable loss function to constrain the consistency between the source hand's motion trajectory and the target dexterous hand's motion trajectory.

[0030] In one embodiment, a differentiable loss function is constructed by fusing multiple constraint losses. It can be represented as:

[0031] Specifically, the differentiable loss function may include the following loss terms: 1) Gesture similarity loss ( ) The purpose of this loss is to ensure that the overall configuration of the target dexterous hand is consistent with that of the source hand. In one embodiment, the gesture similarity loss is achieved by calculating the similarity between the task vectors defined on the corresponding finger chains of the source and dexterous hands. These task vectors are composed of the direction vectors between key links on the finger chains. For example, a vector V1 pointing from the base of the finger joint to the middle joint, and a vector V2 pointing from the middle joint to the fingertip. This loss is achieved by calculating the cosine similarity or L2 distance between the corresponding task vectors of the source and target hands, maximizing their similarity. This method preserves gestures at an abstract geometric level and is robust to size and structural differences between the source and target hands.

[0032] 2) Loss of motion smoothness ( ) The purpose of this loss is to ensure the temporal continuity of the generated trajectory. Specifically, this loss is achieved by penalizing the first-order difference (velocity) and second-order difference (acceleration) of the first parameter tensor in the time dimension, for example:

[0033] in, This is the value of the first parameter tensor at time step t.

[0034] 3) Contact position loss ( ) The purpose of this loss is to ensure consistency between the target dexterous hand and the source hand when interacting with the object. When the motion trajectory parameters of the source hand indicate that the source hand makes contact with the object at time point t, this loss requires the contact point position of the target dexterous hand to be as close as possible to the object surface. Specifically, for source hand links that do not make contact with the object, the fingertip position of the finger corresponding to the target hand link is controlled to be consistent with the fingertip position of the finger corresponding to the source hand link; for source hand links that make contact with the object, the contact point position of the target hand link is controlled to be consistent with the source hand.

[0035] 4) Joint tendency loss ( ) The purpose of this loss is to ensure that the joint angles of the target dexterous hand are as close as possible to the reference joint angles. During optimization, this is achieved by penalizing the deviation between the first parameter tensor of the target dexterous hand and a preset reference joint angle vector. The preset reference joint angle vector is obtained based on the joint angles of the source hand and the link mapping relationship between the source hand and the target dexterous hand.

[0036] 5) Joint limitation loss ( ) The purpose of this loss is to ensure that the optimized joint angles do not exceed the physical limits of the target dexterous hand. Specifically, a maximum and minimum joint angle are pre-set for each joint of the target dexterous hand, and during the optimization process, the values ​​of the joint angles of the target dexterous hand are controlled to remain within their corresponding joint angle ranges.

[0037] 6) Penetration loss ( ) The purpose of this loss term is to reduce penetration between the hand and the object. Specifically, a dense set of sampling points is pre-defined on the surface of the key link of the glove to be checked for collision. These sampling points are mapped onto the target dexterous hand to obtain mapped sampling points. The directed distance field values ​​from these mapped sampling points to the object instance model (which can be constructed based on the object pose and object mesh model) are calculated. When the directed distance field value is positive, it indicates that the point is inside the object surface (i.e., penetration has occurred), and its value represents the depth of penetration; when the directed distance field value is zero, it indicates that the point is on the object surface; when the directed distance field value is negative, it indicates that the point is outside the object surface, and its absolute value represents the nearest distance from the point to the object surface. This loss term only cares about sampling points with positive directed distance values; the deeper the penetration, the greater the penalty. It should be noted that the penetration loss has little effect on some object models (e.g., thin-walled models such as disposable paper cups) and can be used as a selection loss term.

[0038] Step S103: Based on the motion trajectory parameters of the source hand, the optimizer iteratively updates the first parameter tensor to generate an optimized first parameter tensor.

[0039] In one embodiment, before step S103, the method further includes acquiring motion trajectory parameters of the source hand. The source hand is a human hand wearing a data glove, and the motion trajectory parameters include the base pose trajectory of the source hand, the joint angle trajectory of the source hand, and the touch point pose trajectory of the source hand. These parameters can be obtained by collecting and processing relevant information through a data acquisition system. This data acquisition system may include a data glove equipped with joint angle sensors and tactile sensors, and a vision system that records the process of a person grasping an object after wearing the data glove.

[0040] like Figure 2 The diagram shows a structural schematic of a data glove. The data glove includes several joints and several links. Each joint is equipped with a joint angle sensor to collect the joint's bending angle, and some or all of the links are equipped with tactile sensors to collect the tactile pressure felt at the contact points distributed along the links. When a person wearing the data glove grasps an object, the glove collects information on the joint angles, pressing positions, and normal forces of the original hand, while a vision system collects image data of the hand's movement trajectory.

[0041] Furthermore, the data acquisition system processes the acquired data to obtain the motion trajectory parameters of the source hand. Specifically, it first calculates the base pose of the source hand and the pose of the object (in the camera coordinate system) based on image data and pose vision algorithms; then, it calculates the linkage pose of the source hand using the base pose and joint angles of the source hand; finally, it calculates the contact point pose of the source hand based on the linkage pose, pressing position information, and normal force information. Thus, the motion trajectory parameters of the source hand in the camera coordinate system are obtained. Figure 2 As can be seen, several contact points can be distributed on the linkage, and the tactile sensor can capture the pressure information of each contact point. In one embodiment, when calculating the contact point pose of the source hand, regression calculation is also performed on the pressure of all contact points exceeding the pressure threshold to obtain a maximum preset number of contact points. Each contact point corresponds to a force center.

[0042] Generally, the wrist of the data glove is designated as the base link of the source hand, which is the root node of the kinematic tree. In one embodiment, to obtain an accurate and stable base link pose trajectory of the source hand, a rigid wristband is placed on the base of the source hand (e.g., the wrist of the data glove), and a QR code is placed on the rigid wristband. After acquiring image data containing the QR code through a vision system, the base link pose of the source hand is calculated based on the image data and a pose vision algorithm to locate the QR code.

[0043] Specifically, the source hand's base pose trajectory is a sequence formed by the source hand's base pose corresponding to T preset time steps (or frames), with a shape of (T, 4, 4). Here, T is the total number of time steps (i.e., trajectory length), and (4, 4) is the homogeneous transformation matrix recording the position and orientation (i.e., "pose") of the source hand's base in the world coordinate system. The source hand's joint angle trajectory is a sequence formed by the source hand's joint angles corresponding to T preset time steps, with a shape of (T, N). Here, T is the total number of time steps, and N is the number of joint angles. The source hand's contact point pose trajectory is a sequence formed by the contact point position information and normal force information of the source hand contacting the object corresponding to T preset time steps, with a shape of (T, M, 4), where M is the maximum number of contact points, and 4 represents the three-dimensional position and normal force of the contact point.

[0044] In one embodiment, after obtaining the motion trajectory parameters of the source hand, the method further includes: converting the motion trajectory parameters of the source hand to a local coordinate system with the base of the source hand as the origin, and using an optimizer in the local coordinate system to iteratively update the first parameter tensor to generate an optimized first parameter tensor.

[0045] Specifically, first, an automatic differentiation framework (such as PyTorch) is used to calculate the gradient of the differentiable loss function with respect to each element of the first parameter tensor; then, a gradient descent-based optimizer (such as the Adam optimizer) is used to iteratively update the first parameter tensor. For details on the specific implementation of iteratively updating the first parameter tensor, please refer to the section below. Figure 3 The description.

[0046] In some embodiments, when iteratively updating the first parameter tensor using the optimizer, the system may depend not only on the motion trajectory parameters of the source hand but also on the pose trajectory of the object. When the differentiable loss function includes a penetration loss, it is necessary to determine the object instance model based on the object's pose trajectory and the object mesh model, and then determine whether a collision will occur based on the object instance model and the trajectory parameters of the current target dexterous hand.

[0047] Step S104: Redirect the motion trajectory of the target dexterous hand based on the base pose trajectory of the source hand and the optimized first parameter tensor.

[0048] After optimization, the optimized first parameter tensor is obtained, which represents the joint angle trajectory of the target dexterous hand after retargeting. Based on the retargeted joint angle trajectory of the target dexterous hand and the base pose trajectory of the source hand, the target dexterous hand can be driven to reproduce the motion of the source hand. Specifically, the optimized joint angle trajectory is directly set to the target dexterous hand model, and the base of the target dexterous hand is aligned with the base of the source hand in the world coordinate system, thereby generating the final motion trajectory of the target dexterous hand.

[0049] Since the initial poses of the source hand and the dexterous hand may differ, and dexterous hands from different models may also have different initial poses, the redirection method of this application is also used to transform the dexterous hand into a preset initial pose using a rotation matrix. In this initial pose, the fingers of the dexterous hand and the data glove are approximately aligned. Therefore, step S104 further includes: determining the rotation matrix for transforming the dexterous hand into the preset initial pose; obtaining the base pose trajectory of the target dexterous hand based on the base pose trajectory of the source hand and the rotation matrix; and redirecting the motion trajectory of the target dexterous hand based on the base pose trajectory of the target dexterous hand and the optimized first parameter tensor.

[0050] Please refer to Figure 3 , Figure 3 A schematic diagram of the iterative update process for the first parameter tensor is shown, the process including: Step S1031: Perform forward propagation based on the motion trajectory parameters of the source hand and the current first parameter tensor to calculate the total loss.

[0051] Specifically, firstly, based on the base pose trajectory and joint angle trajectory of the source hand, the link pose trajectory of the source hand is calculated using forward kinematics. Then, the link pose trajectory and contact point pose trajectory of the source hand are transformed into a local coordinate system with the base of the source hand as the origin. Finally, the current first parameter tensor is... (i.e., the joint angle trajectory of the target dexterous hand) is used as input and substituted into the forward computation graph for calculation. During the calculation process, firstly using... The link poses of the target dexterous hand at all time steps are calculated using a positive kinematics model; then, based on the first parameter tensor... The link pose trajectory of the target dexterous hand, the link pose trajectory and contact point pose trajectory of the source hand in the local coordinate system, and the preset link mapping relationship and contact point mapping relationship between the source hand and the dexterous hand are used to calculate all loss terms. The loss terms are then weighted and summed according to preset weights to obtain the total loss.

[0052] Step S1032: Perform backpropagation based on the total loss to calculate the gradient tensor of each parameter to be optimized in the first parameter tensor.

[0053] Specifically, in frameworks such as PyTorch, the gradient tensor of the total loss relative to each parameter to be optimized can be calculated by calling a specific function.

[0054] Step S1033: Based on the gradient tensor of each parameter to be optimized in the first parameter tensor, update the first parameter tensor using the optimizer.

[0055] Specifically, the optimizer (such as Adam) obtains the gradient tensor of each parameter to be optimized in the first parameter tensor, calculates an actual parameter update amount according to the internal algorithm of the optimizer, and then updates the first parameter tensor based on the parameter update amount.

[0056] Step S1034: Determine whether the current state meets the stopping condition. If yes, proceed to step S1035; otherwise, the process jumps back to step S1031.

[0057] In one embodiment, the stopping condition is any one of the following: maximum number of iterations (e.g., 2000 times), loss convergence (the decrease in total loss over a number of consecutive iterations is less than a preset threshold), or gradient vanishing (the norm of the gradient tensor becomes extremely small).

[0058] Step S1035: Output the current first parameter tensor as the optimized first parameter tensor.

[0059] The source hand and the target dexterous hand generally have some degree of configurational difference. If the bases of the source hand and the target dexterous hand are strictly fixed, problems may arise such as loss not decreasing or loss becoming too large. To solve this problem, this application introduces a floating base mechanism to better handle the morphological differences between the source hand and the target hand. Please refer to... Figure 4 , Figure 4 A flowchart illustrating another dexterous hand redirection method is shown, the method comprising: Step S401: Parameterize the joint angle trajectory of the target dexterous hand into a first parameter tensor to be optimized, and parameterize the relative pose trajectory of the base of the target dexterous hand and the base of the source hand into a second parameter tensor to be optimized.

[0060] In one embodiment, a floating base is introduced in front of the base of the target dexterous hand, through which the base of the target dexterous hand can be adjusted in 6 degrees of freedom relative to the base of the source hand. Therefore, the shape of the second parameter tensor is (T, 6), where T is the total number of time steps and 6 represents 6 degrees of freedom.

[0061] Step S402: Construct a differentiable loss function to constrain the consistency between the source hand's motion trajectory and the target dexterous hand's motion trajectory.

[0062] and Figure 1 Compared to the previous embodiment, the differentiable loss function in this embodiment may further include a base adjustment loss. This base adjustment loss is used to constrain the second parameter tensor to ensure that the base deviation between the target dexterous hand and the source hand is not too large.

[0063] Step S403: Based on the motion trajectory parameters of the source hand, the optimizer is used to perform a collaborative iterative update of the first parameter tensor and the second parameter tensor to generate an optimized first parameter tensor and an optimized second parameter tensor.

[0064] Specifically, in the local coordinate system, an optimizer is used to iteratively update the first and second parameter tensors together. The optimizer learns how to bend the fingers and how to fine-tune the base position to best mimic the movement of the source hand by minimizing the same differentiable loss function.

[0065] Step S404: Redirect the motion trajectory of the dexterous hand based on the base pose trajectory of the source hand, the optimized first parameter tensor, and the optimized second parameter tensor.

[0066] After optimization, optimized first parameter tensors and optimized second parameter tensors are obtained. The second parameter tensor represents the relative pose trajectory between the base of the target dexterous hand and the base of the source hand. Based on the base pose trajectory of the source hand and the relative pose trajectory, the base pose trajectory of the target dexterous hand can be obtained. The joint angle trajectory of the target dexterous hand is given by the optimized first parameter tensor. Based on the base pose trajectory and joint angle trajectory of the target dexterous hand, the target dexterous hand can be driven to reproduce the movement of the source hand. In this embodiment, by using a floating base, the target dexterous hand can accurately reproduce the finger posture of the source hand while its base can be adaptively fine-tuned relative to the source hand, thereby overcoming the challenges posed by morphological differences and generating a more natural and feasible grasping trajectory.

[0067] According to an embodiment of this application, a device is provided, such as... Figure 5 The diagram shown is a hardware structure schematic of a device according to an embodiment of this application. The device 100 includes a processor 10, a memory 20, and a communication interface 30. The processor 10, memory 20, and communication interface 30 are connected by lines. Figure 5 In the embodiment shown, the processor 10, memory 20, and communication interface 30 are connected to each other via a bus.

[0068] The memory 20 is used to store software programs, computer-executable program instructions, etc. The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the device 100, etc.

[0069] The memory 20 can be a read-only memory (ROM), or other types of static storage devices that can store static information and instructions, or random access memory (RAM), or other types of dynamic storage devices that can store information and instructions, or electrically erasable programmable read-only memory (EEPROM). The specific type is not limited here.

[0070] For example, the aforementioned memory 20 can be Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM). This memory 20 can exist independently but is connected to the processor 10. Optionally, the memory 20 can also be integrated with the processor 10, for example, integrated within one or more chips.

[0071] In some embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and this remote memory may be connected to the device 100 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0072] The processor 10 connects various parts of the entire device 100 using various interfaces and lines. By running or executing software programs stored in the memory 20 and calling data stored in the memory 20, it performs various functions of the device 100 and processes data, such as implementing the methods described in any embodiment of this application.

[0073] The processor 10 can be a field programmable gate array (FPGA), a digital signal processor (DSP), a central processing unit (CPU), or the like.

[0074] Processor 10 can be a single-core processor or a multi-core processor. For example, processor 10 can be composed of multiple FPGAs or multiple DSPs. Furthermore, processor 10 can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Processor 10 can be a standalone semiconductor chip or integrated with other circuits into a single semiconductor chip. For example, it can form a system-on-a-chip (SoC) with other circuits (such as encoding / decoding circuits, hardware acceleration circuits, or various bus and interface circuits), or it can be integrated as a built-in processor within an application-specific integrated circuit (ASIC). This ASIC with integrated processor can be packaged separately or together with other circuits.

[0075] The communication interface 30 can use a transceiver device, such as a transceiver, to enable communication between the device 100 and other devices or communication networks.

[0076] This application also provides a computer storage medium storing instructions or programs that are executed by one or more processors, for example... Figure 5 One of the processors 10 may enable the one or more processors to perform the methods in any of the above method embodiments.

[0077] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general-purpose hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0078] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions within the technical scope disclosed in this application. Therefore, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A dexterous hand redirection method, characterized in that, The method includes: The joint angle trajectory of the target dexterous hand is parameterized into the first parameter tensor to be optimized; A differentiable loss function is constructed to constrain the consistency between the motion trajectory of the source hand and the motion trajectory of the target dexterous hand. The differentiable loss function includes gesture similarity loss, motion smoothness loss, touch point position loss, joint tendency loss, joint limit loss and penetration loss. The gesture similarity constraint is achieved by calculating the similarity between the task vectors defined on the corresponding finger chains of the source hand and the dexterous hand. The task vectors are composed of the direction vectors between key links on the finger chains. Based on the motion trajectory parameters of the source hand, the optimizer iteratively updates the first parameter tensor to generate an optimized first parameter tensor; The motion trajectory of the target dexterous hand is redirected based on the base pose trajectory of the source hand and the optimized first parameter tensor. The step of iteratively updating the first parameter tensor using an optimizer based on the motion trajectory parameters of the source hand includes: Based on the motion trajectory parameters of the source hand, the gradient of the differentiable loss function with respect to the first parameter tensor is calculated using an automatic differentiation framework. The first parameter tensor is iteratively updated using a gradient descent-based optimizer.

2. The method according to claim 1, characterized in that, The motion trajectory parameters of the source hand include the base position posture trajectory of the source hand, the joint angle trajectory of the source hand, and the contact point posture trajectory of the source hand.

3. The method according to claim 1 or 2, characterized in that, The motion trajectory parameters based on the source hand are iteratively updated using an optimizer on the first parameter tensor to generate an optimized first parameter tensor, including: The motion trajectory parameters of the source hand are transformed into a local coordinate system with the base of the source hand as the origin; The first parameter tensor is iteratively updated using an optimizer in the local coordinate system to generate an optimized first parameter tensor.

4. The method according to claim 3, characterized in that, The method further includes: The relative pose trajectory of the target dexterous hand's base and the source hand's base is parameterized into a second parameter tensor to be optimized; An optimizer is used in the local coordinate system to perform a cooperative iterative update on the first parameter tensor and the second parameter tensor to generate an optimized first parameter tensor and an optimized second parameter tensor; The motion trajectory of the dexterous hand is redirected based on the base pose trajectory of the source hand, the optimized first parameter tensor, and the optimized second parameter tensor.

5. A device, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 4.

6. A computer storage medium, characterized in that, The computer storage medium stores instructions or programs that, when executed by at least one processor, cause the at least one processor to perform the method as described in any one of claims 1 to 4.

Images