Robust estimation and reorientation method and system of hand key points based on spatiotemporal constraints

Abstract

This invention discloses a robust estimation and redirection method and system for hand keypoints based on spatiotemporal constraints, relating to the fields of virtual reality interaction, robot teleoperation, human-machine collaboration, robotic arm control, and dexterous hand redirection. First, a calibration matrix is ​​established, and the operator's wrist pose and hand keypoint coordinates are acquired in real time. A high-order continuous kinematics prediction is constructed based on the stable posterior state of the previous control cycle. A keypoint-level multimodal reliability index is formed. Unstable observations are projected into hand Cartesian keypoint states that satisfy spatiotemporal consistency. Stable Cartesian points are transformed to a local coordinate system, and the dexterous hand joint angles are solved through inverse kinematics optimization. This invention performs robust estimation of VR hand Cartesian keypoints under physical continuity constraints before inverse kinematics solution, suppressing non-physical jumps while preserving high-speed motion responsiveness, avoiding distortion of whole-hand control, and improving the continuity, stability, and naturalness of heterogeneous dexterous hand teleoperation.

Description

Technical Field

[0001] This invention belongs to the fields of virtual reality interaction, robot teleoperation, human-machine collaboration, robotic arm control and dexterous hand repositioning technology, specifically involving a robust estimation and repositioning method and system for hand key points based on spatiotemporal constraints. Background Technology

[0002] Dual-arm robotic arms and dexterous end-effector teleoperations are widely used in remote maintenance, hazardous environment operations, disaster relief, industrial assembly, medical assistance, and dexterous grasping scenarios. These systems typically require mapping the operator's wrist spatial pose to the target pose of the robotic arm's end effector, and mapping the operator's finger 3D keypoints or joint movements to the dexterous hand joint angles, to achieve real-time transmission from human hand movements to robot motion. Existing virtual reality hand tracking devices or visual hand recognition models can usually output the wrist's six-DOF pose and the 3D coordinates of several key points of the hand. However, in actual teleoperation, the operator's hand may exhibit non-stationary, highly dynamic movements such as rapid grasping, rapid opening, rapid obstacle avoidance, or rapid turning. In such cases, VR recognition or visual tracking systems are easily affected by sampling rate limitations, model detection delays, occlusion, feature blurring, and communication latency, leading to instantaneous jumps, drifts, loss, or skeletal topological distortions in local keypoints. It should be noted that the keypoints output by VR recognition may be discontinuous, but human hand movements in the real physical world are not arbitrary jumps. Human fingers are composed of bone segments and joints. The Cartesian trajectory of their distal ends is continuous over a short period of time. Velocity, acceleration, and jerk are limited by the movement capacity of muscles and joints, and the lengths of adjacent bone segments remain essentially constant. Therefore, abrupt changes in hand recognition cannot be directly equated with real high-speed movement, but require a combination of continuous kinematic priors and skeleton topological constraints for discrimination.

[0003] Traditional methods typically apply low-pass filtering, exponential smoothing, or Kalman filtering to the joint angles of a dexterous hand at the robot's joint output level. While these methods can suppress some jitter, they have significant shortcomings: First, fixed-parameter filtering introduces phase delay during high-speed movements, smoothing real rapid grasping or unfolding actions into sluggish movements and reducing the sense of immersion in teleoperation. Second, if abnormal Cartesian keypoints are entered into the inverse kinematics solver without physical consistency verification, local keypoint errors may be amplified by nonlinear mapping into significant jitter in the dexterous hand's joint space. Third, overly coarse granularity in freezing the entire hand or determining overall validity / invalidity can easily lead to the interruption of control of the entire robotic hand due to anomalies in a single finger or a single keypoint. Fourth, simply smoothing at the joint angle output level cannot correct non-physical jumps in VR-recognized coordinates at their source. Therefore, it is necessary to propose a new technical solution: before inverse kinematics retargeting, robust state estimation of the Cartesian key points of the VR hand is performed by utilizing the continuity of real human hand motion, the boundedness of velocity or acceleration, and the constraint of skeleton length, so as to obtain stable, continuous and physically reasonable Cartesian points of the hand, and then the dexterous hand joint angle is solved by positional or vector-based retargeting constraints. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a robust estimation and redirection method and system for hand key points based on spatiotemporal constraints. This method is applicable to acquiring operator hand movements using VR hand tracking, visual hand recognition, or wearable interactive devices, and mapping them to remote operation control scenarios of heterogeneous robotic arms and end-effector dexterous hands. It is used to solve the problems of discontinuous and unstable key point output in VR hand recognition under conditions of high-speed movement, occlusion, and time delay, as well as the problem of dexterous hand joint tremors caused by abnormal points entering inverse kinematics.

[0005] The technical solution of this invention is as follows: A robust estimation and retargeting method for hand keypoints based on spatiotemporal constraints includes the following steps: Obtain the calibration matrix from the virtual reality coordinate system to the robot base coordinate system and initialize the structural parameters of the robotic arm and end effector.

[0006] Acquire the operator's wrist pose and key coordinates of the operator's hand, output by the virtual reality device or visual hand tracking module.

[0007] Using the calibration matrix from the virtual reality coordinate system to the robot base coordinate system, the target pose of the robotic arm end effector is generated based on the operator's wrist pose.

[0008] Prior predictions are made based on the stable posterior states of each hand key point in the previous control cycle to obtain the prior prediction results of each hand key point.

[0009] Based on the coordinates of key hand points and prior prediction results, the multimodal reliability index of each key hand point in the end-effector dexterity hand is calculated.

[0010] Based on the multimodal reliability index of key points in the hand, the finger-level reliability of each finger in the dexterous hand is determined.

[0011] Based on the finger-level reliability of each finger in the dexterous hand, trajectory reconstruction is performed on each hand key point to obtain the stable hand Cartesian key points corresponding to each hand key point.

[0012] By optimizing and redirecting the Cartesian key points of the stable hand and performing inverse kinematic calculations, the target joint angles of the end-effector dexterity hand can be obtained.

[0013] Based on the target pose at the end of the robotic arm and the target joint angle of the end effector, the final control vector is obtained and sent to the robotic arm and its end effector.

[0014] Furthermore, let the virtual reality coordinate system be... The robot base coordinate system is The calibration matrix between the two is represented as follows: (1); in, This is the calibration matrix from the virtual reality coordinate system to the robot base coordinate system; For rotation matrix, It is a translation vector.

[0015] Simultaneously initialize the nominal length of each finger bone segment of the robotic arm's end effector. Topological parent nodes of each key hand point in the hand kinematic skeleton chain Control step size Speed ​​limit , acceleration upper limit Sliding window length and dexterous hand joint limit set The key points of the hand refer to the center points of each joint of the human hand and the fingertips as identified by virtual reality devices or visual hand tracking modules. The key point is numbered; the topological parent node of a key point is the key point adjacent to it along the finger bones towards the wrist.

[0016] The virtual reality tracking coordinate system The origin is set at the center of the physical activity space as indicated by the virtual reality device or visual hand tracking module. axis, shaft and The axes correspond to the horizontal rightward, vertical upward, and horizontal backward directions within the physical activity space, respectively.

[0017] The robot base coordinate system The origin is set at the geometric center of the mounting flange surface of the robotic arm base. The axis is perpendicular to the plane of the robot arm base and faces upward. The axis points directly in front of the robot arm in its default baseline posture. The axis is determined according to the right-hand rule.

[0018] Furthermore, for any hand, the operator's wrist position is represented as follows: (2); The set of key observation points of the operator's hand is represented as follows: (3); in, Number the control cycle. Indicates the first The operator's wrist pose in the virtual reality coordinate system during each control cycle; Indicates the first The rotation matrix of the operator's wrist relative to the virtual reality coordinate system during each control cycle; Indicates the first The translation vector of the operator's wrist in the virtual reality coordinate system during each control cycle; This represents the set of key observation points on the operator's hand. For the first The original Cartesian coordinates of each hand key point in the virtual reality coordinate system For the first Confidence level of detection of key hand features This refers to the number of key points on the hand.

[0019] Furthermore, the acquisition of the operator's wrist pose and key point coordinates of the operator's hand, output by the virtual reality device or visual hand tracking module, specifically involves: Short-term prediction of the operator's wrist pose is performed, and the predicted operator's wrist position is expressed as follows: (4); in, Indicates the first Predicted wrist position after compensation for delay in one control cycle; This represents the time scale coefficient for location prediction.

[0020] When wrist pose prediction is represented by rotation matrix increments: (5); in, Indicates the first Predicted wrist posture value after compensation for delay in one control cycle; For attitude prediction time scale coefficients; , They represent special orthogonal groups respectively. Its Lie algebra The exponential and logarithmic mappings between them This is a transpose.

[0021] Set the tool attitude compensation matrix as follows The end-point tool bias is The spatial scale coefficient is The target pose at the end of the robotic arm is then represented as: (6); in, Indicates the first The robot arm end-effector pose is generated in the robot base coordinate system in each control cycle.

[0022] Furthermore, the prior prediction based on the stable posterior state of the previous control cycle to obtain the prior prediction result is specifically as follows: For the Each key hand point is used to set the stable posterior state of the previous control cycle, including position. ,speed and acceleration Then the current prior prediction result is expressed as: (7); (8); in, Indicates the first The first control cycle, the... Prior predictions of the positions of key hand points; Indicates the first The first control cycle, the... Prior velocity predictions for each key hand point; , and These represent the output values ​​after robust estimation in the previous control cycle. The position, velocity, and acceleration of key points on the hand.

[0023] Furthermore, the first The multimodal reliability index of each key hand point is expressed as follows: (9); in, Indicates the first The first control cycle, the... Multimodal reliability indicators for key hand points; and They represent the first Skeletal length error and higher-order motion anomaly degree at key hand points; , and These are the bandwidth parameters for the corresponding Gaussian kernel function.

[0024] The skeleton length error can be expressed as: (10); in, Indicates the first Within the first control cycle, the first The three-dimensional observation position of the topological parent node of each key hand point; This represents the Euclidean distance between vectors.

[0025] For the The first finger, denoted as the set of key hand points it contains. And calculate finger-level reliability: (12); in, Indicates finger-level reliability. Indicates the first The weight coefficient of each key hand point in the reliability calculation of its respective finger level.

[0026] Furthermore, based on the finger-level reliability of each finger in the dexterous hand, trajectory reconstruction is performed on each hand key point to obtain the stable hand Cartesian key points corresponding to each hand key point, specifically including: When finger-level reliability When the threshold is below the finger level, the update of stable hand Cartesian keypoints corresponding to each hand keypoint on that finger is suppressed, and the set local anomaly handling method is executed.

[0027] The remaining fingers and each key point on the wrist are at a length of Local sliding window Internal execution trajectory reconstruction: For the Each key hand point is represented by its true continuous trajectory as follows: And construct the following optimization objective: (13); in, For a moment, The time number; Represents a robust kernel function; Indicates the first Key points of the hand The original Cartesian coordinates in the virtual reality coordinate system at any given moment; Indicates the first step that needs optimization. Key points of the hand in the continuous time domain The true continuous trajectory on; and Representing the true continuous trajectory The first and second derivatives with respect to time; and These are the weight penalty coefficients for the velocity smoothing regularization term and the acceleration smoothing regularization term, respectively. Indicates the first The topological parent node of each key hand point in the continuous time domain The true continuous trajectory on; This represents the weight penalty coefficient for the skeleton topology consistency constraint term. To optimize variables. The optimization objective simultaneously satisfies the following dynamic boundary conditions: (14); Under dynamic boundary constraints, the optimization objective is optimized. After optimization, the current time step is taken. Posterior value corresponding to the true continuous trajectory As a key point for stabilizing the hand, Descartes.

[0028] Furthermore, the optimization, redirection, and inverse kinematics calculation of the Cartesian key points of the stable hand to obtain the target joint angles of the dexterous hand specifically includes: After obtaining the Cartesian keypoints of the stable hand, transform the Cartesian keypoints of the stable hand to the local coordinate system. The relative Cartesian point is obtained from the middle. The local coordinate system Taking the geometric center of the palm of the dexterous hand as the origin, and the vector direction from the center of the wrist to the metacarpophalangeal joint of the middle finger as... The axis is defined by the direction of the normal vector perpendicular to the palm and pointing upwards. The axis is determined according to the right-hand rule. axis.

[0029] When using position-type redirection, several relative Cartesian points are selected as position references, and the target joint angles of the end effector are solved: (15); in, For the current target joint angle of the dexterous hand, This represents the joint angle configuration vector of the dexterous hand to be optimized; For end-effector dexterity The forward kinematic position relative to a Cartesian point The scale mapping factor from human hand to end-effector dexterity. and The weighting coefficients are used to constrain the time continuity of joint angles and the velocity amplitude limit. To represent the dexterity of the end effector The weight coefficient of a relative Cartesian point in positional relocation optimization; To indicate the first One control cycle, with the end effector's first... The relative Cartesian points correspond to the stable Cartesian key points of the hand.

[0030] When using vector-based redirection, it is based on a set of hand keypoint pairs. Construct a relative vector reference and solve for the target joint angles of the end effector: (16); in, For the set of key points of the hand A relative Cartesian point pair; To represent relative Cartesian point pairs The weight coefficients of the relative vectors formed in vector-based retargeting optimization. , To represent the first dexterity of the end effector The relative Cartesian point and the first The forward kinematic position relative to a Cartesian point , To represent respectively the first Each control cycle forms a relative Cartesian point pair. The The and the first The set of hand key point pairs includes several relative Cartesian point pairs. .

[0031] Furthermore, for the left and right hands, the target poses of the left and right robotic arm end caps are obtained respectively. , and the target joint angles of the left and right distal dexterous hands , The final control vector is expressed as: (17).

[0032] On the other hand, the present invention also provides a robust estimation and redirection system for hand keypoints based on spatiotemporal constraints, for implementing a robust estimation and redirection method for hand keypoints based on spatiotemporal constraints, including: The multimodal virtual reality perception module is used to acquire the operator's wrist pose and key coordinates of the operator's hand, output by the virtual reality device or the visual hand tracking module.

[0033] The coordinate system calibration module is used to obtain the calibration matrix from the virtual reality coordinate system to the robot base coordinate system; using the calibration matrix from the virtual reality coordinate system to the robot base coordinate system, the target pose of the robotic arm end effector is generated based on the operator's wrist pose.

[0034] The advanced spatiotemporal kinematics prediction module is used to make prior predictions based on the stable posterior states of each hand key point in the previous control cycle, and obtain the prior prediction results of each hand key point.

[0035] The adaptive reliability fusion module is used to calculate the multimodal reliability index of each hand key point in the end-effector dexterity hand based on the coordinates of the hand key points and the prior prediction results; and to determine the finger-level reliability of each finger in the end-effector dexterity hand based on the multimodal reliability index of the hand key points.

[0036] The local spatiotemporal sliding window reconstruction module is used to perform trajectory reconstruction on each hand key point based on the finger-level reliability of each finger in the dexterous hand, so as to obtain the stable hand Cartesian key points corresponding to each hand key point. The hierarchical inverse kinematics solution module is used to optimize and redirect the Cartesian key points of the stable hand and perform inverse kinematics calculations to obtain the target joint angles of the end-effector dexterity hand.

[0037] The end effector drive sending module is used to obtain the final control vector based on the target pose of the robotic arm end and the target joint angle of the end dexterous hand, and send it to the robotic arm and its end dexterous hand.

[0038] Thirdly, this application proposes an electronic device comprising: one or more processors, and a memory for storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the spatiotemporally constrained robust estimation and retargeting method for hand keypoints.

[0039] Fourthly, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the spatiotemporally constrained robust estimation and redirection method for hand keypoints.

[0040] Fifthly, this application proposes a computer program product, including a computer program or instructions that, when executed by a processor, implement the aforementioned spatiotemporal constraint-based robust estimation and redirection method for hand key points.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. It can distinguish between real high-speed hand movements and visual recognition noise. When keypoints move at high speed but conform to the predicted state and skeleton constraints, the system does not misclassify them as noise, thus preserving high-speed motion responsiveness.

[0042] 2. Correct non-physical jumps of VR key points before solving inverse kinematics to avoid local Cartesian anomalies being amplified into spatial jitter of dexterous hand joints.

[0043] 3. By using a local sliding window and a robust kernel function, occlusion, point loss, and transient drift are suppressed, ensuring that the output keypoint trajectory satisfies both continuity and dynamic feasibility.

[0044] 4. Employ keypoint-level, finger-level, or local sub-graph-level anomaly handling to prevent the entire hand from freezing due to anomalies at a single keypoint, thereby improving the continuity of complex operations.

[0045] 5. Compatible with positional and vector-based hand repositioning frameworks, it can be used for dexterous hand control based on fingertip position, as well as grasping and pinching control based on thumb-fingertip relative vectors.

[0046] 6. The robotic arm control branch and the dexterous hand control branch are clearly separated: the wrist position is responsible for the large-scale movement of the robotic arm end effector, while the stable finger Cartesian point is responsible for the fine movements of the dexterous hand. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the robust estimation and retargeting method for hand key points based on spatiotemporal constraints in this embodiment of the invention.

[0048] Figure 2 This is a flowchart illustrating the mapping of wrist pose to target pose at the end of the robotic arm in an embodiment of the present invention.

[0049] Figure 3 This is a flowchart illustrating the stable recognition process for the continuity constraint of Cartesian key points in the VR hand, as described in an embodiment of the present invention.

[0050] Figure 4 This is a flowchart illustrating the inverse solution of the stable Cartesian key point to the joint angle of the dexterous hand in an embodiment of the present invention. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the following specific embodiments are used to illustrate the technical concept of the present invention and are not intended to limit the scope of protection of the present invention. Without departing from the core concept of the present invention, those skilled in the art can make equivalent substitutions or combinations for the number of key points, the length of the sliding window, the optimizer type, the dexterity hand model, and the control interface.

[0052] Example 1: The key idea of ​​this invention is that instead of directly treating the VR recognition output as the real hand state, it is treated as an observation with noise and outliers; the real hand state should satisfy spatiotemporal continuity, local smoothness, skeleton topological consistency, and dynamic boundary constraints. Therefore, this invention adds a physically consistent Cartesian key point state reconstruction module between the "detection output" and the "dexterous hand inverse kinematics solution", using stable hand Cartesian points as inverse kinematics input.

[0053] Robust estimation and redirection methods for hand keypoints based on spatiotemporal constraints, such as Figure 1 As shown, it includes the following steps: Step 1: Obtain the calibration matrix from the virtual reality coordinate system to the robot base coordinate system and initialize the structural parameters of the robotic arm and the end effector.

[0054] Let the virtual reality coordinate system be The robot base coordinate system is The calibration matrix between the two is represented as follows: (1); in, This is the calibration matrix from the virtual reality coordinate system to the robot base coordinate system; For rotation matrix, As a translation vector, this calibration matrix can be obtained through hand-eye calibration, spatial point correspondence calibration, or preset coordinate system installation relationship.

[0055] Simultaneously initialize the nominal length of each finger bone segment of the robotic arm's end effector. The topological parent nodes of each key hand point (i.e., wrist, joints, and fingertips) in the hand kinematic skeletal chain. Control step size Speed ​​limit , acceleration upper limit Sliding window length and dexterous hand joint limit set The key points of the hand refer to the center points of each joint of the human hand (such as the metacarpophalangeal joints and proximal interphalangeal joints) and the fingertips, as identified by virtual reality devices or visual hand tracking modules. There are usually 21 or 26 key points in a hand. The key points of the hand are numbered; because the hand is modeled as a tree with the wrist as the root node, the topological parent node of a key point of the hand is the key point of the hand adjacent to it along the finger bones towards the wrist.

[0056] The virtual reality tracking coordinate system The origin is set at the center of the physical activity space (or the center of the main positioning base station) as indicated by the virtual reality device or the visual hand tracking module. axis, shaft and The axes correspond to the horizontal rightward, vertical upward, and horizontal backward directions in the physical activity space, respectively, and together they form a three-dimensional rectangular coordinate system that satisfies the right-hand rule.

[0057] The robot base coordinate system The origin is set at the geometric center of the mounting flange surface of the robotic arm base. The axis is perpendicular to the plane of the robot arm base and faces upward. The axis points directly in front of the robot arm in its default baseline posture. The axes are determined according to the right-hand rule, and together they form a three-dimensional rectangular coordinate system.

[0058] Step 2: Obtain the operator's wrist pose and key coordinates of the operator's hand output by the virtual reality device or visual hand tracking module; For any hand, the operator's wrist position is represented as follows: (2); The set of key observation points of the operator's hand is represented as follows: (3); in, Number the control cycle. Indicates the first The operator's wrist pose in the virtual reality coordinate system during each control cycle; Indicates the first During each control cycle, the rotation matrix of the operator's wrist relative to the virtual reality coordinate system is used to describe the spatial posture of the wrist. Indicates the first During each control cycle, the translation vector of the operator's wrist in the virtual reality coordinate system (i.e., the three-dimensional spatial coordinates of the wrist). This represents the set of key observation points on the operator's hand. For the first The original Cartesian coordinates of each hand key point in the virtual reality coordinate system For the first The detection confidence level for each key hand point can be determined. If the device does not provide a detection confidence level, segmented detection confidence levels can be constructed based on whether the key hand points are outside the field of view, whether they are invalid values, whether they are lost briefly, and whether they meet the anatomical range. This refers to the number of key points on the hand.

[0059] Step 3: Using the calibration matrix from the virtual reality coordinate system to the robot base coordinate system, generate the target pose of the robotic arm end effector based on the operator's wrist pose.

[0060] like Figure 2As shown, the specific processing procedure of the robotic arm branch is as follows: To reduce the lag caused by communication latency and recognition delay, a short-term prediction of the operator's wrist pose (operator's wrist position and posture) can be performed. The operator's wrist position prediction can be expressed as: (4); in, Indicates the first Predicted wrist position after compensation for delay in one control cycle; This represents the time scale coefficient for location prediction, used to adjust the look-ahead magnitude of predictions based on the first-order location difference (i.e., instantaneous velocity).

[0061] Wrist pose prediction can be achieved using rotation matrices, quaternions, or Lie algebra increments. When expressed as rotation matrix increments, it can be written as: (5); in, Indicates the first Wrist posture prediction (rotation matrix form) after compensation for delay in one control cycle. For attitude prediction time scale coefficients; , They represent special orthogonal groups respectively. Its Lie algebra The exponential and logarithmic mappings between these are used here to calculate and scale-extrapolate the relative attitude increments between adjacent periods in the manifold space. This is a transpose.

[0062] Set the tool attitude compensation matrix as follows The end-point tool bias is The spatial scale coefficient is The target pose at the end of the robotic arm can then be represented as: (6); in, Indicates the first The robot arm end-effector pose in the robot base coordinate system generated in each control cycle; The left and right hands obtain the target poses of the left and right robotic arms end-effectors respectively in the above manner. This branch is responsible for the large-scale displacement and attitude following of the robotic arm, and does not directly participate in the solution of the dexterous finger joint angle.

[0063] Step 4: In the dexterous hand branch of this invention, the first step is not directly applied. Instead of using the inverse kinematics of the hand key points acquired in each control cycle, the system performs prior predictions based on the stable posterior states of each hand key point in the previous control cycle to obtain the prior prediction results for each hand key point.

[0064] like Figure 3As shown, for the first Each key hand point is used to set the stable posterior state of the previous control cycle, including position. ,speed and acceleration Then the current prior prediction result is expressed as: (7); (8); in, Indicates the first The first control cycle, the... Prior predictions of the (three-dimensional) positions of key hand points; Indicates the first The first control cycle, the... (3D space) velocity prior prediction values ​​of key hand points; , and These respectively represent the control period in the previous control cycle (i.e., the 1st control period). After robust estimation, the output of the first control cycle is... The position, velocity, and acceleration of key points on the hand.

[0065] This prediction is not intended to replace the current observation, but rather to provide a physical reference for determining whether the current observation conforms to the continuous movement of a real human hand. If the current observation changes significantly relative to the previous frame but is consistent with the predicted state, then the change is more likely to be a real high-speed movement; if the current observation deviates from the predicted state and disrupts the skeletal structure, then it is more likely to identify jumps or occlusion noise.

[0066] Step 5: Calculate the multimodal reliability index of each hand key point based on the hand key point coordinates and prior prediction results.

[0067] This multimodal reliability index includes at least a detection confidence term, a prediction residual term, a skeleton length consistency term, and a higher-order motion anomaly term. The multimodal reliability index of each key hand point is expressed as follows: (9); in, Indicates the first The first control cycle, the... Multimodal reliability indicators for key hand points; and They represent the first Skeletal length error and higher-order motion anomaly degree at key hand points; , and These are the bandwidth parameters (or adjustment coefficients) of the corresponding Gaussian kernel function, used to control the sensitivity of each residual to the overall multimodal reliability index. The prediction residual term reflects whether the observation deviates from the continuous kinematic prior; the skeleton length consistency term reflects whether the distance between the current hand key point and its topological parent node deviates from the nominal bone length; the higher-order motion anomaly term reflects whether the acceleration or jerk derived from the current observation exceeds the boundary of human hand movement capability.

[0068] The skeleton length error can be expressed as: (10); in, Indicates the first Within the first control cycle, the first The three-dimensional observation position of the topological parent node (i.e. the adjacent joint closer to the wrist) of each key hand point; The Euclidean distance (L2 norm) of the vector is used to calculate the actual spatial distance between two adjacent hand keypoints under the current observation.

[0069] Higher-order motion anomalies can be constructed based on instantaneous velocity, acceleration, or jerk, for example: (11); Indicates the first The control cycle, derived from current observations, is the [number]th [cycle]. The instantaneous (or observed) acceleration vector of each key point on the hand.

[0070] By employing the aforementioned multimodal reliability metrics, this invention avoids treating all high-frequency variations as noise in traditional filtering methods. As long as high-speed variations satisfy the prediction prior and skeleton topology, they are still given a high weight, thereby maintaining the responsiveness of rapid hand movements.

[0071] Step 6: Determine the finger-level reliability of each finger in the dexterous hand based on the multimodal reliability index of key hand points.

[0072] For the A single finger can be denoted as the set of key hand points it contains. And calculate finger-level reliability: (12); in, Indicates finger-level reliability. Indicates the first The weight coefficient of each key hand point in the reliability calculation of its respective finger level.

[0073] Step 7: Based on the finger-level reliability of each finger in the dexterous hand, perform trajectory reconstruction on each hand key point to obtain the stable hand Cartesian key points corresponding to each hand key point.

[0074] In this invention, when finger-level reliability When the threshold is below the finger level, the entire hand is not frozen directly. Instead, the update of stable hand Cartesian keypoints corresponding to each keypoint on that finger is suppressed, and the set local anomaly handling method is executed.

[0075] The local anomaly handling methods described in this embodiment include maintaining the stable Cartesian keypoint of the hand in the previous control cycle, predicting and deducing based on the kinematic model, interpolating adjacent effective states, reducing the weight of the hand keypoint in the redirection optimization, or limiting its update amount.

[0076] The remaining fingers and each key point on the wrist are at a length of Local sliding window Internal execution trajectory reconstruction mechanism avoids the problem of discontinuous teleoperation caused by traditional whole-hand freeze strategy.

[0077] The trajectory reconstruction method is specifically as follows: For the Each key hand point is represented by its true continuous trajectory as follows: And construct the following optimization objective: (13); in, For a moment, The time number; This refers to robust kernel functions (such as the Huber kernel or the Geman-McClure kernel) used to reduce the impact of outlier observations that deviate significantly from the kinematics on optimization. Indicates the first Key points of the hand The original Cartesian coordinates in the virtual reality coordinate system at any given moment; Indicates the first step that needs optimization. Key points of the hand in the continuous time domain The true continuous trajectory on; and Representing the true continuous trajectory The first derivative with respect to time (velocity) and the second derivative with respect to time (acceleration); and These are the weight penalty coefficients for the velocity smoothing regularization term and the acceleration smoothing regularization term, respectively. Indicates the first The topological parent node of each key hand point in the continuous time domain The true continuous trajectory on; This represents the weight penalty coefficient for the skeleton topology consistency constraint term. To optimize the variables, the first term is an observation fitting term with reliability weights and a robust kernel function, used to reduce the impact of outlier observations on the estimation results; the second term is velocity and acceleration smoothing regularization, used to ensure that the output trajectory is continuously differentiable in time; the third term is skeleton topology constraint, used to maintain the consistency of the length of adjacent bone segments in the human hand structure.

[0078] The optimization objective simultaneously satisfies the following dynamic boundary conditions: (14); Under dynamic boundary constraints, the optimization objective is optimized. After optimization, the current time step is taken. Posterior value corresponding to the true continuous trajectory As a key point for stabilizing the hand, Descartes.

[0079] The actual continuous trajectory in this embodiment Parameterization can be performed using cubic splines, quintic splines, polynomial basis functions, B-splines, or small neural implicit representations. Preferably, the real-space trajectory must at least satisfy C2 continuity to ensure a smooth transition of the hand key point positions, velocities, and accelerations between control cycles.

[0080] Step 8: Optimize and redirect the Cartesian key points of the stable hand and perform inverse kinematic calculations to obtain the target joint angles of the end-effector dexterity hand.

[0081] like Figure 4 As shown, after obtaining the stable hand Cartesian keypoints, the coordinates of the stable hand Cartesian keypoints are transformed to the local coordinate system by subtracting the origin pose or multiplying by the inverse local transformation matrix. The relative Cartesian point is obtained from the middle. This eliminates the coupling effect of large-scale displacement and posture changes of the operator's arm on the fine grasping motion of the fingers.

[0082] The local coordinate system Taking the geometric center of the palm of the dexterous hand as the origin, and the vector direction from the center of the wrist to the metacarpophalangeal joint of the middle finger as... The axis is defined by the direction of the normal vector perpendicular to the palm and pointing upwards. The axis is determined according to the right-hand rule. axis.

[0083] When using position-type redirection, several relative Cartesian points are selected as position references, and the target joint angles of the end effector are solved: (15); in, For the current target joint angle of the dexterous hand, This represents the joint angle configuration vector of the dexterous hand to be optimized; For end-effector dexterity The forward kinematic position relative to a Cartesian point The scale mapping factor from human hand to end-effector dexterity. and The weighting coefficients are used to constrain the time continuity of joint angles and the velocity amplitude limit. To represent the dexterity of the end effector The weight coefficient of a relative Cartesian point in positional relocation optimization; To indicate the first One control cycle, with the end effector's first... The relative Cartesian points correspond to the stable Cartesian key points of the hand.

[0084] When using vector-based redirection, it is based on a set of hand keypoint pairs. Construct a relative vector reference and solve for the target joint angles of the end effector: (16); in, For the set of key points of the hand A relative Cartesian point pair; To represent relative Cartesian point pairs The weight coefficients of the relative vectors formed in vector-based retargeting optimization. , To represent the first dexterity of the end effector The relative Cartesian point and the first The forward kinematic position relative to a Cartesian point , To represent respectively the first Each control cycle forms a relative Cartesian point pair. The The and the first The set of hand key point pairs includes several relative Cartesian point pairs. Based on the set of hand key points The relative vectors are constructed, including vectors from the palm to each fingertip, vectors connecting adjacent knuckles, and relative vectors from the thumb to the fingertips of the index, middle, ring, and little fingers. This vector-based retargeting is particularly suitable for tasks involving grasping, pinching, and maintaining relative finger relationships.

[0085] Step 9: Based on the target pose of the robotic arm end effector and the target joint angle of the end effector dexterous hand, obtain the final control vector and send it to the robotic arm and its end effector dexterous hand.

[0086] In this embodiment, the target poses of the left and right robotic arm ends are obtained for the left and right hands, respectively. , and the target joint angles of the left and right distal dexterous hands , The final control vector can be expressed as: (17); The above process is repeated for each fixed control cycle in the subsequent process.

[0087] In this embodiment, the operator wears a VR device to perform a rapid opening and grasping motion. Due to the high speed of finger movement, the key points of the index finger and thumb tips output by the visual tracking module show instantaneous jumps within several frames. Directly inputting the original hand key points into the dexterous hand inverse kinematics would cause fingertip tremors during closure. This invention first performs continuous kinematic prediction of the current fingertip position based on the stable fingertip position, velocity, and acceleration of the previous control cycle. If the current observation point moves significantly relative to the previous frame, but the residual with the predicted position is small, and the lengths of the bone segments of the thumb and index finger do not change significantly, then the observation is judged as a real high-speed movement and assigned a high reliability weight. Subsequently, trajectory reconstruction is performed within a local sliding window, outputting continuous Cartesian points of the thumb and index finger. The dexterous hand inverse kinematics solves the joint angles based on the stabilized fingertip points, achieving natural following of the rapid grasping motion and avoiding the significant lag caused by traditional low-pass filtering.

[0088] In summary, this invention addresses the keypoint instability problem in VR hand recognition under high-speed motion and occlusion scenarios, proposing a robust estimation method for Cartesian keypoints utilizing the continuity of real human hand motion and skeleton topological constraints. This method outputs stable, continuous, and physically reasonable Cartesian points of the hand before solving the inverse kinematics of the dexterous hand, suppressing the impact of recognition jumps on robot control at the source. Furthermore, it achieves dexterous hand joint angle calculation through positional or vector-based redirection optimization. This scheme balances high-speed response, low jitter, local fault tolerance, and heterogeneous redirection adaptation, demonstrating high engineering practical value.

[0089] Example 2: A robust estimation and redirection system for hand keypoints based on spatiotemporal constraints is provided to implement a robust estimation and redirection method for hand keypoints based on spatiotemporal constraints, including: The multimodal virtual reality perception module is used to acquire the operator's wrist pose and key coordinates of the operator's hand, output by the virtual reality device or the visual hand tracking module.

[0090] The coordinate system calibration module is used to obtain the calibration matrix from the virtual reality coordinate system to the robot base coordinate system; using the calibration matrix from the virtual reality coordinate system to the robot base coordinate system, the target pose of the robotic arm end effector is generated based on the operator's wrist pose.

[0091] The advanced spatiotemporal kinematics prediction module is used to make prior predictions based on the stable posterior states of each hand key point in the previous control cycle, and obtain the prior prediction results of each hand key point.

[0092] The adaptive reliability fusion module is used to calculate the multimodal reliability index of each hand key point in the end-effector dexterity hand based on the coordinates of the hand key points and the prior prediction results; and to determine the finger-level reliability of each finger in the end-effector dexterity hand based on the multimodal reliability index of the hand key points.

[0093] The local spatiotemporal sliding window reconstruction module is used to perform trajectory reconstruction on each hand key point based on the finger-level reliability of each finger in the dexterous hand, so as to obtain the stable hand Cartesian key points corresponding to each hand key point. The hierarchical inverse kinematics solution module is used to optimize and redirect the Cartesian key points of the stable hand and perform inverse kinematics calculations to obtain the target joint angles of the end-effector dexterity hand.

[0094] The end effector drive sending module is used to obtain the final control vector based on the target pose of the robotic arm end and the target joint angle of the end dexterous hand, and send it to the robotic arm and its end dexterous hand.

[0095] Example 3: This embodiment proposes an electronic device, including: one or more processors, and a memory for storing instructions, which, when executed by the one or more processors, cause the one or more processors to execute the robust estimation and retargeting method for hand key points based on spatiotemporal constraints.

[0096] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the robust estimation and redirection method for hand keypoints based on spatiotemporal constraints as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and communication components.

[0097] The processor is used to execute all or part of the steps in the spatiotemporally constrained robust estimation and redirection method for hand keypoints as described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0098] The processor can be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic components, and is used to execute the robust estimation and retargeting method for hand key points based on spatiotemporal constraints described in the above embodiments.

[0099] Example 4: This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0100] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the robust estimation and redirection method for hand key points based on spatiotemporal constraints described in the various embodiments of this application.

[0101] The aforementioned storage media include: flash memory, hard disks, multimedia cards, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory), random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disks, optical discs, servers, APP (Application) app stores, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the aforementioned spatiotemporal constraint-based robust estimation and redirection method for hand key points.

[0102] Example 5: This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned robust estimation and relocation method for hand key points based on spatiotemporal constraints.

[0103] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0104] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0105] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A robust estimation and retargeting method for hand keypoints based on spatiotemporal constraints, characterized in that, Includes the following steps: Obtain the calibration matrix from the virtual reality coordinate system to the robot base coordinate system and initialize the structural parameters of the robotic arm and the end effector. Acquire the operator's wrist pose and coordinates of key points of the operator's hand, output by the virtual reality device or visual hand tracking module; Using the calibration matrix from the virtual reality coordinate system to the robot base coordinate system, the target pose of the robotic arm end effector is generated based on the operator's wrist pose. Prior predictions are made based on the stable posterior states of each hand key point in the previous control cycle to obtain the prior prediction results of each hand key point. Based on the coordinates of key hand points and prior prediction results, the multimodal reliability index of each key hand point in the end-effector dexterity hand is calculated. Based on the multimodal reliability index of key points in the hand, the finger-level reliability of each finger in the dexterous end effector is determined; Based on the finger-level reliability of each finger in the dexterous hand, trajectory reconstruction is performed on each hand key point to obtain the stable hand Cartesian key points corresponding to each hand key point. By optimizing and redirecting the Cartesian key points of the stable hand and performing inverse kinematics calculations, the target joint angles of the end-effector dexterity hand can be obtained. Based on the target pose at the end of the robotic arm and the target joint angle of the end effector, the final control vector is obtained and sent to the robotic arm and its end effector.

2. The robust estimation and redirection method for hand key points based on spatiotemporal constraints according to claim 1, characterized in that, Let the virtual reality coordinate system be The robot base coordinate system is The calibration matrix between the two is represented as follows: （1）； in, This is the calibration matrix from the virtual reality coordinate system to the robot base coordinate system; For rotation matrix, It is a translation vector; Simultaneously initialize the nominal length of each finger bone segment of the robotic arm's end effector. Topological parent nodes of each key hand point in the hand kinematic skeleton chain Control step size Speed ​​limit , acceleration upper limit Sliding window length and dexterous hand joint limit set The key points of the hand refer to the center points of each joint of the human hand and the fingertips as identified by virtual reality devices or visual hand tracking modules. The key point is numbered; the topological parent node of a key point is the key point of the hand that is adjacent to it along the finger bones towards the wrist. The virtual reality tracking coordinate system The origin is set at the center of the physical activity space as indicated by the virtual reality device or visual hand tracking module. axis, shaft and The axes correspond to the horizontal rightward, vertical upward, and horizontal backward directions within the physical activity space, respectively. The robot base coordinate system The origin is set at the geometric center of the mounting flange surface of the robotic arm base. The axis is perpendicular to the plane of the robot arm base and faces upward. The axis points directly in front of the robot arm's default baseline posture. The axis is determined according to the right-hand rule.

3. The robust estimation and redirection method for hand key points based on spatiotemporal constraints according to claim 1, characterized in that, For any hand, the operator's wrist position is represented as follows: （2）； The set of key observation points of the operator's hand is represented as follows: （3）； in, Number the control cycle. Indicates the first Operator's wrist pose in virtual reality coordinate system during each control cycle; Indicates the first The rotation matrix of the operator's wrist relative to the virtual reality coordinate system during each control cycle; Indicates the first The translation vector of the operator's wrist in the virtual reality coordinate system during each control cycle; This represents the set of key observation points on the operator's hand. For the first The original Cartesian coordinates of each hand key point in the virtual reality coordinate system For the first Confidence level of detection of key hand features This refers to the number of key points on the hand.

4. The robust estimation and redirection method for hand key points based on spatiotemporal constraints according to claim 1, characterized in that, The acquisition of the operator's wrist pose and key hand coordinates output by the virtual reality device or visual hand tracking module specifically includes: Short-term prediction of the operator's wrist pose is performed, and the predicted operator's wrist position is expressed as follows: （4）； in, Indicates the first Predicted wrist position after compensation for delay in one control cycle; This represents the time scale coefficient for location prediction; When wrist pose prediction is represented by rotation matrix increments: （5）； in, Indicates the first Predicted wrist posture value after compensation for delay in one control cycle; For attitude prediction time scale coefficients; , They represent special orthogonal groups respectively. Its Lie algebra The exponential and logarithmic mappings between them For transpose; Set the tool attitude compensation matrix as follows The end-point tool bias is The spatial scale coefficient is The target pose at the end of the robotic arm is then represented as: （6）； in, Indicates the first The robot arm end-effector pose is generated in the robot base coordinate system in each control cycle.

5. The robust estimation and redirection method for hand key points based on spatiotemporal constraints according to claim 1, characterized in that, The prior prediction based on the stable posterior state of the previous control cycle is used to obtain the prior prediction result, specifically as follows: For the Each key hand point is used to set the stable posterior state of the previous control cycle, including position. ,speed and acceleration Then the current prior prediction result is expressed as: （7）； （8）； in, Indicates the first The first control cycle, the... Prior predictions of the positions of key hand points; Indicates the first The first control cycle, the... Prior velocity predictions for each key hand point; , and These represent the output values ​​after robust estimation in the previous control cycle. The position, velocity, and acceleration of key points on the hand.

6. The robust estimation and redirection method for hand keypoints based on spatiotemporal constraints according to claim 1, characterized in that, No. The multimodal reliability index of each key hand point is expressed as follows: （9）； in, Indicates the first The first control cycle, the... Multimodal reliability indicators for key hand points; and They represent the first Skeletal length error and higher-order motion anomaly degree at key hand points; , and These are the bandwidth parameters of the corresponding Gaussian kernel function; The skeleton length error can be expressed as: （10）； in, Indicates the first Within the first control cycle, the first The three-dimensional observation position of the topological parent node of each key hand point; Represents the Euclidean distance between vectors; For the The first finger, denoted as the set of key hand points it contains. And calculate finger-level reliability: （12）； in, Indicates finger-level reliability. Indicates the first The weight coefficient of each key hand point in the reliability calculation of its respective finger level.

7. The robust estimation and redirection method for hand keypoints based on spatiotemporal constraints according to claim 1, characterized in that, The process involves reconstructing the trajectory of each hand key point based on the finger-level reliability of each finger in the dexterous hand, thereby obtaining the stable Cartesian key points corresponding to each hand key point. Specifically, this includes: When finger-level reliability When the threshold is below the finger level, the update of stable hand Cartesian key points corresponding to each hand key point on that finger is suppressed, and the set local anomaly handling method is executed. The remaining fingers and each key point on the wrist are at a length of Local sliding window Internal execution trajectory reconstruction: For the Each key hand point is represented by its true continuous trajectory as follows: And construct the following optimization objective: （13）； in, For a moment, The time number; Represents a robust kernel function; Indicates the first Key points of the hand The original Cartesian coordinates in the virtual reality coordinate system at any given moment; Indicates the first step that needs optimization. Key points of the hand in the continuous time domain The true continuous trajectory on; and Representing the true continuous trajectory The first and second derivatives with respect to time; and These are the weight penalty coefficients for the velocity smoothing regularization term and the acceleration smoothing regularization term, respectively. Indicates the first The topological parent node of each key hand point in the continuous time domain The true continuous trajectory on; This represents the weight penalty coefficient for the skeleton topology consistency constraint term. To optimize variables; The optimization objective simultaneously satisfies the following dynamic boundary conditions: （14）； Under dynamic boundary constraints, the optimization objective is optimized. After optimization, the current time step is taken. Posterior value corresponding to the true continuous trajectory As a key point for stabilizing the hand, Cartesian.

8. The robust estimation and retargeting method for hand keypoints based on spatiotemporal constraints according to claim 1, characterized in that, The optimization, redirection, and inverse kinematics calculation of the Cartesian key points of the stable hand to obtain the target joint angles of the end-effector dexterity hand specifically includes: After obtaining the Cartesian keypoints of the stable hand, transform the Cartesian keypoints of the stable hand to the local coordinate system. The relative Cartesian point is obtained from the middle. The local coordinate system Taking the geometric center of the palm of the dexterous hand as the origin, and the vector direction from the center of the wrist to the metacarpophalangeal joint of the middle finger as... The axis is defined by the direction of the normal vector perpendicular to the palm and upwards. The axis is determined according to the right-hand rule. axis; When using position-type redirection, several relative Cartesian points are selected as position references, and the target joint angles of the end effector are solved: （15）； in, For the current target joint angle of the dexterous hand, This represents the joint angle configuration vector of the dexterous hand to be optimized; For end-effector dexterity The forward kinematic position relative to a Cartesian point The scale mapping factor from human hand to end-effector dexterity. and The weighting coefficients are used to constrain the time continuity of joint angles and the velocity amplitude limit. To represent the dexterity of the end effector The weight coefficient of a relative Cartesian point in positional relocation optimization; To indicate the first One control cycle, with the end effector's first... The position of the stable hand Cartesian key point corresponding to a relative Cartesian point; When using vector-based redirection, it is based on a set of hand keypoint pairs. Construct a relative vector reference and solve for the target joint angles of the end effector: （16）； in, For the set of key points of the hand A relative Cartesian point pair; To represent relative Cartesian point pairs The weight coefficients of the relative vectors formed in vector-based retargeting optimization. , To represent the first dexterity of the end effector The relative Cartesian point and the first The forward kinematic position relative to a Cartesian point , To represent respectively the first Each control cycle forms a relative Cartesian point pair. The The and the first The set of hand key point pairs includes several relative Cartesian point pairs. .

9. The robust estimation and redirection method for hand key points based on spatiotemporal constraints according to claim 1, characterized in that, For the left and right hands, obtain the target poses of the left and right robotic arm end caps, respectively. , and the target joint angles of the left and right distal dexterous hands , The final control vector is expressed as: （17）。 10. A robust estimation and redirection system for hand keypoints based on spatiotemporal constraints, used to implement the robust estimation and redirection method for hand keypoints based on spatiotemporal constraints as described in any one of claims 1-9, characterized in that, include: A multimodal virtual reality perception module is used to acquire the operator's wrist pose and coordinates of key points of the operator's hand, output by a virtual reality device or a visual hand tracking module. The coordinate system calibration module is used to obtain the calibration matrix from the virtual reality coordinate system to the robot base coordinate system; using the calibration matrix from the virtual reality coordinate system to the robot base coordinate system, the target pose of the robotic arm end effector is generated based on the operator's wrist pose. The high-order spatiotemporal kinematics prediction module is used to make prior predictions based on the stable posterior state of the previous control cycle and obtain the prior prediction results. The adaptive reliability fusion module is used to calculate the multimodal reliability index of each hand key point in the end-effector dexterity hand based on the coordinates of the hand key points and the prior prediction results; and to determine the finger-level reliability of each finger in the end-effector dexterity hand based on the multimodal reliability index of the hand key points. The local spatiotemporal sliding window reconstruction module is used to perform trajectory reconstruction on each hand key point based on the finger-level reliability of each finger in the dexterous hand, so as to obtain the stable hand Cartesian key points corresponding to each hand key point. The hierarchical inverse kinematics solution module is used to optimize and redirect the Cartesian key points of the stable hand and perform inverse kinematics calculations to obtain the target joint angles of the end-effector dexterity hand. The end effector drive sending module is used to obtain the final control vector based on the target pose of the robotic arm end and the target joint angle of the end dexterous hand, and send it to the robotic arm and its end dexterous hand.

Images