Dexterous hand control method and device, teleoperation glove, storage medium and program product

Abstract

The application discloses a dexterous hand control method and device, a teleoperation glove, a storage medium and a program product, and belongs to the technical field of dexterous hands. The dexterous hand is in communication connection with the teleoperation glove. The teleoperation glove comprises a plurality of magnetic encoders which are respectively arranged at finger joints of the teleoperation glove. The method comprises the following steps: acquiring target pose data of the teleoperation glove; and controlling the dexterous hand to operate based on the target pose data and a preset mapping relationship. The preset mapping relationship is used for representing the motion mapping between the joints of the dexterous hand and the joints of the teleoperation glove. The preset mapping relationship is constructed based on the calibration pose data of the teleoperation glove under the conditions of single finger motion and relative motion of multiple fingers. The dexterous hand control method can accurately and stably reproduce the hand action of a wearer, and improves the teleoperation precision and following stability of the dexterous hand.

Description

Technical Field

[0001] This application belongs to the field of dexterous hand technology, and particularly relates to a dexterous hand control method, device, remote control glove, storage medium and program product. Background Technology

[0002] In the field of remote control of robot dexterous hands, most hand motion capture solutions in related technologies use resistive bending sensors or IMU inertial sensors, which have problems such as positioning drift, insufficient accuracy, and complex calibration, making it difficult to achieve high-precision real-time remote control of dexterous hands. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a dexterous hand control method, device, remote control glove, storage medium, and program product, enabling the dexterous hand to accurately and stably reproduce the wearer's hand movements, thereby improving the teleoperation accuracy and tracking stability of the dexterous hand.

[0004] In a first aspect, this application provides a method for controlling a dexterous hand, wherein the dexterous hand is communicatively connected to a remote-controlled glove, and the remote-controlled glove includes a plurality of magnetic encoders respectively disposed at each finger joint of the remote-controlled glove, the method comprising: Obtain the target pose data of the remote control glove; Based on the target pose data and the preset mapping relationship, the dexterous hand is controlled to operate; wherein... The preset mapping relationship is used to characterize the motion mapping between the joints of the dexterous hand and the joints of the teleoperated glove. The preset mapping relationship is constructed based on the calibration pose data of the teleoperated glove in single-finger movement and multi-finger relative movement.

[0005] According to the dexterous hand control method of this application, target pose data is acquired by setting magnetic encoders at the joints of each finger of the telescopic glove. The high-resolution characteristics of the encoders enable high-precision capture of joint movements, improving the accuracy of angle positioning and effectively reducing pose drift during long-term use. At the same time, the preset mapping relationship is constructed based on the calibration pose data of the telescopic glove under single-finger movement and multi-finger relative movement conditions. This can adapt to the individual hand movement range of different wearers and accurately reflect the relative positional relationship between each finger. Thus, the dexterous hand can accurately and stably reproduce the wearer's hand movements, improving the teleoperation accuracy and following stability of the dexterous hand.

[0006] According to one embodiment of this application, the preset mapping relationship is constructed based on the calibration pose data of the remote control glove under single-finger movement and multi-finger relative movement conditions, and includes: Based on the calibration pose data under the single-finger movement, the motion constraints corresponding to each finger joint are calibrated; Based on the calibration pose data under the relative motion of the multiple fingers, the calibration motion constraints are optimized, and based on the optimized motion constraints, the preset mapping relationship is constructed.

[0007] According to one embodiment of this application, the calibration of motion constraints corresponding to each finger joint based on the calibration pose data under the single-finger movement condition includes: From the calibrated pose data under the single-finger movement condition, obtain the angle data of each finger joint during flexion and extension activities; The angle data are statistically analyzed to determine the first and second range of motion of each finger joint. Based on the first and second movement angles of each finger joint, a corresponding normalization function is constructed, and the normalization function is used as the motion constraint for each finger joint.

[0008] According to one embodiment of this application, the step of optimizing the calibrated motion constraints based on the calibration pose data under the relative motion of the multiple fingers, and constructing the preset mapping relationship based on the optimized motion constraints, includes: Based on the optimized motion constraints and the motion posture of the dexterous hand during the execution of the corresponding action by the remote control glove, the preset mapping relationship is constructed.

[0009] According to one embodiment of this application, the step of optimizing the calibrated motion constraints based on the calibration pose data under the relative motion of the multiple fingers, and constructing the preset mapping relationship based on the optimized motion constraints, includes: Obtain the roll level adjustment parameters; Based on the roll level adjustment parameters, the roll axis mapping of the remote control glove is adjusted when performing relative movements of multiple fingers, and the pose compensation data of each finger joint in the roll degree of freedom is determined. The optimized motion constraints are updated based on the pose compensation data, and the preset mapping relationship is generated based on the compensated motion constraints.

[0010] According to one embodiment of this application, the calibration pose data under the condition of relative movement of multiple fingers includes: the pose of each finger joint when any two fingers perform a finger-opposing action.

[0011] According to one embodiment of this application, controlling the operation of the dexterous hand based on the target pose data and a preset mapping relationship includes: Based on the target pose data and the preset mapping relationship, the target joint angle data corresponding to the dexterous hand is obtained; Determine the target communication protocol that matches the device model of the dexterous hand, and configure the corresponding target operating mode; Based on the target communication protocol and in accordance with the target working mode, the target joint angle data is sent to the dexterous hand to drive the dexterous hand to perform the corresponding joint movements.

[0012] Secondly, this application provides a dexterous hand control device, which includes: The first processing module is used to acquire the target pose data of the remote control glove; The second processing module is used to control the operation of the dexterous hand based on the target pose data and the preset mapping relationship; wherein, the preset mapping relationship is used to characterize the motion mapping between the joints of the dexterous hand and the joints of the remote control glove, and the preset mapping relationship is constructed based on the calibration pose data of the remote control glove in the case of single finger movement and multi-finger relative movement.

[0013] According to the dexterous hand control device of this application, target pose data is acquired by setting magnetic encoders at the joints of each finger of the telescopic glove. The high resolution characteristics of the encoders enable high-precision capture of joint movements, improving the accuracy of angle positioning and effectively reducing pose drift during long-term use. At the same time, the preset mapping relationship is constructed based on the calibration pose data of the telescopic glove under single-finger movement and multi-finger relative movement conditions. It can adapt to the individual hand movement range of different wearers and accurately reflect the relative positional relationship between each finger. Thus, the dexterous hand can accurately and stably reproduce the wearer's hand movements, improving the teleoperation accuracy and following stability of the dexterous hand.

[0014] Thirdly, this application provides a remote-controlled glove, which is communicatively connected to a dexterous hand, and the remote-controlled glove includes: Multiple magnetic encoders; the multiple magnetic encoders are respectively disposed on the lateral swing, root and fingertip of each finger of the remote control glove, corresponding to at least fifteen joint degrees of freedom; As described in the second aspect, the dexterous hand control device includes a plurality of magnetic encoders electrically connected to the dexterous hand control device.

[0015] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dexterous hand control method as described in the first aspect above.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the dexterous hand control method as described in the first aspect above.

[0017] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: By installing magnetic encoders at the finger joints of the remote control glove to acquire target pose data, the high-resolution characteristics of the encoders enable high-precision capture of joint movements, improving the accuracy of angle positioning and effectively reducing pose drift during long-term use. At the same time, the preset mapping relationship is constructed based on the calibration pose data of the remote control glove under single-finger movement and multi-finger relative movement conditions. It can adapt to the individual hand movement range of different wearers and accurately reflect the relative positional relationship between each finger. This enables the dexterous hand to accurately and stably reproduce the wearer's hand movements, improving the teleoperation accuracy and following stability of the dexterous hand.

[0018] Furthermore, based on the calibration pose data of single-finger movements and relative movements of multiple fingers (such as finger-opposing movements), the motion constraints of each finger joint are calibrated and optimized. This allows for automatic adaptation to the individual range of motion of different wearers and accurate establishment of a human-machine mapping relationship. Simultaneously, by using roll level adjustment parameters to determine pose compensation data in the roll degree of freedom, the optimized motion constraints are updated, effectively reducing spatial torsional deviations. This two-level calibration and dynamic compensation mechanism can quickly adapt to different users, highly match the movement space of dexterous hands, lower the usage threshold, and improve calibration accuracy.

[0019] Furthermore, by configuring a target communication protocol and corresponding target working mode that match the model of the dexterous hand device, the remote control glove can flexibly adapt to various models of dexterous hands and is compatible with working modes such as direct connection and indirect connection. This mechanism effectively improves the hardware compatibility of the system and gives the product good scalability and versatility.

[0020] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0021] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating the dexterous hand control method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the dexterous hand control device provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0023] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0024] The dexterous hand control method, device, remote control glove, storage medium, and program product provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0025] The dexterous hand control method provided in this application embodiment can be executed by an electronic device or a functional module, functional entity or system in an electronic device that can implement the dexterous hand control method. The dexterous hand control method provided in this application embodiment is described below using a remote-controlled glove as the execution subject.

[0026] During the research and development process, the inventors discovered that in the field of remote operation of dexterous hands in robots, most hand motion capture solutions use resistive bending sensors or IMU inertial sensors, which suffer from problems such as positioning drift, insufficient accuracy, and complex calibration, making it difficult to achieve high-precision real-time remote control of dexterous hands. Furthermore, different models of dexterous hands have different joint configurations and ranges of motion, lacking a universal mapping and adaptation solution.

[0027] To address the issues of hand pose capture drift and difficulty in accurately mapping and adapting to the joint space of dexterous hands in related technologies, the inventors designed a dexterous hand control method. This method involves a dexterous hand communicating with a remote glove, which includes multiple magnetic encoders located at each finger joint. The method includes: acquiring target pose data of the remote glove; and controlling the dexterous hand based on the target pose data and a preset mapping relationship. The preset mapping relationship characterizes the motion mapping between the joints of the dexterous hand and the joints of the remote glove, and is constructed based on the calibration pose data of the remote glove under single-finger movement and multi-finger relative movement conditions.

[0028] According to the dexterous hand control method provided in this application embodiment, target pose data is acquired by setting magnetic encoders at the finger joints of the telescopic glove. The high-resolution characteristics of the encoders enable high-precision capture of joint movements, improving the accuracy of angle positioning and effectively reducing pose drift during long-term use. At the same time, the preset mapping relationship is constructed based on the calibration pose data of the telescopic glove under single-finger movement and multi-finger relative movement conditions. This can adapt to the individual hand movement range of different wearers and accurately reflect the relative positional relationship between each finger. Thus, the dexterous hand can accurately and stably reproduce the wearer's hand movements, improving the teleoperation accuracy and following stability of the dexterous hand.

[0029] The dexterity hand control method provided in this application can be applied to dexterity hands and remote-controlled gloves that are communicatively connected to dexterity hands.

[0030] In some embodiments, the remote control glove includes a plurality of magnetic encoders respectively disposed at each finger joint of the remote control glove.

[0031] In this embodiment, each magnetic encoder may include a cooperating magnetic induction element and a magnetic element. One of the magnetic induction element and the magnetic element is disposed on a mounting base of the corresponding finger mechanism, and the other is disposed on a connecting rod rotatably connected to the corresponding mounting base.

[0032] The magnetic induction element is configured to collect changes in the magnetic field of the corresponding magnetic element, and to obtain relative rotation angle information between the mounting base and the corresponding connecting rod, thereby providing high-resolution angle measurement data.

[0033] In actual operation, multiple magnetic encoders are respectively installed on the lateral, root, and fingertip of each finger of the remote control glove, corresponding to at least fifteen joint degrees of freedom. The remote control glove is equipped with a five-finger mechanism, each with multiple degrees of freedom. The proximal end of each finger mechanism is rotatably connected to the fixed end via a first axis to form a lateral joint, and the magnetic encoders located therebetween are used to collect the lateral pose data of the fingers. Multiple mounting bases are arranged sequentially, and adjacent mounting bases are rotatably connected via a second axis or its parallel axis to form multi-stage flexion-extension joints. Magnetic encoders located between each stage of mounting base and its corresponding link are used to collect the flexion-extension pose data of the fingers at the root and fingertip. The lateral, root, and fingertip flexion-extension degrees of freedom of the five-finger mechanism together constitute a spatial motion capture network with at least fifteen joint degrees of freedom.

[0034] like Figure 1 As shown, the dexterous hand control method includes steps 110 and 120.

[0035] Step 110: Obtain the target pose data of the remote control glove; In this step, the target pose data is a portion of all the data generated during the wearer's hand movements, and this portion of data is used to characterize the motion state of each finger joint of the remote control glove.

[0036] In some embodiments, the target pose data may include, but is not limited to, one or more of the following: The angle data of each finger joint, such as the bending angle of the root joint of each finger relative to the palm plane, the bending angle of the middle joint or fingertip joint, and the lateral swing angle of the root joint in the palm plane, etc. In some embodiments, angle data can be acquired in real time based on multiple magnetic encoders set at each finger joint of the remote control glove. The multiple magnetic encoders are set to correspond to the three dimensions of each finger: lateral swing, root, and fingertip, covering a total of 15 degrees of freedom of the hand joints. The magnetic encoders can adopt a 16-bit precision specification, and the angle capture accuracy of each finger joint can reach 0.1°.

[0037] In this embodiment, the collected angle data may include: Angular velocity or angular acceleration data for each finger joint; Data on the movement trajectory of each finger joint over time; Data on the relative positional relationships between the fingers, such as the relative distance, relative angle, or relative pose between the fingertips; The overall posture data of the remote control glove, such as the spatial orientation of the hand and the rotation angle of the palm plane relative to the inertial coordinate system.

[0038] In some embodiments, the target pose data can be the target pose data generated by the wearer in the case of single finger movement and / or relative movement of multiple fingers.

[0039] In this embodiment, the calibration pose data is used to construct a preset mapping relationship between the joints of the teleoperated glove and the joints of the dexterous hand. The specific construction method of the preset mapping relationship is described in step 120 below, and will not be repeated here.

[0040] In some embodiments, single-finger movement refers to the movement in which the wearer independently moves one of their fingers while the other fingers remain essentially stationary, including but not limited to the following: A finger bends from its straight position to its maximum bending position; A finger extends from a bent position to its maximum extension position; A single finger is swung laterally along the plane of the palm to collect data on the range of motion in the lateral dimension. One finger is moved sequentially at its base joint, middle joint, and fingertip joint to collect range of motion data for each joint. A single finger is subjected to continuous reciprocating bending and stretching movements to collect data on the angle changes of that finger at different movement speeds.

[0041] In some embodiments, relative movement of multiple fingers refers to the movement corresponding to a change in the relative position between two or more fingers caused by the wearer, including but not limited to the following: The thumb then performs a finger-opposing motion in sequence with the index, middle, ring, and little fingers; Two or more fingers simultaneously move from an open position toward the center of the palm, or simultaneously move from a closed position toward the outside. Perform alternating bending movements between different fingers, such as bending the index and middle fingers alternately; Multiple fingers work together to perform grasping or pinching actions to simulate the grasping posture of real objects; The thumb and the other fingers sequentially perform finger-to-finger movements with different contact positions or different contact intensities to collect relative position calibration data at multiple levels.

[0042] In other embodiments, the target pose data may also be angle data obtained in real time based on multiple magnetic encoders set at each finger joint of the remote control glove to collect the angle of each finger joint.

[0043] In this embodiment, multiple magnetic encoders can cover three dimensions of each finger: lateral movement, base, and fingertip.

[0044] The collected target pose data may include: Data on the bending angle of the base joint of each finger relative to the palm plane; Data on the bending angle of the mid-joint or fingertip joint of each finger; Data on the lateral angle of the base joint of each finger within the plane of the palm.

[0045] The magnetic encoder can continuously acquire the angle of each finger joint based on a preset sampling frequency, and has high acquisition accuracy (e.g., up to 0.1°).

[0046] In actual operation, after the remote control glove is powered on, multiple magnetic encoders work continuously. The magnetic encoders convert the angle changes of each finger joint into corresponding digital signals. For example, the current position of each finger joint is encoded with 16-bit precision and then output. After the digital signals are format decoded, unit converted (for example, converted into signed angle values ​​in units of 0.1°) and filtered, they are output to the subsequent processing module in real time as target pose data. The target pose data is processed differently according to its working stage: in the calibration stage, the acquired target pose data is recorded as calibration pose data and stored in the memory to build a preset mapping relationship; During normal teleoperation, the acquired target pose data is sent to the mapping processing stage in real time. After being combined with the preset mapping relationship, the target joint angle data that matches the dexterous hand joint space is obtained, thereby driving the dexterous hand to reproduce the wearer's hand movements.

[0047] Step 120: Control the dexterous hand to move based on the target pose data and the preset mapping relationship; In this step, a preset mapping relationship is used to characterize the motion mapping between the joints of the dexterous hand and the joints of the remote control glove.

[0048] The preset mapping relationship is constructed based on the calibration pose data of the remote control glove in the case of single finger movement and multi-finger relative movement.

[0049] In actual execution, step 120 includes two processes: constructing a preset mapping relationship and controlling the dexterous hand in real time based on the preset mapping relationship.

[0050] In the process of constructing the preset mapping relationship, the motion constraints corresponding to each finger joint of the remote control glove (such as the range of motion of each finger joint, which can be characterized by the extreme values ​​of the angles of each finger joint) are first calibrated based on the calibration pose data under the single finger movement condition. Then, based on the calibration pose data of multiple fingers in relative motion (e.g., the pose of each finger joint during the thumb's sequential finger-to-finger action with the index, middle, ring, and little fingers), the motion constraints are optimized to further reflect the relative positional relationship between the fingers; finally, a preset mapping relationship is generated based on the optimized motion constraints.

[0051] In some embodiments, the process of constructing the preset mapping relationship may also incorporate the motion posture information of the dexterous hand performing the corresponding action in the remote control glove, or introduce a roll level adjustment parameter to compensate for the mapping relationship, so as to adapt to the differentiated mapping requirements under different finger-to-finger effects.

[0052] In some embodiments, the remote control glove performs mapping processing based on real-time acquired target pose data and a preset mapping relationship to obtain target joint angle data corresponding to the dexterous hand.

[0053] In actual operation, the remote control glove can further determine the target communication protocol that matches the dexterous hand's device model (e.g., L6, L10, L20, L21, etc.) and configure the corresponding working mode (e.g., the direct connection mode where the remote control glove directly connects to the dexterous hand, or the indirect connection mode where the remote control glove is relayed through the host computer SDK). Finally, based on the target communication protocol and according to the configured working mode, the target joint angle data is sent to the dexterous hand to drive the dexterous hand's joints to reproduce the wearer's hand movements.

[0054] In some embodiments, the preset mapping relationship is constructed based on the calibration pose data of the remote control glove in single-finger movement and multi-finger relative movement, including: Based on the calibration pose data of single-finger movement, the motion constraints corresponding to each finger joint are calibrated; Based on the calibration pose data under the condition of relative movement of multiple fingers, the calibration motion constraints are optimized, and a preset mapping relationship is constructed based on the optimized motion constraints.

[0055] In this embodiment, the process of constructing the preset mapping relationship is implemented using a two-level calibration method, namely rapid calibration and depth calibration.

[0056] Motion constraints at least characterize the upper and lower limits of movement of each finger joint, reflecting the individual physiological characteristics of the wearer's hand.

[0057] Among them, the rapid calibration is based on the calibration posture data under single finger movement. By allowing the wearer to fully move each finger, the individual hand movement range of the wearer is adaptively determined, and then the movement constraints corresponding to each finger joint are initially calibrated. Depth calibration is based on the calibration pose data under the condition of relative movement of multiple fingers (such as the pose data generated when the wearer performs the finger-opposing action of the thumb with the index finger, middle finger, ring finger and little finger in sequence). The motion constraints mentioned above are optimized so that the motion constraints not only reflect the individual range of movement of the wearer, but also include the relative positional relationship information between each finger. Based on the optimized motion constraints, a preset mapping relationship is constructed to establish the mapping correspondence from the wearer's hand joint space to the dexterity hand joint space.

[0058] In some embodiments, based on the calibration pose data under single-finger movement, the motion constraints corresponding to each finger joint are calibrated, including: From the calibrated pose data of single finger movement, obtain the angle data of each finger joint during flexion and extension activities; Statistical analysis of angle data was conducted to determine the first and second range of motion for each finger joint. Based on the first and second range of motion of each finger joint, a corresponding normalization function is constructed, and the normalization function is used as the motion constraint for each finger joint.

[0059] In this embodiment, the first angle of movement is the maximum angle of movement reached by each finger joint during flexion and extension, and the second angle of movement is the minimum angle of movement reached by each finger joint during flexion and extension.

[0060] During the Quick Calibration process, the system rapidly (e.g., ≤30s) adapts to the individual's finger range of motion, whether the wearer is wearing the device for the first time or when a new wearer is selected. The wearer can trigger calibration via a preset physical button (e.g., long press of the K2 button). The remote control glove then sends a calibration start command (CALIBRATION_CMD_QUICK_START) via a wireless communication protocol (e.g., ESP-NOW). Upon receiving the command, the receiver provides a running status code (STATUS_CODE_QUICK_CAL_RUNNING), while the remote control glove's interface (e.g., an OLED screen) prompts the wearer to fully move their fingers. The wearer sequentially performs actions such as opening, clenching a fist, and spreading all five fingers several times to cover the maximum flexion and extension range of each finger joint. During this process, the receiver collects and statistically analyzes the minimum and maximum range of motion (θmin_j) for each degree of freedom (DoF) in real time.

[0061] After the wearer triggers the calibration end command (CALIBRATION_CMD_FINISH) again via the physical button and the receiver returns a completion status code (STATUS_CODE_QUICK_CAL_COMPLETE), the system establishes a corresponding normalization function for each degree of freedom based on the statistically obtained first and second activity angles.

[0062] In some embodiments, a corresponding normalization function is established for each degree of freedom, which can be expressed by the formula: φ_j(raw) = (raw) min(R_j) ) / ( max(R_j) min(R_j) ).

[0063] Wherein, φ_j(raw) represents the standard mapping value output after normalization of the j-th finger joint degree of freedom, and its value range can be limited to the standard data interval [0, 1]; raw represents the raw angle data of the j-th finger joint degree of freedom that is collected in real time and continuously input; R_j represents the set of active angle data covered by the j-th finger joint degree of freedom during the calibration phase; max(R_j) represents the second active angle corresponding to the finger joint degree of freedom, that is, the maximum active angle obtained during flexion and extension activities; min(R_j) represents the first active angle corresponding to the finger joint degree of freedom, that is, the minimum active angle obtained during flexion and extension activities.

[0064] This normalization function dynamically maps continuously input raw angle data to a standard data range of [0, 1], which the system then stores as motion constraint parameters for the corresponding finger joints. Through complete communication interaction and underlying data statistics processes, this mechanism can quickly and accurately adapt to the individual finger movement range of different wearers, reducing the interference of individual physiological size differences on spatial mapping accuracy.

[0065] In some embodiments, based on the calibration pose data under the condition of relative movement of multiple fingers, the calibrated motion constraints are optimized, and based on the optimized motion constraints, a preset mapping relationship is constructed, including: Based on the optimized motion constraints and the motion posture of the dexterous hand during the execution of corresponding actions in the remote control glove, a preset mapping relationship is constructed.

[0066] In some embodiments, the calibration pose data under relative movement of multiple fingers includes: the pose of each finger joint when any two fingers are performing a finger-opposing action.

[0067] In this embodiment, the user wears a remote control glove and sequentially performs finger-op actions with the thumb, index finger, middle finger, ring finger, and little finger to establish multiple corresponding finger-op anchor points. These multiple finger-op anchor points are then used as calibration pose data for the relative movement of multiple fingers.

[0068] In actual execution, let the original vectors of the three degrees of freedom (DoF) of the thumb joints collected by the telescopic glove during the k-th finger-to-finger interaction be: P k = (P_yaw^k, P_pitch^k, P_tip^k), where P_yaw^k represents the original angle value of the thumb root joint in the lateral degree of freedom, P_pitch^k represents the original angle value of the thumb root joint in the pitch degree of freedom, and P_tip^k represents the original angle value of the thumb tip joint in the flexion degree of freedom.

[0069] At the same time, record the DoF value Q_k of the fingertip of the corresponding finger among the other four fingers (where k = 1, 2, 3, 4, corresponding to the index finger, middle finger, ring finger, and little finger, respectively).

[0070] This results in four finger anchor points: A_k = (P_k, Q_k), k = 1, 2, 3, 4.

[0071] Where A_k represents the finger-pairing anchor point corresponding to the k-th finger-pairing action, P_k represents the original pose vector of the thumb during this finger-pairing action, and Q_k represents the fingertip degree-of-freedom angle values ​​of the other fingers participating in this finger-pairing action. These multiple finger-pairing anchor points will be saved to a storage medium for use in subsequent control processes to optimize calibrated motion constraints and construct preset mapping relationships.

[0072] In actual use, the aforementioned rapid calibration and depth calibration can solve the problems of static range adaptation and target pose alignment. However, when the wearer performs the pinching action in real-time, issues such as state oscillation, pose alignment deviation, rebound after contact, and ambiguity in the determination of the pinching critical state may still occur, along with problems of finger instability and mechanical play. The main reasons are: the flexibility of human hand joints is limited, and there is slight unconscious tremor of the knuckles when maintaining the pinching action (e.g., 1°~3° tremor); there is slight physical rebound of the soft tissue of the fingertips after contact (e.g., 1~2mm rebound); there is an inherent mechanical gap (i.e., mechanical play) between the remote control glove mechanism and the human hand; and the quantization error and filtering noise of the sensor are easily amplified by the linear mapping mechanism near the pinching boundary.

[0073] To address the instability and mechanical play issues during real-time pinching, some embodiments optimize the pre-defined mapping relationship by defining a separate attraction basin for each finger (k = 1..4): when the measured hand posture P is relative to the k-th anchor point P... k When the distance d_k is less than the threshold, the robot arm's target pose is nonlinearly "sucked" into anchor point A. k The corresponding target is that the inhalation intensity smoothly decreases to 0 with distance, and boundary oscillations are reduced through hysteresis and velocity gating.

[0074] It should be noted that the above algorithm only activates the "attraction" effect within the small neighborhood of thumb-to-palm contact. When the action falls outside the attraction domain, it degenerates into a regular linear mapping, thus not affecting other non-finger-to-palm actions.

[0075] The finger-attraction algorithm implemented based on the above overall idea can be divided into four independent and collaborative sub-mechanisms, including: Attraction Band mechanism, Soft Attraction Potential mechanism, Hysteresis Lock mechanism, and Velocity Gating mechanism.

[0076] In some embodiments, the attraction threshold band mechanism includes: Obtain the joint pose vector of the thumb and target finger of the telescopic glove in the current frame; Calculate the normalized Euclidean distance between the joint attitude vector and the corresponding finger anchor point, and use it as the spatial pose deviation parameter for finger state switching. Based on the comparison results between the spatial pose deviation parameters and the preset distance threshold, the control system enters or exits the attraction activation state.

[0077] In this embodiment, the normalized Euclidean distance between the joint attitude vector and the corresponding finger anchor point can be expressed by the formula: d_k = || W ⊙ ( P P_k ) ||2.

[0078] Where d_k represents the spatial pose deviation parameter (i.e., normalized Euclidean distance) calculated for the k-th target finger in the current frame; P represents the joint pose vector of the thumb and target finger of the telescopic glove acquired in real time in the current frame; P_k represents the pre-calibrated pose vector of the finger-opposing anchor point corresponding to the k-th target finger; W represents the preset joint degree of freedom weight vector, used to configure and characterize the dominant weight or contribution difference of different joint degrees of freedom (such as yaw, pitch, fingertip flexion) to the finger-opposing action judgment result; ⊙ represents element-wise multiplication.

[0079] In some embodiments, for the joint degree-of-freedom weight vector (W), the weight value of its yaw degree of freedom (Yaw) can be set to 0.3, the weight value of its pitch degree of freedom (Pitch) can be set to 0.5, and the weight value of its fingertip flexion degree of freedom (Tip) can be set to 1.0, thereby reflecting the dominant role of fingertip distance in the success or failure of finger-to-finger movements; the above weight values ​​can be configured and stored in a non-volatile storage system (NVS) and can be dynamically adjusted.

[0080] In some embodiments, based on the comparison result of spatial pose deviation parameters and a preset distance threshold, the system is controlled to enter or exit the attraction activation state, including: When the spatial pose deviation parameter decreases monotonically and is less than or equal to the first distance threshold for the first time, it is determined that the current frame enters the attraction activation state. When the spatial pose deviation parameter increases monotonically and is greater than or equal to the second distance threshold for the first time, the current frame is determined to exit the attraction activation state.

[0081] In this embodiment, the first distance threshold is less than the second distance threshold. For example, the first distance threshold can be set to 8° and the second distance threshold can be set to 14°. The difference between the two can effectively absorb minor joint vibrations within a range of ±3°.

[0082] In actual execution, the numerical range between the first distance threshold and the second distance threshold constitutes the hysteresis band. Within the hysteresis band, the activation state of the system remains unchanged, thereby reducing the reciprocating oscillations and frequent state switching at the action boundary.

[0083] In some embodiments, the soft attraction potential function includes: When it is determined that the current frame has entered the attraction activation state, the linear mapping pose of the telescopic glove is obtained by conventional linear mapping calculation in the current frame; Calculate the corresponding smoothing coefficient based on the spatial pose deviation parameters of the current frame; The linearly mapped pose and the corresponding finger anchor point are combined using a convex combination based on a smoothing coefficient to generate and output a smoothly transitioned mapped pose.

[0084] In this embodiment, the above calculation process can be expressed by the formula: Output(P) = (1 λ(d_k) ) · M_linear(P) + λ(d_k) · A_k λ(d) = 3·t² 2·t³ The formula for calculating the intermediate variable t is as follows: t = clip( (d_out d) / (d_out d_in), 0, 1) Where Output(P) represents the mapped output pose after smooth transition; λ(d_k) represents the smoothing coefficient calculated based on the spatial pose deviation parameter, and its value range is limited to [0, 1]; M_linear(P) represents the linear mapped pose corresponding to the current pose of the telescopic glove; A_k represents the corresponding finger anchor point; d_k represents the spatial pose deviation parameter in the current frame; d_in represents the preset first distance threshold; d_out represents the preset second distance threshold; and clip represents the numerical clipping function that limits the internal calculation results to between 0 and 1.

[0085] In actual execution, when the spatial pose deviation parameter is greater than or equal to the second distance threshold, the smoothing coefficient is calculated to be 0, and the system outputs a completely linear mapped pose. When the spatial pose deviation parameter is less than or equal to the first distance threshold, the smoothing coefficient is calculated to be 1, and the system completely attaches the target pose to the finger anchor point. When the spatial pose deviation parameter is between the first and second distance thresholds, the smoothing coefficient can achieve a non-linear smooth transition. By introducing the above-mentioned three-fold smoothing interpolation logic, the system can ensure that the first and second derivatives of the issued mapped output pose with respect to time remain continuous at the threshold boundaries, thereby ensuring the smooth differentiability of the actuator on the time scale and reducing the risk of mechanical twitching of the dexterous hand due to transient command changes.

[0086] In some embodiments, the hysteresis locking mechanism includes: When the spatial pose deviation parameter is greater than the locking threshold, the system is determined to be in an unlocked state, and a mapped output pose is generated based on the smoothness coefficient calculated in real time. When the spatial pose deviation parameter decreases monotonically and is less than or equal to the locking threshold for the first time, the system is determined to enter the locking state, and the smoothing coefficient is set to the preset convergence maximum value. When the spatial pose deviation parameter increases monotonically and is greater than or equal to the release threshold for the first time, the system is determined to enter the release observation state. If the duration of time that the spatial pose deviation parameter is greater than the release threshold reaches the release time threshold, the control system exits the locking logic and returns to the unlocked state.

[0087] In this embodiment, the release threshold is greater than the lock threshold.

[0088] In some embodiments, the locking threshold can be set to 5°, the release threshold can be set to 12°, the release time threshold can be set to 80ms, and the preset convergence maximum can be set to 1.

[0089] In actual execution, to filter out disturbances caused by physical rebound of the fingertips or slight relaxation of muscles during prolonged pinching and holding, the system introduces a three-state logic mechanism including an unlocked state, a locked state, and a release observation state. The three-state logic mechanism is as follows: When the spatial pose deviation parameter is greater than the locking threshold, the system enters the unlocked state, and the robot arm performs convex combination behavior according to the soft attraction potential function. When the spatial pose deviation parameter is first less than or equal to the locking threshold, the system enters the locked state. At this time, the system forcibly sets the smoothing coefficient to the preset convergence maximum value, so that the controlled end strictly adheres to and is fixed at the finger anchor point, while ignoring the subsequent local micro-fluctuations of the deviation parameter.

[0090] When the spatial pose deviation parameter first exceeds or equals the release threshold, the system enters the release observation state; only when the parameter remains above the release threshold for a duration reaching the release time threshold does the system determine that the opening action is valid and exit the locking logic. This mechanism effectively reduces the risk of momentary rebound of the robotic hand caused by physiological disturbances or sensor rebound, and improves the stability of continuous physical grip.

[0091] In some embodiments, the speed gating mechanism includes: Calculate the rate of change of posture of the corresponding fingers of the remote control glove; Based on the comparison between the attitude change rate and the preset speed threshold, and in combination with the preset time stability condition, the system state switching command is intercepted or allowed.

[0092] In this embodiment, the specific process of calculating the attitude change rate can be expressed by the formula: v_k = || P[n] P[n 1] || / Δt Where v_k represents the pose change rate calculated for the target finger in the current frame; P[n] represents the joint pose vector acquired in real time in the current frame; P[n 1] represents the joint attitude vector acquired in the previous control cycle (i.e., the previous frame); Δt represents the time interval between two adjacent frames.

[0093] In some embodiments, based on a comparison between the attitude change rate and a preset speed threshold, and in conjunction with a preset time stability condition, the system state switching command is intercepted or allowed, including: When the rate of change of posture is greater than the preset speed threshold, the current gesture is determined to be a rapid active action. At this time, the system allows the immediate triggering of the switching command to enter the attraction activation state or exit the locked state. When the rate of change of posture is less than or equal to the speed threshold, the current gesture is determined to be a slow change or physiological tremor. After the duration of the rate of change of posture being less than or equal to the speed threshold reaches a preset stable time threshold, the control system performs the corresponding state switch.

[0094] In this embodiment, the speed threshold can be set to 30° / s, and the settling time threshold can be set to 40 ms.

[0095] In practical implementation, the speed gating mechanism is mainly used in the underlying control link to distinguish between the wearer's active posture adjustment intentions and unintentional physical jitter. By introducing this dynamic response gating logic, the system can not only improve the timeliness of the device's response when the wearer performs active operations (such as actively and quickly opening their fingers), but also effectively filter out single-frame data spikes (such as I2C crosstalk) caused by hardware-level communication crosstalk or quantization noise, reducing the risk of pseudo-state switching of the control system at threshold boundaries, thereby further enhancing the overall robustness of multi-finger finger-to-finger control.

[0096] Below is the pseudocode for the complete flow of the finger attraction algorithm. for each frame n: P = current_pose(n) # Current thumb + corresponding finger joint pose vector for k in {Index, Middle, Ring, Pinky}: d = weighted_distance(P, P_k)#distance v = || P[n] P[n 1] || / Δt# velocity # ----- Hysteresis Locked State Machine ----- if state[k] == Idle: if d <d_lock and stable(v, T_stable): state[k] = Lock elif state[k] == Lock: if d>d_release and stable(v, T_stable): state[k] = ReleaseWatch t_release_start[k] = now elif state[k] == ReleaseWatch: if d ≤ d_release: state[k] = Lock # Return to Lock elif now t_release_start[k]>T_release: state[k] = Idle # ----- Calculate λ ----- if state[k] == Lock: λ_k = 1.0 elif state[k] == ReleaseWatch: λ_k = smoothstep(d_out, d_in, d) λ_k = max(λ_k, 0.7) # Prevents mutations during release. else:# Idle λ_k = smoothstep(d_out, d_in, d) # ----- Output Convex Combination ----- target_k = (1 λ_k) · M_linear(P) + λ_k · A_k # Multi-anchor point fusion (the thumb only serves one opposing finger anchor point at a time) k_dom = argmin_k(d_k) Output = target_{k_dom} According to the dexterous hand control method provided in the embodiments of this application, by constructing a multi-dimensional finger-attachment mechanism including an attraction threshold band, a soft attraction potential function, hysteresis locking, and speed gating, adaptive adsorption and smooth transition of the finger target pose are achieved during the depth calibration process. This effectively overcomes problems such as pose alignment deviation, rebound after contact, and critical state oscillation caused by physiological micro-tremors of human fingers, contact rebound of fingertip soft tissue, inherent mechanical play, and sensor noise. This enables the dexterous hand to accurately and stably reproduce the wearer's hand movements, improving the teleoperation accuracy and following stability of the dexterous hand.

[0097] In some embodiments, based on the calibration pose data under the condition of relative movement of multiple fingers, the calibrated motion constraints are optimized, and based on the optimized motion constraints, a preset mapping relationship is constructed, including: Obtain the roll level adjustment parameters; Adjust the roll axis mapping of the remote control glove when performing relative movements of multiple fingers based on the roll level adjustment parameters, and determine the pose compensation data of the target finger in the roll degree of freedom. The motion constraints are updated and optimized based on the pose compensation data, and a preset mapping relationship is generated based on the compensated motion constraints.

[0098] In this embodiment, the roll level adjustment parameter is an adjustable level parameter used to adjust the mapping intensity between the remote control glove and the dexterous hand in the roll degree of freedom, in order to adapt to the differentiated finger-pinning reproduction effect under different finger-pinning action styles or different models of dexterous hands.

[0099] After the relative movements of multiple fingers are calibrated, differences in the mechanical structure of the target fingers (e.g., the thumb) in dexterity hands of different device models may lead to spatial torsional deviations when reproducing finger-to-finger movements, such as crossing too shallowly or too deeply. To address this, a roll level adjustment parameter can be introduced to configure corresponding compensation levels (e.g., low compensation level and high compensation level). Different compensation levels correspond to different roll compensation amounts of the target fingers during the palm-to-palm process.

[0100] In actual operation, users can switch and confirm the roll level adjustment parameters through the interactive interface and preset physical buttons. After receiving and confirming the parameters, the remote control device determines the pose compensation data of the target finger in the roll degree of freedom based on the roll level adjustment parameters, updates the optimized motion constraints, and writes the selected roll level adjustment parameters into the non-volatile storage area for saving so that they will take effect automatically when the device is powered on next time.

[0101] In some embodiments, the roll level adjustment parameter can be obtained by at least one of the following methods: User input via physical interactive elements (such as knobs, buttons, or DIP switches) on the remote control glove; Setting commands transmitted through the human-machine interface of a host computer or terminal device that is connected to the remote control glove; The remote control glove has pre-stored default configuration files corresponding to different dexterity hand models; Model information or recommended gear parameters are provided from the dexterity hand.

[0102] According to the dexterous hand control method provided in this application, based on the calibration posture data of single-finger movement and relative multi-finger movement (such as finger-opposing movements), the motion constraints of each finger joint are calibrated and optimized. This allows for automatic adaptation to the individual range of motion of different wearers and accurate establishment of a human-machine mapping relationship. Simultaneously, by using roll level adjustment parameters to determine posture compensation data in the roll degree of freedom, the optimized motion constraints are updated, effectively reducing spatial torsional deviation. This two-level calibration and dynamic compensation mechanism can quickly adapt to different users, highly match the movement space of the dexterous hand, lower the usage threshold, and improve calibration accuracy.

[0103] In some embodiments, step 120 further includes: Based on the target pose data and the preset mapping relationship, the target joint angle data corresponding to the dexterous hand is obtained; Determine the target communication protocol that matches the device model of the dexterous hand, and configure the corresponding target operating mode; Based on the target communication protocol and in accordance with the target working mode, the target joint angle data is sent to the dexterous hand to drive the dexterous hand to perform the corresponding joint movements.

[0104] In this embodiment, the remote control glove is configured to use a preset universal protocol switching mechanism to adapt to different device models of the dexterous hand (e.g., L6, L10, L20, L21, etc.).

[0105] The target operating modes include a direct connection mode where the glove directly controls the dexterous hand, and an indirect connection mode via a host computer.

[0106] According to the dexterity hand control method provided in the embodiments of this application, by configuring a target communication protocol and a corresponding target working mode that match the model of the dexterity hand device, the remote control glove can flexibly adapt to various models of dexterity hands and is compatible with working modes such as direct connection and indirect connection; this mechanism effectively improves the hardware compatibility of the system and gives the product good scalability and versatility.

[0107] In some embodiments, the target communication protocol can be the ESP-NOW wireless communication protocol, which provides low-latency data transmission (e.g., transmission latency of less than 10ms) between the remote glove and the receiver corresponding to the dexterous hand. Furthermore, based on the ESP-NOW wireless communication protocol, the remote glove can support a stable 1-to-2 pairing connection, that is, simultaneously establish communication connections with the receivers corresponding to two dexterous hands, thereby enabling the wearer to perform synchronous remote operation of the two dexterous hands by wearing two remote gloves on the left and right hands respectively.

[0108] According to the dexterous hand control method provided in the embodiments of this application, the data transmission link between the remote glove and the receiver is constructed using the ESP-NOW wireless communication protocol, which realizes data transmission with low latency (e.g., less than 10ms). Furthermore, through the one-to-two pairing connection supported by the protocol, the synchronous remote operation of the two dexterous hands is realized, thereby improving the real-time performance and fidelity of the dexterous hand in reproducing hand movements.

[0109] The dexterous hand control method provided in this application can be executed by a dexterous hand control device. This application uses the example of a dexterous hand control device executing the dexterous hand control method to illustrate the dexterous hand control device provided in this application.

[0110] This application also provides a dexterous hand control device.

[0111] like Figure 2 As shown, the dexterous hand control device includes: a first processing module 210 and a second processing module 220.

[0112] The first processing module 210 is used to acquire the target pose data of the remote control glove; The second processing module 220 is used to control the operation of the dexterous hand based on the target pose data and the preset mapping relationship. The preset mapping relationship is used to characterize the motion mapping between the joints of the dexterous hand and the joints of the remote control glove. The preset mapping relationship is constructed based on the calibration pose data of the remote control glove in the case of single finger movement and multi-finger relative movement.

[0113] According to the dexterous hand control device provided in the embodiments of this application, target pose data is acquired by setting magnetic encoders at the joints of each finger of the remote control glove. The high-resolution characteristics of the encoders enable high-precision capture of joint movements, improving the accuracy of angle positioning and effectively reducing pose drift during long-term use. At the same time, the preset mapping relationship is constructed based on the calibration pose data of the remote control glove under single-finger movement and multi-finger relative movement conditions. It can adapt to the individual hand movement range of different wearers and accurately reflect the relative positional relationship between each finger. Thus, the dexterous hand can accurately and stably reproduce the wearer's hand movements, improving the teleoperation accuracy and following stability of the dexterous hand.

[0114] In some embodiments, the second processing module 220 may also be used for: The calibration pose data under the basic single-finger movement condition are used to calibrate the motion constraints corresponding to each finger joint; Based on the calibration pose data under the condition of relative movement of multiple fingers, the calibration motion constraints are optimized, and a preset mapping relationship is constructed based on the optimized motion constraints.

[0115] In some embodiments, the second processing module 220 may also be used for: Based on the calibration pose data of single-finger movements, the motion constraints corresponding to each finger joint are calibrated, including: From the calibration pose data of single finger movement, obtain the angle data of each finger joint during finger movement; Based on the angle data, determine the extreme angle values ​​of each finger joint; Based on the extreme values ​​of the angles, the motion constraints corresponding to each finger joint are calibrated.

[0116] In some embodiments, the second processing module 220 may also be used for: Based on the optimized motion constraints and the motion posture of the dexterous hand during the execution of corresponding actions in the remote control glove, a preset mapping relationship is constructed.

[0117] In some embodiments, the second processing module 220 may also be used for: Obtain the roll level adjustment parameters; Based on the roll level adjustment parameters, the roll axis mapping of the remote control glove is adjusted when performing relative movements of multiple fingers, and the pose compensation data of each finger joint in the roll degree of freedom is determined. The motion constraints are updated and optimized based on the pose compensation data, and a preset mapping relationship is generated based on the compensated motion constraints.

[0118] In some embodiments, the second processing module 220 may also be used for: Based on the target pose data and the preset mapping relationship, the target joint angle data corresponding to the dexterous hand is obtained; Determine the target communication protocol that matches the device model of the dexterous hand, and configure the corresponding target operating mode; Based on the target communication protocol and in accordance with the target working mode, the target joint angle data is sent to the dexterous hand to drive the dexterous hand to perform the corresponding joint movements.

[0119] The dexterous hand control device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0120] The dexterous hand control device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0121] The dexterous hand control device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0122] This application provides a remote control glove that is communicatively connected to a dexterous hand.

[0123] In some embodiments, the remote control glove includes: a plurality of magnetic encoders and a dexterous hand control device based on any of the embodiments described above.

[0124] In this embodiment, multiple magnetic encoders are electrically connected to the dexterous hand control device.

[0125] Multiple magnetic encoders are respectively installed on the lateral movement, base, and fingertips of each finger of the remote control glove, corresponding to at least fifteen joint degrees of freedom. The magnetic encoders can be high-precision 16-bit magnetic encoders, with an angle capture accuracy of up to 0.1° for each finger joint.

[0126] In actual execution, multiple magnetic encoders detect the angle changes of each finger joint in three dimensions: lateral movement, root, and fingertip, and transmit the detected angle data to the dexterous hand control device. The dexterous hand control device executes the dexterous hand control method as described in any of the aforementioned embodiments based on the received angle data, thereby realizing the real-time reproduction of the wearer's hand movements by the dexterous hand.

[0127] The remote control glove provided in the embodiments of this application uses a 16-bit high-precision magnetic encoder to detect the angle of the hand joints, which helps to improve the accuracy and stability of the remote control glove in capturing the angle of each finger joint, and is suitable for application scenarios where the glove can work continuously for a long time.

[0128] In some embodiments, such as Figure 3 As shown, this application embodiment also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the program is executed by the processor 301, it implements the various processes of the above-described dexterous hand control method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0129] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0130] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described dexterous hand control method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0131] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described dexterous hand control method.

[0133] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0134] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described dexterous hand control method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0135] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0136] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0137] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0138] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0139] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0140] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for controlling a dexterous hand, characterized in that, The dexterous hand is communicatively connected to the remote control glove, the remote control glove including multiple magnetic encoders respectively disposed at each finger joint of the remote control glove, the method comprising: Obtain the target pose data of the remote control glove; Based on the target pose data and the preset mapping relationship, the dexterous hand is controlled to operate; wherein... The preset mapping relationship is used to characterize the motion mapping between the joints of the dexterous hand and the joints of the teleoperated glove. The preset mapping relationship is constructed based on the calibration pose data of the teleoperated glove in single-finger movement and multi-finger relative movement. The calibration pose data under the relative movement of multiple fingers includes: the pose of each finger joint when any two fingers perform a finger-opposing action; wherein, the thumb of the remote control glove is sequentially made to perform a finger-opposing action with the index finger, middle finger, ring finger, and little finger to establish multiple corresponding finger-opposing anchor points, and the multiple finger-opposing anchor points are used as the calibration pose data under the relative movement of multiple fingers. The step of controlling the dexterous hand based on the target pose data and the preset mapping relationship includes: controlling the dexterous hand based on the finger-attraction algorithm; the finger-attraction algorithm includes: attraction threshold mechanism, soft attraction potential function mechanism, hysteresis locking mechanism and velocity gating mechanism; The soft attraction potential function includes: When it is determined that the current frame has entered the attraction activation state, the linear mapping pose of the telescopic glove is obtained by conventional linear mapping calculation in the current frame; Calculate the corresponding smoothing coefficient based on the spatial pose deviation parameters of the current frame; The linearly mapped pose and the corresponding finger anchor point are combined using a convex combination based on a smoothing coefficient to generate and output the smoothly transitioned mapped pose. The soft attraction potential function is expressed by the formula: Output(P) = ( 1 - λ(d_k) ) · M_linear(P) + λ(d_k) · A_k; λ(d) = 3·t² - 2·t³; The formula for calculating the intermediate variable t is as follows: t = clip( (d_out - d) / (d_out - d_in), 0, 1 ); Wherein, Output(P) represents the mapped output pose; λ(d_k) represents the smoothing coefficient; M_linear(P) represents the linear mapped pose corresponding to the current posture of the telescopic glove; A_k represents the finger anchor point; d_k represents the spatial pose deviation parameter; d_in represents the preset first distance threshold; d_out represents the preset second distance threshold; and clip represents the numerical limiting function. The speed gating mechanism includes: Calculate the rate of change of posture of the corresponding fingers of the remote control glove; Based on the comparison between the attitude change rate and the preset speed threshold, and in conjunction with the preset time stability condition, the system state switching command is intercepted or allowed.

2. The dexterous hand control method according to claim 1, characterized in that, The preset mapping relationship is constructed based on the calibration pose data of the remote control glove under single-finger movement and multi-finger relative movement conditions, and includes: Based on the calibration pose data under the single-finger movement, the motion constraints corresponding to each finger joint are calibrated; Based on the calibration pose data under the relative motion of the multiple fingers, the calibration motion constraints are optimized, and based on the optimized motion constraints, the preset mapping relationship is constructed.

3. The dexterous hand control method according to claim 2, characterized in that, The calibration of motion constraints corresponding to each finger joint based on the calibration pose data under the single-finger movement condition includes: From the calibrated pose data under the single-finger movement condition, obtain the angle data of each finger joint during flexion and extension activities; The angle data are statistically analyzed to determine the first and second range of motion of each finger joint. Based on the first and second movement angles of each finger joint, a corresponding normalization function is constructed, and the normalization function is used as the motion constraint for each finger joint.

4. The dexterous hand control method according to claim 2, characterized in that, The process of optimizing the calibrated motion constraints based on the calibration pose data under the relative motion of the multiple fingers, and constructing the preset mapping relationship based on the optimized motion constraints, includes: Based on the optimized motion constraints and the motion posture of the dexterous hand during the execution of the corresponding action by the remote control glove, the preset mapping relationship is constructed.

5. The dexterous hand control method according to claim 2, characterized in that, The process of optimizing the calibrated motion constraints based on the calibration pose data under the relative motion of the multiple fingers, and constructing the preset mapping relationship based on the optimized motion constraints, includes: Obtain the roll level adjustment parameters; Based on the roll level adjustment parameters, the roll axis mapping of the remote control glove is adjusted when performing relative movements of multiple fingers, and the pose compensation data of the target finger in the roll degree of freedom is determined. The optimized motion constraints are updated based on the pose compensation data, and the preset mapping relationship is generated based on the compensated motion constraints.

6. The dexterous hand control method according to any one of claims 1-5, characterized in that, The step of controlling the dexterous hand based on the target pose data and a preset mapping relationship includes: Based on the target pose data and the preset mapping relationship, the target joint angle data corresponding to the dexterous hand is obtained; Determine the target communication protocol that matches the device model of the dexterous hand, and configure the corresponding target operating mode; Based on the target communication protocol and in accordance with the target working mode, the target joint angle data is sent to the dexterous hand to drive the dexterous hand to perform the corresponding joint movements.

7. A dexterous hand control device, characterized in that, The dexterous hand is communicatively connected to the remote control glove, the remote control glove including multiple magnetic encoders respectively disposed at each finger joint of the remote control glove, the device including: The first processing module is used to acquire the target pose data of the remote control glove; The second processing module is used to control the operation of the dexterous hand based on the target pose data and a preset mapping relationship. The preset mapping relationship characterizes the motion mapping between the joints of the dexterous hand and the joints of the teleoperated glove. This preset mapping relationship is constructed based on the calibration pose data of the teleoperated glove under single-finger movement and multi-finger relative movement conditions. The calibration pose data under multi-finger relative movement conditions includes the pose of each finger joint when any two fingers perform a finger-opposing action. Specifically, the thumb of the teleoperated glove is sequentially made to perform finger-opposing actions with the index, middle, ring, and little fingers to establish multiple corresponding finger-opposing anchor points, and these multiple finger-opposing anchor points are then... Points serve as calibration pose data for relative movements of multiple fingers; they are also used to control the dexterous hand's operation based on a finger-attachment algorithm; the finger-attachment algorithm includes: an attraction threshold band mechanism, a soft attraction potential function mechanism, a hysteresis locking mechanism, and a velocity gating mechanism; it is also used to obtain the linearly mapped pose of the teleoperated hand calculated using conventional linear mapping in the current frame when the current frame is determined to be in an attraction activation state; based on the spatial pose deviation parameters of the current frame, the corresponding smoothing coefficient is calculated; the linearly mapped pose and the corresponding finger-attachment anchor points are subjected to convex combination processing based on the smoothing coefficient to generate and output the smoothly transitioned mapped output pose; the soft attraction potential function is expressed by the formula: Output(P) = (1 - λ(d_k)) · M_linear(P) + λ(d_k) · A_k; λ(d) = 3·t² - 2·t³; The formula for calculating the intermediate variable t is: t = clip((d_out - d) / (d_out - d_in), 0, 1); where Output(P) represents the mapped output pose; λ(d_k) represents the smoothing coefficient; M_linear(P) represents the linear mapped pose corresponding to the current pose of the telescopic glove; A_k represents the finger anchor point; d_k represents the spatial pose deviation parameter; d_in represents the preset first distance threshold; d_out represents the preset second distance threshold; clip represents the numerical limiting function; it is also used to calculate the pose change rate of the corresponding finger of the telescopic glove; based on the comparison result of the pose change rate and the preset speed threshold, and combined with the preset time stability condition, the system state switching command is intercepted or allowed.

8. A remote-controlled glove, characterized in that, The remote-controlled glove is communicatively connected to a dexterous hand, and the remote-controlled glove includes: Multiple magnetic encoders; the multiple magnetic encoders are respectively disposed on the lateral swing, root and fingertip of each finger of the remote control glove, corresponding to at least fifteen joint degrees of freedom; The dexterous hand control device as described in claim 7, wherein the plurality of magnetic encoders are electrically connected to the dexterous hand control device.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dexterous hand control method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the dexterous hand control method as described in any one of claims 1-6.

Images