A hand part pose determination method and device, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, electronic device, and storage medium for determining hand pose, relating to the field of robotics. The method is applied to an exoskeleton glove. The exoskeleton glove is equipped with a segmented finger joint self-alignment rail. The wearer aligns their own fingers with the exoskeleton glove's fingers by adjusting the position of a slider in the segmented finger joint self-alignment rail. The method includes: receiving first angle information collected by an angle information acquisition unit configured in the exoskeleton glove's hand joints; and substituting the first angle information into a pre-constructed wearer's hand kinematic model to obtain a first joint pose of the wearer's hand joints. The wearer's hand kinematic model is constructed based on the wearer's personalized hand information, which characterizes the wearer's actual hand parameters. Applying the solution provided in this application can improve the accuracy of determining the hand pose of the exoskeleton glove wearer.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a method, apparatus, electronic device, and storage medium for determining hand pose. Background Technology

[0002] Hand exoskeleton gloves have significant application value in fields such as dexterous hand teleoperation in robots. For example, in robot teleoperation scenarios, exoskeleton gloves can map the wearer's fine hand movements into dexterous hand movements through motion mapping, thereby enabling the dexterous hand to perform corresponding operations.

[0003] However, the actual hand parameters, such as the length of the joint bones and the width of the palm, vary among different people. This makes it difficult for exoskeleton gloves to accurately determine the actual hand position of the wearer based on the joint angle, which in turn affects the execution of downstream tasks. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, electronic device, and storage medium for determining hand pose, so as to improve the accuracy of the determined hand pose of the exoskeleton glove wearer. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide a hand pose determination method applied to an exoskeleton glove. The exoskeleton glove is equipped with a segmented knuckle self-alignment slide rail. The wearer aligns their own knuckles with the knuckles of the exoskeleton glove by adjusting the position of the slider in the segmented knuckle self-alignment slide rail. The method includes:

[0006] Receive the first angle information collected by the angle information acquisition unit configured in the hand joints of the exoskeleton glove;

[0007] Substituting the first angle information into the constructed hand kinematic model of the wearer, the first joint pose of the wearer's hand joint is obtained, wherein the hand kinematic model is constructed based on the wearer's personalized hand information, and the personalized hand information represents the wearer's actual hand parameters.

[0008] In one possible implementation, the hand personalization information is determined in the following manner:

[0009] Receive second angle information collected by the angle information acquisition unit under multiple different hand postures of the wearer;

[0010] Based on the constructed exoskeleton kinematic model, forward kinematics calculation is performed on the second angle information to obtain the back of the hand and finger bone posture of the wearer's hand under different hand postures;

[0011] Based on the back of the hand pose and the finger bone pose, the wearer's actual hand parameters are generated as personalized hand information.

[0012] In one possible implementation, before performing forward kinematic calculation on the second angle information based on the constructed exoskeleton kinematic model, the method further includes:

[0013] Obtain the slider position data of the segmented knuckle self-aligning slide rail;

[0014] Based on the slider position data, the model parameters of the exoskeleton kinematic model are corrected.

[0015] In one possible implementation, generating the wearer's actual hand parameters based on the back-of-hand pose and the finger bone pose includes:

[0016] Based on the back of the hand posture and the finger bone posture, the initial hand parameters of the wearer are determined;

[0017] The initial hand parameters are corrected based on prior parameters of the human hand skeleton to obtain the wearer's actual hand parameters.

[0018] In one possible implementation, the actual hand parameters include at least one of the following hand parameters of the wearer:

[0019] The position of the hand joints, the length of the joint bones, the width of the palm, the relative positional relationship between the bases of the fingers, and the relative positional relationship between the fingertips and the wrist.

[0020] In one possible implementation, the multiple sets of different hand gestures include at least two of the following:

[0021] Finger straight posture, finger joint posture when bent, and finger clenched fist posture.

[0022] In one possible implementation, the method further includes:

[0023] The first joint pose is mapped to the second joint pose of the dexterous hand to be manipulated, and the second joint pose is sent to the dexterous hand to manipulate the dexterous hand to reach the second joint pose.

[0024] Secondly, embodiments of this application provide a hand pose determination device applied to an exoskeleton glove. The exoskeleton glove is equipped with a segmented knuckle self-alignment slide rail. The wearer aligns their own knuckles with the knuckles of the exoskeleton glove by adjusting the position of the slider in the segmented knuckle self-alignment slide rail. The device includes:

[0025] An angle information receiving module is used to receive the first angle information collected by the angle information acquisition unit configured on the hand joint of the exoskeleton glove;

[0026] The joint pose determination module is used to substitute the first angle information into the constructed kinematic model of the wearer's hand to obtain the first joint pose of the wearer's hand joints, wherein the kinematic model of the hand is constructed based on the personalized information of the wearer's hand, and the personalized information of the hand represents the actual hand parameters of the wearer.

[0027] In one possible implementation, the hand personalization information is determined according to the following modules:

[0028] The parameter calibration module is used to receive the second angle information collected by the angle information acquisition unit under multiple different hand postures of the wearer; based on the constructed exoskeleton kinematic model, the module performs forward kinematic calculation on the second angle information to obtain the back of the hand pose and finger bone pose of the wearer's hand under the different hand postures; based on the back of the hand pose and finger bone pose, the module generates the wearer's actual hand parameters as personalized hand information.

[0029] In one possible implementation, the parameter calibration module is specifically used to obtain the slider position data of the segmented knuckle self-aligning slide rail before performing forward kinematic calculation on the second angle information based on the constructed exoskeleton kinematic model; and to correct the model parameters of the exoskeleton kinematic model based on the slider position data.

[0030] In one possible implementation, the parameter calibration module is specifically used to determine the wearer's initial hand parameters based on the back of the hand pose and finger bone pose; and to correct the initial hand parameters based on prior parameters of the human hand skeleton to obtain the wearer's actual hand parameters.

[0031] In one possible implementation, the actual hand parameters include at least one of the following hand parameters of the wearer:

[0032] The position of the hand joints, the length of the joint bones, the width of the palm, the relative positional relationship between the bases of the fingers, and the relative positional relationship between the fingertips and the wrist.

[0033] In one possible implementation, the multiple sets of different hand gestures include at least two of the following:

[0034] Finger straight posture, finger joint posture when bent, and finger clenched fist posture.

[0035] In one possible implementation, the device further includes:

[0036] The pose mapping module is used to map the first joint pose to the second joint pose of the dexterous hand to be manipulated, and send the second joint pose to the dexterous hand to manipulate the dexterous hand to achieve the second joint pose.

[0037] Thirdly, embodiments of the present invention provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0038] Memory, used to store computer programs;

[0039] When a processor executes a program stored in memory, it implements the steps of the method described in the first aspect.

[0040] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0041] Fifthly, embodiments of the present invention provide a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the method described in the first aspect.

[0042] Beneficial effects of the embodiments in this application:

[0043] As can be seen from the above, the solution provided in this application uses a segmented finger joint self-alignment rail to achieve physical alignment between the exoskeleton and the human finger joints during wear. After alignment, the wearer's personalized hand information is pre-calibrated, and a hand kinematic model is constructed based on this personalized information. Thus, after the wearer actually wears the exoskeleton glove, the exoskeleton glove acquires joint angles in real time, which are then substituted into the wearer's hand kinematic model constructed based on the personalized hand information to calculate the wearer's true joint pose. It is evident that by using a rail connection to synchronize the exoskeleton and human hand joint positions and specifically calibrating the wearer's individualized hand information, the impact of individual hand shape differences on determining hand pose is reduced, thereby improving the accuracy of the determined hand pose of the wearer.

[0044] Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0046] Figure 1 This is a schematic diagram illustrating one application scenario of an embodiment of this application;

[0047] Figure 2 A flowchart illustrating a hand pose determination method provided in an embodiment of this application;

[0048] Figure 3 A schematic diagram illustrating a personalized hand information calibration scenario provided in an embodiment of this application;

[0049] Figure 4 This is a schematic diagram of a hand pose determination device provided in an embodiment of this application;

[0050] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0052] First, the application scenarios of the embodiments of this application will be introduced.

[0053] The solution provided in this application is applied to the scenario of dexterous hands of exoskeleton gloves for remote operation robots, specifically including scenarios such as embodied intelligent dexterous operation data acquisition and precision operations in high-risk environments.

[0054] Among them, such as Figure 1 As shown, the exoskeleton glove is worn on the operator's hand. It collects hand movement information through joint sensors, and after the processing unit completes the pose calculation and retargeting mapping, it controls the dexterous hand to perform corresponding actions synchronously.

[0055] It should be noted that the above figures are for illustrative purposes only and do not limit the specific structure of exoskeleton gloves or dexterous hands.

[0056] In order to improve the accuracy of the determined hand position of the exoskeleton glove wearer, and thus improve the operational precision of the dexterous hand, this application provides a hand position determination scheme.

[0057] In the solution provided in this application embodiment, the exoskeleton glove is equipped with a segmented knuckle self-alignment slide rail. The self-alignment slide rail includes a slider. The passive movement of the slider eliminates the shearing force caused by the misalignment of the wearer's finger midline and the exoskeleton midline, thereby achieving self-alignment. Before implementing this solution, the wearer adjusts the position of the slider in the segmented knuckle self-alignment slide rail so that the base of each exoskeleton finger is aligned with the midpoint of their own finger joint and then locked in place. At this time, the exoskeleton connecting rods are fixedly connected to the corresponding finger bones of the wearer, and the rotation angle of the exoskeleton joints is synchronized with the rotation angle of the corresponding joints in the human body.

[0058] See Figure 2 This is a flowchart illustrating a hand pose determination method provided in an embodiment of this application. The method includes the following steps S201-S202:

[0059] Step S201: Receive the first angle information collected by the angle information acquisition unit of the hand joint configuration of the exoskeleton glove.

[0060] The angle information acquisition unit will be described below:

[0061] For fingers other than the thumb, the angle information acquisition unit may include encoders deployed at each joint, covering the flexion-extension and lateral swing degrees of freedom of the metacarpophalangeal joint (MCP), the flexion-extension degrees of freedom of the proximal interphalangeal joint (PIP), and the flexion-extension degrees of freedom of the distal interphalangeal joint (DIP). Preferably, the MCP joints of the fingers other than the thumb may consist of two approximately orthogonal joints, namely the extension / flexion (E / F) joint and the lateral swing / adduction / abduction (A / A) joint, which may be referred to as the MCP E / F joint and the MCP A / A joint, respectively; the PIP and DIP joints of the aforementioned fingers may each only include the E / F joint.

[0062] For the thumb, the angle information acquisition unit may include encoders deployed at each joint, covering multiple degrees of freedom of the thumb carpometacarpal joint (CMC), and corresponding degrees of freedom of the MCP and interphalangeal joints (IP). Preferably, the thumb CMC joint may consist of three approximately orthogonal joints: the E / F joint, the A / A joint, and the axial rotation / opposition rotation (Rot / Opp joint), which can be referred to as the CMC E / F joint, the CMC A / A joint, and the CMC Rot / Opp joint, respectively; the thumb MCP joint may consist of two approximately orthogonal joints: the E / F joint and the A / A joint, which can be referred to as the MCP E / F joint and the MCP A / A joint, respectively; the thumb IP joint may consist of only the E / F joint.

[0063] In this way, the angle information acquisition unit can collect the rotation angle data of each joint in real time.

[0064] Step S202: Substitute the first angle information into the constructed wearer's hand kinematic model to obtain the first joint pose of the wearer's hand joints.

[0065] Among them, the wearer's hand kinematic model is constructed based on the wearer's personalized hand information, which represents the wearer's actual hand parameters.

[0066] Specifically, the wearer's hand kinematic model is a geometric mapping model of hand movement specifically created for the wearer, unlike a general standardized human hand model. The geometric constants included in the model (length of each bone segment, coordinates of each joint center, relative joint offset, palm width, finger root spacing, etc.) are all matched to the wearer's actual hand physiological dimensions, thereby eliminating the position calculation deviation caused by different hand shapes and fitting offsets, and achieving accurate conversion from the mechanical perspective of the exoskeleton to the actual human hand posture.

[0067] The following is a brief introduction to the construction method of the hand kinematic model.

[0068] The operator wears an exoskeleton glove and adjusts the slider position of the sliding segmented knuckle self-alignment rail to align and lock the base of each exoskeleton with their own knuckles. The operator is guided to sequentially perform multiple sets of standard hand postures, including finger extension at zero position, independent bending of a single joint, and full fist clenching. Angle data of all joints output by the exoskeleton are collected for each posture. The joint angles of each set are input into the exoskeleton kinematic model, and the corresponding back of the hand and finger bone spatial poses are obtained through forward kinematics calculation. Based on multiple sets of pose data, an overdetermined set of equations is established to generate personalized information (human factors parameters) of the operator's hand. All the obtained human factors parameters (joint position, length of each bone segment, palm width, relative position of finger root / finger tip, etc.) are filled into a general human hand kinematic topology framework to generate a unique kinematic model of the wearer's hand.

[0069] The exoskeleton kinematic model is a fixed motion model describing the geometric topology of the glove's mechanical links, joints, and sliders. It only represents the structure of the exoskeleton hardware itself and does not involve the physiological dimensions of the human hand. It can be pre-built and will not be elaborated here. The input to the exoskeleton kinematic model is the rotation angle of each joint of the glove and the displacement of the slider. The output is the three-dimensional spatial pose of each mechanical link of the exoskeleton.

[0070] Next, we will explain how to determine the personalized information on the wearer's hands.

[0071] To improve the accuracy and efficiency of determining personalized hand information, one possible implementation involves receiving second angle information collected by multiple angle information acquisition units under different hand postures of the wearer; based on a pre-built exoskeleton kinematic model, forward kinematics calculation is performed on the second angle information to obtain the back-of-hand pose and finger bone pose of the wearer's hand under different hand postures; based on the back-of-hand pose and finger bone pose, the wearer's actual hand parameters are generated as personalized hand information.

[0072] Specifically, in calibration mode, the wearer can be guided to complete multiple sets of standard hand postures sequentially, simultaneously acquiring the second angle information output by the angle information acquisition unit under each posture, as a set of hand postures. Then, for each set of postures, the second angle information is substituted into the exoskeleton kinematics model for forward kinematics (FK) calculation to obtain the spatial pose corresponding to each link of the exoskeleton. Since the exoskeleton's back-of-hand base and knuckle base are fixed to the wearer's back of hand and finger bones respectively, the exoskeleton link poses can directly correspond to the wearer's back-of-hand pose and finger bone pose.

[0073] The following describes how to generate the wearer's actual hand parameters.

[0074] Based on multiple sets of hand back poses and phalanx poses under different postures, parameters such as the rotation center position of each joint and the length of each phalanx can be solved through geometric fitting to obtain the wearer's actual hand parameters, that is, to obtain the wearer's personalized hand information. It can be seen that the above calibration process does not require external motion capture equipment, and can complete the individualized parameter calibration solely based on the exoskeleton's own sensing.

[0075] Another possible implementation involves using optical motion capture markers to mark the positions of the wearer's hand joints, and then using a motion capture camera to analyze the pose of these markers to obtain personalized information about the operator's hand. This method relies on additional motion capture equipment and is suitable for scenarios requiring high precision.

[0076] Next, we will introduce the method for determining the position of the first joint.

[0077] The constructed wearer hand kinematic model is an individualized human hand kinematic model built based on the personalized information of the wearer's hand obtained through pre-calibration. The model parameters correspond one-to-one with the wearer's real hand physiological structure, which is different from the general standard hand model.

[0078] The first angle information collected in real time is substituted into the individualized kinematic model, and the spatial position and posture of each joint and finger bone are derived step by step through homogeneous coordinate transformation. Finally, the complete pose information of the wearer's hand joints, such as the three-dimensional position and rotation posture, is obtained, namely the first joint pose.

[0079] As can be seen from the above, the solution provided in this application uses a segmented finger joint self-alignment rail to achieve physical alignment between the exoskeleton and the human finger joints during wear. After alignment, the wearer's personalized hand information is pre-calibrated, and a hand kinematic model is constructed based on this personalized information. Thus, after the wearer actually wears the exoskeleton glove, the exoskeleton glove acquires joint angles in real time, which are then substituted into the wearer's hand kinematic model constructed based on the personalized hand information to calculate the wearer's true joint pose. It is evident that by using a rail connection to synchronize the exoskeleton and human hand joint positions and specifically calibrating the wearer's individualized hand information, the impact of individual hand shape differences on determining hand pose is reduced, thereby improving the accuracy of the determined hand pose of the wearer.

[0080] In one possible implementation, to ensure that the pose calculation reference is consistent with the actual wearing state and to further improve the accuracy of the determined wearer's hand pose, the following steps can be performed before performing forward kinematics calculation on the second angle information:

[0081] Obtain the slider position data of the segmented knuckle self-aligning slide rail; based on the slider position data, correct the model parameters of the pre-built exoskeleton kinematic model.

[0082] Specifically, the segmented knuckle self-aligning slide rail is equipped with a displacement sensing unit that can output real-time position data of the slider. After being put on and locked, the slider position remains fixed, and its value directly determines the initial length of each link of the exoskeleton, the offset of the joint base, and other geometric parameters.

[0083] Before performing forward kinematics calculations, the exoskeleton glove first reads the slider position data, writes the geometric offset corresponding to the slider position into the default exoskeleton kinematics model, and corrects the fixed structural parameters in the model, such as the link length and the relative position of the joints, so that the exoskeleton kinematics model is completely matched with the actual mechanical structure under the current wearing state, ensuring the reference accuracy of subsequent forward kinematics calculations.

[0084] It should be noted that the slider position data is only used for the initialization correction of the exoskeleton model, and will not be read and iterated again during the human factor parameter calibration and fitting stage.

[0085] To eliminate errors in the process of determining personalized hand information and improve the rationality and accuracy of personalized hand information, one possible approach is to introduce prior parameters of the human hand skeleton to constrain the generation process of human factors parameters.

[0086] Specifically, the wearer's initial hand parameters can be determined based on the back of the hand and finger bone positions; then, the initial hand parameters are corrected based on the prior parameters of the human hand skeleton to obtain the wearer's actual hand parameters.

[0087] In this process, the initial hand parameters can be substituted into the prior constraints for optimization, outlier solutions that exceed the reasonable physiological range can be eliminated, the fitting bias can be corrected, and finally the actual hand parameters that conform to the human physiological structure can be obtained, thereby further improving the accuracy and rationality of personalized hand information.

[0088] To improve the matching degree between the constructed individualized human hand kinematic model and the actual human hand, and to improve the accuracy of subsequent pose calculation, in one possible implementation, the actual hand parameters may include at least one of the following: the wearer's hand joint position, joint bone length, palm width, relative positional relationship between the finger roots, and relative positional relationship between the fingertips and the wrist.

[0089] The specific parameters included in the actual hand parameters can be set according to the actual needs of the scenario, and this application embodiment does not limit this.

[0090] For example, for simple grasping tasks that only use the index finger and thumb, only the position of the thumb and index finger joints and the length of the joint bones can be determined, while the palm width and the relative position data of the other fingers can be discarded, in order to reduce the amount of computation and adapt to low-cost embedded devices.

[0091] Preferably, the actual hand parameters may include all of the above parameters. In this way, the above parameters together constitute the complete geometric input of the individualized human hand kinematic model, improving the matching degree between the constructed human hand kinematic model and the wearer's actual hand.

[0092] In order to balance the accuracy and efficiency of personalized hand information calibration, in one possible implementation, multiple different hand postures include at least two of the following: finger extended posture, finger bent joint posture, and finger clenched fist posture.

[0093] Among them, multiple sets of different hand postures may include any one of the following: finger straight posture, finger bent joint posture, and finger clenched fist posture; or, multiple sets of different hand postures may include all three of the above postures.

[0094] like Figure 3 As shown, the wearer can first assume a finger-extended posture, at which point the exoskeleton glove collects angle information in this posture; then, the wearer assumes a fist-clenched posture, at which point the exoskeleton glove collects angle information in this posture. Based on these two sets of angle information, personalized hand information of the wearer can be calculated.

[0095] Specifically, the extended finger posture serves as the zero-position reference for motion, used to determine the initial angle origin of each joint and unify the solution coordinate system; the joint postures of the bent finger, including the hand postures of single-joint and multi-joint bending, are used to locate the rotation center and rotation axis direction of the corresponding joints, and are the core constraint postures for solving the joint position and joint bone length; the clenched finger posture is used to verify and correct the parameters obtained by fitting single joints, improving the overall accuracy of parameter solution.

[0096] To ensure that the solution can be directly applied to various remote operation scenarios and to guarantee the integrity and executability of the entire remote operation process, one possible implementation is to map the first joint pose to the second joint pose of the dexterous hand, and send the second joint pose to the dexterous hand to be controlled, so as to control the dexterous hand to achieve the second joint pose.

[0097] Specifically, after obtaining the wearer's first joint pose, a motion redirection algorithm is used to complete the cross-morphological mapping from the human hand to the dexterous hand, generating a second joint pose adapted to the dexterous hand's mechanical structure. The second joint pose is then converted into control commands recognizable by the dexterous hand's drive unit and sent to the slave dexterous hand via the communication bus, driving the dexterous hand to move to the corresponding pose, thus completing the teleoperation replication of the human hand's movements. This redirection and sending process can be implemented using conventional motion mapping methods in the field, and will not be elaborated upon in this embodiment.

[0098] Corresponding to the above-described hand pose determination method, this application also provides a hand pose determination device.

[0099] See Figure 4 This is a schematic diagram of a hand pose determination device provided in an embodiment of this application, applied to an exoskeleton glove. The device includes:

[0100] Angle information receiving module 401 is used to receive the first angle information collected by the angle information acquisition unit configured in the hand joint of the exoskeleton glove;

[0101] The joint pose determination module 402 is used to substitute the first angle information into the constructed wearer's hand kinematic model to obtain the first joint pose of the wearer's hand joint. The wearer's hand kinematic model is constructed based on the wearer's personalized hand information. The personalized hand information represents the wearer's actual hand parameters and is used to receive the first angle information collected by the angle information acquisition unit configured in the exoskeleton glove's hand joint.

[0102] As can be seen from the above, the solution provided in this application uses a segmented finger joint self-alignment rail to achieve physical alignment between the exoskeleton and the human finger joints during wear. After alignment, the wearer's personalized hand information is pre-calibrated, and a hand kinematic model is constructed based on this personalized information. Thus, after the wearer actually wears the exoskeleton glove, the exoskeleton glove acquires joint angles in real time, which are then substituted into the wearer's hand kinematic model constructed based on the personalized hand information to calculate the wearer's true joint pose. It is evident that by using a rail connection to synchronize the exoskeleton and human hand joint positions and specifically calibrating the wearer's individualized hand information, the impact of individual hand shape differences on determining hand pose is reduced, thereby improving the accuracy of the determined hand pose of the wearer.

[0103] In one possible implementation, the hand personalization information is determined in the following manner:

[0104] Receive second angle information collected by the angle information acquisition unit under multiple different hand postures of the wearer;

[0105] Based on the constructed exoskeleton kinematic model, forward kinematics calculation is performed on the second angle information to obtain the back of the hand and finger bone posture of the wearer's hand under different hand postures;

[0106] Based on the back of the hand pose and finger bone pose, the wearer's actual hand parameters are generated as personalized hand information.

[0107] This can improve the accuracy and efficiency of determining personalized hand information.

[0108] In one possible implementation, before performing forward kinematic calculation on the second angle information based on the pre-built exoskeleton kinematic model, the method further includes:

[0109] Obtain the slider position data of the segmented knuckle self-aligning slide rail;

[0110] Based on the slider position data, the model parameters of the exoskeleton kinematic model are corrected.

[0111] This ensures that the pose calculation benchmark is consistent with the actual wearing state, further improving the accuracy of the determined wearer's hand pose.

[0112] In one possible implementation, generating the wearer's actual hand parameters based on the back-of-hand pose and finger bone pose includes:

[0113] Based on the back of the hand posture and finger bone posture, the initial hand parameters of the wearer are determined;

[0114] The initial hand parameters are corrected based on prior parameters of the human hand skeleton to obtain the wearer's actual hand parameters.

[0115] This can eliminate errors in the process of determining personalized hand information and improve the rationality and accuracy of personalized hand information.

[0116] In one possible implementation, the actual hand parameters include at least one of the following hand parameters of the wearer:

[0117] The position of the hand joints, the length of the joint bones, the width of the palm, the relative positional relationship between the bases of the fingers, and the relative positional relationship between the fingertips and the wrist.

[0118] This can improve the matching degree between the constructed individualized human hand kinematic model and the actual human hand, and improve the accuracy of subsequent pose calculation.

[0119] In one possible implementation, the multiple sets of different hand gestures include at least two of the following:

[0120] Finger straight posture, finger joint posture when bent, and finger clenched fist posture.

[0121] This approach balances the accuracy and efficiency of personalized hand information calibration.

[0122] In one possible implementation, the method further includes:

[0123] The first joint hand position is determined as the second joint position of the dexterous hand, and the second joint position is sent to the dexterous hand to control the dexterous hand to reach the second joint position.

[0124] This allows the solution to be directly applied to various remote operation scenarios, ensuring the integrity and executability of the entire remote operation process.

[0125] Corresponding to the above-described method for determining hand pose, this application also provides an electronic device, a storage medium, and a program product.

[0126] This application also provides an electronic device, such as... Figure 5 As shown, the device includes a processor 501, a communication interface 502, a memory 503, and a communication bus 504. The processor 501, communication interface 502, and memory 503 communicate with each other via the communication bus 504. The memory 503 stores computer programs; the processor 501 executes the programs stored in the memory 503 to implement the aforementioned hand pose determination method. The electronic device can be a charging compartment device, a battery transfer device, or a robot. The communication bus mentioned in the electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the electronic device and other devices. The memory can include Random Access Memory (RAM) or Non-Volatile Memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor. The aforementioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0127] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described hand pose determination methods.

[0128] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the hand pose determination methods described in the above embodiments.

[0129] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0130] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system, apparatus, electronic device, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0132] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for determining hand pose, characterized in that, An exoskeleton glove is provided with a segmented knuckle self-alignment slide rail. The wearer aligns their knuckles with the knuckles of the exoskeleton glove by adjusting the position of the slider in the segmented knuckle self-alignment slide rail. The method includes: Receive the first angle information collected by the angle information acquisition unit configured in the hand joints of the exoskeleton glove; Substituting the first angle information into the constructed hand kinematic model of the wearer, the first joint pose of the wearer's hand joint is obtained, wherein the hand kinematic model is constructed based on the wearer's personalized hand information, and the personalized hand information represents the wearer's actual hand parameters.

2. The method according to claim 1, characterized in that, The personalized hand information is determined in the following manner: Receive second angle information collected by the angle information acquisition unit under multiple different hand postures of the wearer; Based on the constructed exoskeleton kinematic model, forward kinematics calculation is performed on the second angle information to obtain the back of the hand and finger bone posture of the wearer's hand under different hand postures; Based on the back of the hand pose and the finger bone pose, the wearer's actual hand parameters are generated as personalized hand information.

3. The method according to claim 2, characterized in that, Before performing forward kinematics calculation on the second angle information based on the constructed exoskeleton kinematic model, the method further includes: Obtain the slider position data of the segmented knuckle self-aligning slide rail; Based on the slider position data, the model parameters of the exoskeleton kinematic model are corrected.

4. The method according to claim 2 or 3, characterized in that, The process of generating the wearer's actual hand parameters based on the back-of-hand pose and the finger bone pose includes: Based on the back of the hand posture and the finger bone posture, the initial hand parameters of the wearer are determined; The initial hand parameters are corrected based on prior parameters of the human hand skeleton to obtain the wearer's actual hand parameters.

5. The method according to any one of claims 1 to 4, characterized in that, The actual hand parameters include at least one of the following hand parameters of the wearer: The position of the hand joints, the length of the joint bones, the width of the palm, the relative positional relationship between the bases of the fingers, and the relative positional relationship between the fingertips and the wrist.

6. The method according to any one of claims 2 to 4, characterized in that, The multiple sets of different hand gestures include at least two of the following: Finger straight posture, finger joint posture when bent, and finger clenched fist posture.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The first joint pose is mapped to the second joint pose of the dexterous hand to be manipulated, and the second joint pose is sent to the dexterous hand to manipulate the dexterous hand to reach the second joint pose.

8. A hand position posture determination device, characterized in that, An exoskeleton glove is provided, wherein the exoskeleton glove is equipped with a segmented knuckle self-alignment slide rail. The wearer aligns his / her knuckles with the knuckles of the exoskeleton glove by adjusting the position of the slider in the segmented knuckle self-alignment slide rail. The device includes: An angle information receiving module is used to receive the first angle information collected by the angle information acquisition unit configured on the glove hand joint of the exoskeleton; The joint pose determination module is used to substitute the first angle information into the constructed kinematic model of the wearer's hand to obtain the first joint pose of the wearer's hand joints, wherein the kinematic model of the hand is constructed based on the personalized information of the wearer's hand, and the personalized information of the hand represents the actual hand parameters of the wearer.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, a communication bus, and an image acquisition device. The processor, communication interface, and memory communicate with each other through the communication bus. Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1 to 7.

Images