Biomechanical modeling and normalized training data generation method, device and equipment for hand posture estimation and medium

Abstract

This application relates to a method, apparatus, device, and medium for biomechanical modeling and normalized training data generation for human hand posture estimation. The method includes: acquiring target three-dimensional hand posture data for each target object; for each target object, acquiring reference static biomechanical features and dynamic biomechanical features corresponding to each target three-dimensional hand posture; for each target three-dimensional hand posture, determining the three-dimensional joint coordinates and corresponding two-dimensional joint coordinates of the target three-dimensional hand posture based on the reference static and dynamic biomechanical features; wherein the three-dimensional joint coordinates represent the spatial position of the hand joints; and determining human hand posture training data based on each three-dimensional joint coordinate and its corresponding two-dimensional joint coordinate. This method can eliminate individual biomechanical structural differences and more accurately and reasonably expand the amount of training data.

Description

Technical Field

[0001] This application relates to the field of machine learning model technology, and in particular to a method, apparatus, computer device, and computer-readable storage medium for biomechanical modeling and normalized training data generation for human hand posture estimation. Background Technology

[0002] 3D gesture estimation technology has significant application prospects in fields such as virtual reality (VR), augmented reality (AR), human-computer interaction, and sign language recognition. In recent years, methods based on deep neural networks have become the mainstream research approach. However, the performance of these methods is highly dependent on the quality and distribution consistency of the training data.

[0003] Currently widely used 3D hand pose datasets suffer from significant differences in hand biomechanical features (such as bone length and palmar structure). Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer device, and computer-readable storage medium for biomechanical modeling and normalized training data generation for human hand posture estimation that can eliminate differences, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for biomechanical modeling and normalized training data generation for human hand posture estimation, the method comprising:

[0006] Obtain the 3D hand pose data of each target object;

[0007] For each target object, reference static biomechanical features and dynamic biomechanical features corresponding to each of the target object's three-dimensional hand poses are obtained; wherein, the reference static biomechanical features correspond one-to-one with each target object, and the reference static biomechanical features are determined by normalizing the static biomechanical features of each of the target object's three-dimensional hand poses; the static biomechanical features are used to characterize the palm features corresponding to the target three-dimensional hand pose; the dynamic biomechanical features are used to characterize the spatial angles of the fingers corresponding to the target three-dimensional hand pose.

[0008] For each target three-dimensional hand posture, based on the reference static biomechanical features and the dynamic biomechanical features, the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates are determined; wherein, the three-dimensional joint coordinates characterize the spatial position of the hand joints;

[0009] Based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates, human hand posture training data is determined.

[0010] In one embodiment, obtaining the reference static biomechanical features corresponding to each of the target object's three-dimensional hand poses includes:

[0011] Based on the target object's three-dimensional hand posture, calculate the static biomechanical characteristics of each target's three-dimensional hand posture; the static biomechanical characteristics include calcaneal length, the angle between calcaneal bones, and palmar surface curvature;

[0012] Based on the length of the calcaneus, the angle between the calcaneus, and the curvature of the palm surface of each of the three-dimensional hand postures of the target object, the reference static biomechanical characteristics corresponding to the target object are determined.

[0013] In one embodiment, calculating the static biomechanical characteristics of each of the target three-dimensional hand postures based on the target object's respective target three-dimensional hand postures includes:

[0014] For each of the target three-dimensional hand poses of the target object, obtain the root bone vectors of the palm corresponding to the target three-dimensional hand pose;

[0015] The Euclidean distance is calculated based on each of the root bone vectors to obtain the root bone length corresponding to the three-dimensional hand posture of the target.

[0016] The angle between the root vectors of adjacent root bones is calculated by using the inverse cosine function to obtain the angle between the root bones corresponding to the three-dimensional hand posture of the target; wherein, the number of angles between root bones is less than the number of root vectors;

[0017] Based on the root bone vectors of adjacent root bones, a basis vector is determined, and the palm curvature corresponding to the three-dimensional hand posture of the target is determined based on the basis vector; wherein, the basis vector represents the orthogonal vector of adjacent root bone vectors.

[0018] In one embodiment, obtaining the target object's three-dimensional hand poses includes:

[0019] Obtain the initial 3D hand poses of the target object;

[0020] For each initial three-dimensional hand pose, an orthogonal reference coordinate system corresponding to the initial three-dimensional hand pose is obtained; the orthogonal reference coordinate system is a coordinate system with the palm center joint of each initial three-dimensional hand pose as the origin and the palm normal vector as the target direction;

[0021] Each initial three-dimensional hand pose is transformed and rotated at the root node in the corresponding orthogonal reference coordinate system to obtain the target three-dimensional hand pose.

[0022] In one embodiment, the dynamic biomechanical features include the angles of flexion / extension and abduction of the finger bone vectors; acquiring the dynamic biomechanical features corresponding to each of the target object's three-dimensional hand postures includes:

[0023] For each of the target three-dimensional hand poses of the target object, obtain the bone vectors of each finger corresponding to the target three-dimensional hand pose and the local reference coordinate system of each finger; wherein, the local reference coordinate system is determined based on the reference plane where the root bone vector of the palm is located;

[0024] For each finger, the angles of flexion-extension and abduction of the finger bone vector are obtained in the local reference coordinate system where each finger is located; wherein, the angle of each flexion-extension angle is within the flexion-extension angle reference range; and the angle of each abduction angle is within the abduction angle reference range.

[0025] In one embodiment, the reference static biomechanical features include reference calcaneal length, the angle between reference calcaneal bones, and reference palmar surface curvature; the dynamic biomechanical features include the flexion-extension angle and abduction angle of the finger bone vectors; based on the reference static biomechanical features and the dynamic biomechanical features, determining the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates includes:

[0026] Based on the angles of flexion and extension of each finger bone vector corresponding to the target three-dimensional hand posture, the angles of abduction of each finger bone vector, the length of the reference root bone of the palm corresponding to the target three-dimensional hand posture, the angle between the reference root bones, the curvature of the reference palm surface, and the surface direction constraints, the vector encoding is determined.

[0027] Based on the vector encoding, three-dimensional joint coordinates are determined; wherein, the three-dimensional joint coordinates represent the spatial position of the joint points;

[0028] By performing data augmentation on the three-dimensional joint coordinates through affine transformation, different types of three-dimensional hand poses can be obtained;

[0029] By projecting the viewpoint onto each type of 3D hand pose, the 2D joint coordinates of each type of 3D hand pose are obtained.

[0030] Secondly, this application provides a method for training a pose estimation model, including:

[0031] Obtain the hand posture training data generated by the method described above;

[0032] The three-dimensional posture estimation model is trained using the hand posture training data; wherein the three-dimensional posture estimation model is used to process the two-dimensional joint coordinates in the hand posture training data to obtain the three-dimensional joint coordinates.

[0033] Thirdly, this application provides a biomechanical modeling and normalized training data generation device for human hand posture estimation, the device comprising:

[0034] The first acquisition module is used to acquire the three-dimensional hand posture data of each target object;

[0035] The second acquisition module is used to acquire, for each target object, reference static biomechanical features and dynamic biomechanical features corresponding to each of the target three-dimensional hand postures of the target object; wherein, the reference static biomechanical features correspond one-to-one with the target object, and the reference static biomechanical features are determined by normalizing the static biomechanical features of each of the target three-dimensional hand postures of each target object; the static biomechanical features are used to characterize the palm features corresponding to the target three-dimensional hand posture; the dynamic biomechanical features are used to characterize the spatial angles of the fingers corresponding to the target three-dimensional hand posture;

[0036] The first determining module is used to determine, for each target three-dimensional hand posture, the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates of the three-dimensional joint coordinates based on the reference static biomechanical characteristics and the dynamic biomechanical characteristics; wherein, the three-dimensional joint coordinates characterize the spatial position of the hand joint points;

[0037] The second determining module is used to determine human hand posture training data based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates.

[0038] Fourthly, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0039] Fifthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0040] The aforementioned biomechanical modeling and normalized training data generation method, apparatus, computer equipment, and computer-readable storage medium for human hand posture estimation normalize static biomechanical features to obtain reference static biomechanical features, thereby eliminating the negative impact of differences in data annotation and individual structure; it retains only the differences in dynamic biomechanical features to enhance the model's sensitivity to posture changes and robustness to individual structural differences; through the reference static and dynamic biomechanical features of the target 3D hand posture, the target 3D hand posture data can be expanded to obtain the 3D joint coordinates and corresponding 2D joint coordinates of the target 3D hand posture that conform to biomechanics; thus obtaining human hand posture training data with a reasonable scale and diversity.

[0041] Significant differences in hand biomechanical features lead to unstable 2D-3D mapping relationships learned by the model, thus limiting the model's generalization performance. By eliminating these differences in hand biomechanical features and expanding the diversity of human hand posture training data, the posture estimation model can be trained using diverse expanded training data. This improves both the accuracy and generalization performance of the posture estimation model. Furthermore, the model outputs three-dimensional joint coordinates, reducing reliance on manually labeled data. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a biomechanical modeling and normalized training data generation method for human hand posture estimation in one embodiment.

[0044] Figure 2 This is a schematic diagram illustrating the hand posture in one embodiment. Figure 2 (a) in the diagram is a schematic diagram of joint annotation. Figure 2 (b) in the figure is a schematic diagram showing the annotation of the root bone vector of the palm and the annotation of the finger bone vector of the finger. Figure 2 (c) in the diagram is a schematic diagram of the angle between the root vectors; Figure 2 (d) in the diagram is a schematic representation of the basis vectors; Figure 2 (e) in the diagram is a schematic of a local reference coordinate system;

[0045] Figure 3 This is a flowchart illustrating the calculation of the static biomechanical characteristics of the three-dimensional hand posture of each target in one embodiment.

[0046] Figure 4 This is a flowchart illustrating the process of obtaining the three-dimensional hand poses of a target object in one embodiment;

[0047] Figure 5 This is a schematic diagram of the normalized biomechanical features of the initial three-dimensional hand posture in one embodiment;

[0048] Figure 6 This is a flowchart illustrating the process of determining three-dimensional joint coordinates and corresponding two-dimensional joint coordinates based on reference static and dynamic biomechanical features in one embodiment.

[0049] Figure 7 A schematic diagram of the training architecture for a 2D-3D mapping model generated based on biomechanical modeling and normalized training data;

[0050] Figure 8 This is a structural block diagram of a biomechanical modeling and normalized training data generation device for human hand posture estimation in one embodiment.

[0051] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] In one embodiment, such as Figure 1 As shown, a biomechanical modeling and normalized training data generation method for human hand pose estimation is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps S102 to S108. Wherein:

[0054] Step S102: Obtain the three-dimensional hand pose data of each target object.

[0055] Optionally, the terminal acquires different target 3D hand poses of Alice, Bob, and Charlie at different times, such as clenching a fist or making a V-sign. The target 3D hand poses are normalized data.

[0056] Step S104: For each target object, obtain the reference static biomechanical features and dynamic biomechanical features corresponding to the three-dimensional hand posture of each target object.

[0057] Among them, the reference static biomechanical features correspond one-to-one with the target objects. The reference static biomechanical features are determined by normalizing the static biomechanical features of the three-dimensional hand posture of each target object.

[0058] Static biomechanical features are used to characterize the palm features corresponding to the three-dimensional hand posture of a target. The palm contains a set of joints. Research has found that the structure of an individual's palm is stable in space and time and can be used as a static biomechanical characteristic.

[0059] Dynamic biomechanical features are used to characterize the spatial angles of the fingers corresponding to the three-dimensional hand posture of the target.

[0060] like Figure 2 As shown in (a), it includes the joints. to The root joint is , Represents the root node before coordinate transformation. Represents the root node and keypoint after coordinate transformation. Defined as the central joint of the palm. (Joint) The parent node is defined as ; Skeletal vector Defined as from parent joint j p(i+1) Pointing to its subjoint j (i+1) The vector is calculated as shown in formula (1).

[0061] Formula (1)

[0062] The root bone vector set is represented as ,correspond Figure 2 The solid line portion in (b); b0 = j1 - j p1 b4 = j5 - j p5 b8=j9-j p9 b 12 =j 13 -j p13 b 16 =j 17 -j p17 ;where j p1 Represents the parent node of key point j1, j p5 The parent node of key j5, j p9 The parent node of key j9, j p13 Representing key point j 13 parent node, j p17 Representing key point j 17 The parent node of each node is node j0.

[0063] The shape and structure of the fingers, such as Figure 2 As shown by the dashed line in (b), the nodes corresponding to the thumb bone vector are divided into {j1 j2 j3 j4}; the nodes corresponding to the index finger bone vector are divided into {j5 j6 j7 j8}; the nodes corresponding to the middle finger bone vector are divided into {j9 j10 j11 j12}; the nodes corresponding to the ring finger bone vector are divided into {j13 j14 j15 j16}; and the nodes corresponding to the little finger bone vector are divided into {j17 j18 j19 j20}.

[0064] like Figure 2 As shown in (c), the angle between root vectors, such as the angle between root vector b0 and root vector b4, is denoted as... The angle between root bone vectors b4 and b8 is denoted as... ; Root bone vector b8 and root bone vector b 12 The angle between them is denoted as ; Root bone vector b 12 and root bone vector b 16 The angle between them is denoted as .

[0065] like Figure 2 As shown in (d), the orthogonal vector of adjacent root bones is defined as e0 = norm(b0 × b4), the orthogonal vectors of b0 and b4 are normalized to obtain the basis vector e0; similarly, e1 = norm(b4 × b8), the orthogonal vectors of b4 and b8 are normalized to obtain the basis vector e1; based on b8 and b 12 The basis vector e3 can be obtained; based on b 12 and b 16 The basis vector e4 can be obtained.

[0066] like Figure 2 As shown in (e), each finger constructs an independent local reference coordinate system. The local reference coordinate system is established with the palmar surface as the reference plane for the root bone. They are considered to be roughly in the same plane.

[0067] Optionally, for each target object, the terminal acquires the reference static and dynamic biomechanical features corresponding to each target object's three-dimensional hand pose. For example, the reference static and dynamic biomechanical features corresponding to each target object Alice's three-dimensional hand pose, the reference static and dynamic biomechanical features corresponding to each target object Bob's three-dimensional hand pose, and the reference static and dynamic biomechanical features corresponding to each target object Charlie's three-dimensional hand pose.

[0068] Furthermore, after unifying the static biomechanical features of the same target object using reference static biomechanical features, the differences between different target object 3D hand postures are reduced to dynamic biomechanical features. During model training, the "changes" observed only originate from changes in posture and are no longer affected by individual structural differences.

[0069] Step S106: For each target's three-dimensional hand posture, based on reference static biomechanical features and dynamic biomechanical features, determine the three-dimensional joint coordinates of the target's three-dimensional hand posture and the corresponding two-dimensional joint coordinates.

[0070] Among them, the three-dimensional joint coordinates represent the spatial position of the hand joints.

[0071] Optionally, the terminal concatenates the reference static biomechanical features and dynamic biomechanical features of each target's three-dimensional hand pose to obtain the biomechanical features corresponding to different target three-dimensional hand poses, denoted as [reference static biomechanical features, dynamic biomechanical features]. The terminal combines the reference static biomechanical features and dynamic biomechanical features of each target's three-dimensional hand pose to form a complete encoding. The terminal encodes... Processing techniques, such as weakly supervised processing, are applied to obtain the 3D joint coordinates and corresponding 2D joint coordinates for each target's 3D hand pose. No additional manual annotation is required; by modeling the individual hand structure and motion constraints, a large number of physically feasible 3D joint coordinates and their corresponding 2D joint coordinates are automatically generated.

[0072] Step S108: Determine the hand posture training data based on each three-dimensional joint coordinate and the corresponding two-dimensional joint coordinate.

[0073] Optionally, the terminal can concatenate the obtained three-dimensional joint coordinates as 3D joint coordinates and the corresponding two-dimensional joint coordinates as 2D joint coordinates into an attitude pair, which can be denoted as [2D joint coordinates, 3D joint coordinates], to obtain multiple attitude pairs.

[0074] It should be noted that there is no limit to the number of target objects or the number of three-dimensional hand poses of the target.

[0075] In the aforementioned biomechanical modeling and normalized training data generation method for human hand posture estimation, reference static biomechanical features are obtained by normalizing static biomechanical features to eliminate the negative impact of differences in data annotation and individual structure. Only the differences in dynamic biomechanical features are retained to enhance the model's sensitivity to posture changes and robustness to individual structural differences. By using the reference static and dynamic biomechanical features of the target 3D hand posture, the target 3D hand posture data can be expanded to obtain the 3D joint coordinates and corresponding 2D joint coordinates of the target 3D hand posture that conform to biomechanics, thereby obtaining human hand posture training data with a reasonable scale and diversity.

[0076] In an exemplary embodiment, obtaining reference static biomechanical features corresponding to each target three-dimensional hand posture of the target object includes: calculating the static biomechanical features of each target three-dimensional hand posture based on each target three-dimensional hand posture of the target object; wherein, the static biomechanical features include the length of the calcaneus, the angle between the calcaneus and the palmar surface curvature; and determining the reference static biomechanical features corresponding to the target object based on the length of the calcaneus, the angle between the calcaneus and the palmar surface curvature of each target three-dimensional hand posture of the target object.

[0077] The length of the calcaneus is denoted as . abbreviation The angle between the calcaneus is denoted as The curvature of the palm is denoted as The reference calcaneal length is denoted as... The angle between the reference bones is denoted as... The reference palm curvature is denoted as .

[0078] Optionally, the terminal obtains the root bone length corresponding to each target 3D hand pose of the target object Alice. The angle between the calcaneus Palm curvature The root bone length corresponding to each target object Bob's 3D hand pose. The angle between the calcaneus Palm curvature The root bone length corresponding to each target object Charlie's 3D hand pose. The angle between the calcaneus Palm curvature .

[0079] Furthermore, the terminal statistically analyzes the static biomechanical characteristics of all target three-dimensional hand poses for each target object. Since the measurement of the three-dimensional hand pose of a single target may be subject to noise or slight posture influence, taking the average of the static biomechanical characteristics of all target three-dimensional hand poses for each target object can more robustly estimate its true and inherent body structure and reduce measurement errors and posture interference.

[0080] Optionally, the reference static biomechanical features include reference calcaneal length, the angle between reference calcaneal bones, and reference palmar surface curvature. The terminal calculates the average calcaneal length corresponding to each of the target object's three-dimensional hand poses to obtain the target object's reference calcaneal length; calculates the average angle between the calcaneal bones corresponding to each of the target object's three-dimensional hand poses to obtain the angle between the target object's reference calcaneal bones; and calculates the average reference palmar surface curvature corresponding to each of the target object's three-dimensional hand poses to obtain the target object's reference palmar surface curvature. The terminal then uses the static biomechanical features corresponding to each of the target object's three-dimensional hand poses. Replace with reference static biomechanical characteristics .

[0081] It should be noted that all three-dimensional hand poses of the same target object now possess identical reference static biomechanical characteristics. However, the reference static biomechanical characteristics differ between individual target objects.

[0082] In this embodiment, by unifying the static biomechanical characteristics of the same target object and replacing the static biomechanical characteristics with reference static biomechanical characteristics, the differences in body structure between different target objects, i.e. individuals, in the training data are eliminated, thereby forcing the model to focus on learning the dynamic changes in posture itself.

[0083] In one exemplary embodiment, such as Figure 3 As shown, based on the three-dimensional hand poses of each target object, the static biomechanical characteristics of each target's three-dimensional hand pose are calculated, including the following steps S302 to S308. Wherein:

[0084] Step S302: For each target three-dimensional hand pose of the target object, obtain the root bone vectors of the palm corresponding to the target three-dimensional hand pose.

[0085] Wherein, the root bone vector set is represented as ,correspond Figure 2 The solid line portion in (b) shows that each target's three-dimensional hand pose corresponds to 5 root vectors.

[0086] Optionally, the terminal traverses each target object, obtains all target 3D hand poses of each target object, and obtains the 5 root bone vectors of the palm corresponding to each target 3D hand pose.

[0087] Step S304: Calculate the Euclidean distance based on each root bone vector to obtain the root bone length corresponding to the target three-dimensional hand posture.

[0088] Optionally, the terminal calculates the Euclidean distance based on each root vector, such as... , , , and The lengths of the five root bones corresponding to the target's three-dimensional hand pose are obtained, and the set of lengths of each root bone is represented as follows: .

[0089] Step S306: Calculate the angle between the root vectors of adjacent root bones using the inverse cosine function to obtain the angle between the root bones corresponding to the target three-dimensional hand pose.

[0090] The number of angles between the root bones is 4, which is less than the number of root bone vectors, which is 5. (Two vectors) and angular distance The calculation is performed using the inverse cosine function, as shown in formula (2).

[0091] Formula (2)

[0092] Optionally, the terminal calculates the angle between the root vectors of adjacent root bones using the inverse cosine function according to formula (3), thus obtaining the angle between the root bones corresponding to the target three-dimensional hand posture. For example, the angle between root vector b0 and root vector b4 is denoted as... The angle between root bone vectors b4 and b8 is denoted as... ; Root bone vector b8 and root bone vector b 12 The angle between them is denoted as ; Root bone vector b 12 and root bone vector b 16 The angle between them is denoted as .

[0093] Formula (3)

[0094] Step S308: Determine the basis vector based on the root bone vector of the adjacent root bone, and determine the palm curvature corresponding to the three-dimensional hand posture of the target based on the basis vector.

[0095] Here, the basis vectors represent the orthogonal vectors of adjacent root bone vectors. The palmar surface curvature represents the bending relationship between adjacent bone surfaces.

[0096] Optionally, the terminal determines the basis vectors based on the root vectors of adjacent root bones. For example, the orthogonal vectors of b0 and b4 are normalized to obtain the basis vector e0; similarly, e1 = norm(b4 × b8), and the orthogonal vectors of b4 and b8 are normalized to obtain the basis vector e1; based on b8 and b 12 The basis vector e3 can be obtained; based on b 12 and b 16 The basis vector e4 can be obtained. The terminal determines the palm curvature corresponding to the three-dimensional hand posture of the target through each basis vector. Since there are 4 basis vectors in a three-dimensional hand posture of a target, there are 3 palm curvatures. The calculation of the palm curvature is shown in formula (4).

[0097] Formula (4)

[0098] In this embodiment, static biomechanical characteristics, including root bone length, the angle between root bone vectors, and palmar surface curvature, can be calculated using the root bone vector corresponding to a target three-dimensional hand posture.

[0099] In one exemplary embodiment, such as Figure 4 As shown, the three-dimensional hand poses of the target object are obtained, including steps S402 to S406. Wherein:

[0100] Step S402: Obtain the initial three-dimensional hand poses of the target object.

[0101] The different initial 3D hand poses are different gestures from different objects at different times. The target object is a specific person.

[0102] Optionally, the terminal can acquire various publicly available three-dimensional (3D) hand datasets, such as STB, RHD, FreiHAND, and HO-3D hand datasets. These datasets cover various acquisition environments, camera parameters, and hand pose distributions. These datasets include different initial 3D hand poses from different objects at different times. Hand datasets typically include initial 3D hand poses for multiple different targets. For example, the terminal might acquire different initial 3D hand poses (such as clenched fists or V-signs) for Alice, Bob, and Charlie at different times.

[0103] Step S404: For each initial three-dimensional hand pose, obtain the orthogonal reference coordinate system corresponding to the initial three-dimensional hand pose.

[0104] The orthogonal reference coordinate system is a coordinate system with the palm center joint as the origin and the palm normal vector as the target direction, such as the z-axis, for each initial 3D hand pose. The orthogonal reference coordinate system is denoted as... The normal vector, perpendicular to the palm, best represents the initial three-dimensional hand posture and the orientation of the palm. That is, the origin is located at the center joint of the palm. The Z-axis coincides with the palmar normal vector, the X-axis is parallel to the metacarpal extension direction, and the Y-axis is calculated using the cross product to ensure that the three axes are orthogonal.

[0105] For example, the central joint of the palm Figure 2 Key points in Defined as the central joint of the palm, that is Figure 5 The central joint of the middle finger, from top to bottom. The central joint of the palm is the core hub of the hand's skeletal structure, connecting the wrist and fingers. Using it as the origin best represents the "central position" of the hand.

[0106] Optionally, for each initial 3D hand pose data, where the absolute coordinates (x, y, z) in 3D space are different, and the palm may be facing upwards, downwards, tilted, or rotated. Therefore, the orientation of the palm plane in space is unique for each initial 3D hand pose. The terminal obtains an orthogonal reference coordinate system corresponding to each initial 3D hand pose. The orthogonal reference coordinate system is a coordinate system with the palm center joint of each initial 3D hand pose as the origin and the palm normal vector as the target direction, such as the z-axis.

[0107] Step S406: Transform and rotate the root node of each initial three-dimensional hand pose in the corresponding orthogonal reference coordinate system to obtain the three-dimensional hand pose of each target.

[0108] Optionally, the terminal translates the center of the palm of each initial 3D hand pose data to the origin (0,0,0) of the world coordinate system, eliminating the absolute positional differences of the initial 3D hand pose data in space. By rotating the orthogonal reference coordinate system of each initial 3D hand pose data, the Z-axis (calculated based on its own normal vector) of all initial 3D hand pose data is aligned (e.g., all pointing to the Z+ direction of the world coordinate system), eliminating the absolute differences in the palm orientation of each initial 3D hand pose data.

[0109] Furthermore, by transforming all initial 3D hand pose data to their respective local coordinate systems (origin at the center of the palm, Z-axis perpendicular to the palm surface), pose standardization is achieved. Thus, a "thumbs up" pose, regardless of its location in the initial 3D hand pose data or the direction the palm faces, will have very similar joint coordinate relationships in the normalized coordinate system. Figure 5 As shown, an initial 3D hand pose data is normalized at the root node in the corresponding orthogonal reference coordinate system, such as through position transformation and rotation. This means the root node... After changes and rotations The target's three-dimensional hand pose is obtained.

[0110] Optionally, the initial three-dimensional hand pose data is a clenched fist, and the terminal sets the central joint of the palm... Translate to the origin of the coordinate system, while simultaneously rotating the entire hand so that the normal vector coincides with the Z-axis. Transform the center joint of the palm. Fixed at the origin, while the root node is as follows: Figure 2 In After obtaining the new coordinate values, the final result is obtained in... The target 3D hand pose is normalized in coordinates.

[0111] In this embodiment, the standardization of the initial three-dimensional hand pose data is achieved by transforming all the initial three-dimensional hand pose data to their respective local coordinate systems (the origin is in the palm of the hand, and the Z-axis is perpendicular to the palm surface).

[0112] In an exemplary embodiment, the dynamic biomechanical features include the angles of flexion / extension and abduction of the finger bone vectors; obtaining the dynamic biomechanical features corresponding to each target three-dimensional hand posture of the target object includes: for each target three-dimensional hand posture of the target object, obtaining each bone vector of each finger corresponding to the target three-dimensional hand posture and the local reference coordinate system in which each finger is located; wherein, the local reference coordinate system is determined based on the reference plane in which the root bone vector of the palm is located; for each finger, obtaining the angles of flexion / extension and abduction of the finger bone vector in the local reference coordinate system in which each finger is located.

[0113] The angle of each flexion-extension angle is denoted as . Located within the reference range for flexion and extension angles; the angle of each abduction angle is denoted as... Located within the abduction angle reference range. The flexion-extension angle represents the bending angle of the phalanx in the sagittal plane; the abduction angle represents the spreading angle of the phalanx in the coronal plane. The local reference coordinate system is determined based on the reference plane where the calcaneal vector of the palm lies.

[0114] Optionally, the terminal statistically analyzes the numerical distribution of the flexion-extension angle and abduction angle of each finger bone vector in the three-dimensional hand posture of each target, and extracts the maximum and minimum values ​​of the flexion-extension angle, as well as the maximum and minimum values ​​of the abduction angle, for each finger bone vector.

[0115] The abduction and flexion / extension angles of all finger bone vectors are combined into a multi-dimensional space. Random sampling within this multi-dimensional space ensures that the generated gestures 100% meet the limits of human movement (e.g., fingers will not bend backward).

[0116] For each target object's 3D hand pose, the terminal obtains the bone vectors of each finger and the local reference coordinate system of each finger.

[0117] By minimizing the following objective function As shown in formula (5), the normal vector of the reference plane is solved. Make the normal vector and the basis vectors , , By maintaining as much orthogonality as possible, a palm reference hyperplane is constructed. :

[0118] Formula (5)

[0119] In the formula, represents the orthonormal basis vector of the world coordinate system; || ||2 represents the Euclidean distance; This represents minimizing the deviation between the normal vector and the basis vectors; Representative reference plane The normal vector.

[0120] Therefore, local reference coordinate system The three axes are defined as shown in formula (6).

[0121] Formula (6)

[0122] In the formula, Represents the skeletal vector The parent bone vector corresponds to the direction vector of the previous level bone of the current bone. The X-axis, representing the local coordinate system, is geometrically defined as parallel to the palmar extension direction (thumb → little finger). The Y-axis, representing the local coordinate system, is geometrically defined as the direction perpendicular to the palm normal. The Z-axis represents the local coordinate system and is geometrically defined as perpendicular to the palm (normal vector direction).

[0123] For each finger, the terminal uses a local reference coordinate system. The angle of flexion and extension of the finger bone vector is obtained from the data. and the angle of abduction .

[0124] Where α() represents the function for calculating the angle between two vectors, Represents the skeletal vector Projected onto the XZ plane to obtain a two-dimensional vector after eliminating the Y-axis component. The flexion-extension angle is based on the skeletal vector. The angle between the XZ plane projection and the Z-axis is determined, and the abduction angle is based on the skeletal vector. Projection onto the XZ plane and the skeletal vector It is determined by itself.

[0125] In this embodiment, the movement (flexion / extension / abduction) of each finger is described in a local coordinate system that conforms to the anatomical structure, providing a physically reasonable parameterization basis for gesture recognition.

[0126] In one exemplary embodiment, such as Figure 6 As shown, the reference static biomechanical features include the reference calcaneus length, the angle between the reference calcaneus, and the reference palmar surface curvature; the dynamic biomechanical features include the flexion-extension angle and the abduction angle of the finger bone vectors; based on the reference static and dynamic biomechanical features, the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates are determined, including steps S602 to S608. Wherein:

[0127] Step S602: Based on the angles of flexion and extension of each finger bone vector corresponding to the target three-dimensional hand posture, the angles of abduction of each finger bone vector, the length of the reference root bone of the palm corresponding to the target three-dimensional hand posture, the angle between the reference root bones, the curvature of the reference palm surface, and the surface direction constraints, determine the vector encoding.

[0128] The surface orientation constraint is determined based on the reference palm curvature. When calculating the palm curvature using the inverse cosine arccos function, the symmetry of the arccos function may generate multiple palm structure solutions. To ensure the uniqueness of the solution, a relative surface orientation constraint is introduced. As shown in formula (7).

[0129] Formula (7)

[0130] Optionally, the terminal determines the vector encoding based on the flexion-extension angles and abduction angles of the finger bone vectors corresponding to the target three-dimensional hand posture, as well as the reference helix length, the angle between the reference helixes, the reference palm curvature, and the surface direction constraints of the palm corresponding to the target three-dimensional hand posture. This represents the normalized and encoded vector of hand biomechanical parameters. The original vector encoding consists of the flexion / extension angles and abduction angles of each finger bone vector corresponding to a target 3D hand pose, as well as the length of the helix bone, the angle between helix bones, the palmar curvature, and the surface orientation constraints of the palm corresponding to the target 3D hand pose. It should be noted that, and They are the same.

[0131] like Figure 5 As shown, the hand biomechanical parameter vector of the target's three-dimensional hand pose data. Normalization is performed to obtain the vector code. .

[0132] Step S604: Determine the three-dimensional joint coordinates based on vector encoding.

[0133] Among them, the three-dimensional joint coordinates represent the spatial position of the joint points.

[0134] Optionally, the terminal uses a Sequential Least Squares Quadratic Programming (SLSQP) optimization algorithm to solve for the problem. The three-dimensional joint coordinates corresponding to the vector encoding.

[0135] Step S606: Perform data augmentation on the three-dimensional joint coordinates through affine transformation to obtain different types of three-dimensional hand poses.

[0136] Optionally, the terminal can perform data augmentation on the three-dimensional joint coordinates through affine transformations, such as translation and rotation, to enhance their geometric diversity and obtain different types of three-dimensional hand poses.

[0137] Step S608: Project the three-dimensional hand pose of each type through view projection to obtain the two-dimensional joint coordinates of the three-dimensional hand pose of each type.

[0138] Among them, perspective projection simulates the 3D hand posture, or 3D gesture, of each perturbation type as the effect of a camera shot, generating matching 2D key point data.

[0139] Optionally, the terminal obtains the corresponding two-dimensional joint coordinate representation by projecting a random viewpoint onto the three-dimensional hand posture for each disturbance type.

[0140] In this embodiment, acquiring high-quality 3D pose data using traditional techniques is costly and difficult to annotate, which greatly limits the diversity and scale of training data. By perturbation and projection, hundreds of visual representations can be derived from the 3D hand pose of a single target, all of which conform to physical laws, thus expanding diversity and scale. Moreover, no additional manual annotation is required; by modeling the individual hand structure and motion constraints, a large number of physically feasible 2D-3D pose samples can be automatically synthesized.

[0141] In one exemplary embodiment, a pose estimation model training method includes: acquiring human hand pose training data generated by the above method; and using the human hand pose training data to train a three-dimensional pose estimation model.

[0142] Among them, the three-dimensional pose estimation model is used to process the two-dimensional joint coordinates in the human hand pose training data to obtain the three-dimensional joint coordinates.

[0143] Optionally, the hand posture training data generated by the terminal using any of the above methods, such as Figure 7 As shown, the static biomechanical features corresponding to a target object's 3D hand pose are obtained. Normalization is then performed on these static biomechanical features, specifically by normalizing the root node of an initial 3D hand pose data in the corresponding orthogonal reference coordinate system, such as through position transformation and rotation, to obtain the target's 3D hand pose. The static biomechanical features corresponding to this target's 3D hand pose are then analyzed. Replace with the reference static biomechanical features of the target object to which the target's three-dimensional hand pose belongs. Based on the reference static and dynamic biomechanical features of the target's three-dimensional hand posture, vector encoding of the biomechanical features is obtained. Based on the sequence least squares quadratic programming optimization algorithm, the solution is obtained from the result. The 3D keypoint spatial locations are encoded with consistent vectors. Affine transformations are used to expand the 3D keypoint spatial locations through spatial perturbation and other operations. Random viewpoint projection is then applied to the expanded 3D hand gesture pose to obtain corresponding 2D keypoint representations, constructing rich and structurally sound 2D-3D pose pairs. The 2D-3D pose pairs are used to train the 3D pose estimation model. For example, when designing a 3D pose estimation model on the PyTorch platform, typical 2D-3D mapping models such as ST-GCN and VPose3D can be selected. The parameter configuration can be as follows: L2 loss (MSE) is used as the regression loss function; the input image size is 224×224; the initial learning rate is set to 0.001; the batch size is 512; and the total number of training epochs is 500. Mean squared error (MSE) is used as the loss function during training. This configuration is suitable for both weakly supervised training and normalization strategies. In weakly supervised training, 200,000 samples are generated for each individual. The trained 3D pose estimation model is then obtained.

[0144] Significant differences in hand biomechanical features lead to unstable 2D-3D mapping relationships learned by the model, thus limiting the model's generalization performance. By eliminating these differences in hand biomechanical features and expanding the diversity of human hand posture training data, the posture estimation model can be trained using diverse expanded training data. This improves both the accuracy and generalization performance of the posture estimation model. Furthermore, the model outputs three-dimensional joint coordinates, reducing reliance on manually labeled data.

[0145] Using a trained 3D pose estimation model: 2D hand images are captured in real-time by an external image acquisition device for subsequent keypoint extraction. A high-precision 2D pose estimation algorithm (such as HRNet) is used to process the input image, extracting the 2D spatial coordinates of key joints to obtain a standard-format 2D hand pose representation. These 2D spatial coordinates are then input into the trained 3D pose estimation model (such as ST-GCN or VPose3D) to infer the hand pose in 3D space. This regression process maintains real-time performance and exhibits good pose structure consistency and spatial rationality, making it suitable for applications such as human-computer interaction and virtual reality.

[0146] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0147] Based on the same inventive concept, this application also provides a device for generating biomechanical modeling and normalized training data for hand posture estimation, which implements the aforementioned method for generating biomechanical modeling and normalized training data for hand posture estimation. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the device for generating biomechanical modeling and normalized training data for hand posture estimation provided below can be found in the limitations of the method for generating biomechanical modeling and normalized training data for hand posture estimation described above, and will not be repeated here.

[0148] In one exemplary embodiment, such as Figure 8 As shown, a biomechanical modeling and normalized training data generation device for human hand posture estimation is provided, comprising: a first acquisition module 801, a second acquisition module 802, a first determination module 803, and a second determination module 804, wherein:

[0149] The first acquisition module 801 is used to acquire the three-dimensional hand posture data of each target object.

[0150] The second acquisition module 802 is used to acquire, for each target object, the reference static biomechanical features and dynamic biomechanical features corresponding to the three-dimensional hand postures of the target object; wherein, the reference static biomechanical features correspond one-to-one with the target object, and the reference static biomechanical features are determined by normalizing the static biomechanical features of the three-dimensional hand postures of each target object; the static biomechanical features are used to characterize the palm features corresponding to the three-dimensional hand postures of the target; the dynamic biomechanical features are used to characterize the spatial angles of the fingers corresponding to the three-dimensional hand postures of the target.

[0151] The first determining module 803 is used to determine the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates of the three-dimensional hand posture for each target based on reference static biomechanical characteristics and dynamic biomechanical characteristics; wherein, the three-dimensional joint coordinates represent the spatial position of the hand joint points.

[0152] The second determining module 804 is used to determine human hand posture training data based on each three-dimensional joint coordinate and the corresponding two-dimensional joint coordinate.

[0153] In an exemplary embodiment, the second acquisition module 802 is further configured to calculate the static biomechanical characteristics of each target three-dimensional hand posture based on each target three-dimensional hand posture of the target object; the static biomechanical characteristics include the length of the calcaneus, the angle between the calcaneus and the curvature of the palm surface; and determine the reference static biomechanical characteristics corresponding to the target object based on the length of the calcaneus, the angle between the calcaneus and the curvature of the palm surface of each target three-dimensional hand posture of the target object.

[0154] In an exemplary embodiment, the second acquisition module 802 is further configured to acquire, for each target three-dimensional hand pose of the target object, the root bone vectors of the palm corresponding to the target three-dimensional hand pose; calculate the Euclidean distance based on each root bone vector to obtain the root bone length corresponding to the target three-dimensional hand pose; calculate the angle between the root bone vectors of adjacent root bones using the inverse cosine function to obtain the angle between the root bones corresponding to the target three-dimensional hand pose; wherein, the number of angles between root bones is less than the number of root bone vectors; determine the basis vectors based on the root bone vectors of adjacent root bones, and determine the palm curvature corresponding to the target three-dimensional hand pose based on the basis vectors; wherein, the basis vectors represent the orthogonal vectors of adjacent root bone vectors.

[0155] In an exemplary embodiment, the first acquisition module 801 is further configured to acquire each initial three-dimensional hand pose of the target object; for each initial three-dimensional hand pose, acquire the orthogonal reference coordinate system corresponding to the initial three-dimensional hand pose; the orthogonal reference coordinate system is a coordinate system with the palm center joint of each initial three-dimensional hand pose as the origin and the palm normal vector as the target direction; and perform position transformation and rotation of the root node of each initial three-dimensional hand pose in the corresponding orthogonal reference coordinate system to obtain each target three-dimensional hand pose.

[0156] In an exemplary embodiment, the dynamic biomechanical features include the flexion-extension angle and abduction angle of the finger bone vectors; the second acquisition module 802 is further configured to acquire, for each target three-dimensional hand posture of the target object, each bone vector of each finger corresponding to the target three-dimensional hand posture and the local reference coordinate system in which each finger is located; wherein, the local reference coordinate system is determined based on the reference plane in which the root bone vector of the palm is located; for each finger, the flexion-extension angle and abduction angle of the finger bone vector are acquired in the local reference coordinate system in which each finger is located; wherein, the angle of each flexion-extension angle is within the flexion-extension angle reference range; the angle of each abduction angle is within the abduction angle reference range.

[0157] In an exemplary embodiment, the reference static biomechanical features include the reference helix length, the angle between the reference helixes, and the reference palmar surface curvature; the dynamic biomechanical features include the flexion-extension angle and the abduction angle of the finger bone vectors; the first determining module 803 is used to determine the vector encoding based on the flexion-extension angle and the abduction angle of each finger bone vector corresponding to the target three-dimensional hand posture, as well as the reference helix length, the angle between the reference helixes, the reference palmar surface curvature, and the surface direction constraints of the palm corresponding to the target three-dimensional hand posture; based on the vector encoding, the three-dimensional joint coordinates are determined; wherein, the three-dimensional joint coordinates represent the spatial position of the joint points; the three-dimensional joint coordinates are augmented by affine transformation to obtain different three-dimensional hand postures; the three-dimensional hand posture of each perturbation type is projected by view projection to obtain the two-dimensional joint coordinates of the three-dimensional hand posture of each perturbation type.

[0158] In one exemplary embodiment, a pose estimation model training apparatus is provided, comprising: a third acquisition module and a training module, wherein:

[0159] The third acquisition module is used to acquire the hand posture training data generated by any of the above methods.

[0160] The training module is used to train a 3D pose estimation model using human hand pose training data; wherein, the 3D pose estimation model is used to process the 2D joint coordinates in the human hand pose training data to obtain the 3D joint coordinates.

[0161] The modules in the aforementioned biomechanical modeling and normalized training data generation device and posture estimation model training device for human hand posture estimation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0162] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores human hand posture data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When executed by the processor, the computer program implements a biomechanical modeling and normalized training data generation method for human hand posture estimation.

[0163] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0164] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0165] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0166] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0167] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0168] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A biomechanical modeling and normalized training data generation method for human hand posture estimation, characterized in that, The method includes: Obtain the 3D hand pose data of each target object; For each target object, obtain the dynamic biomechanical features corresponding to the three-dimensional hand posture of each target object; Based on the target object's three-dimensional hand posture, calculate the static biomechanical characteristics of each target's three-dimensional hand posture; the static biomechanical characteristics include calcaneal length, the angle between calcaneal bones, and palmar surface curvature; Based on the calcaneal length, the angle between the calcaneal bones, and the palmar surface curvature of each of the target three-dimensional hand postures of the target object, a reference static biomechanical feature corresponding to the target object is determined; wherein, the reference static biomechanical feature corresponds one-to-one with the target object, and the reference static biomechanical feature is determined by normalizing the static biomechanical features of each of the target three-dimensional hand postures of each target object; the static biomechanical feature is used to characterize the palm features corresponding to the target three-dimensional hand posture; the dynamic biomechanical feature is used to characterize the spatial angles of the fingers corresponding to the target three-dimensional hand posture; For each target three-dimensional hand posture, based on the reference static biomechanical features and the dynamic biomechanical features, the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates are determined; wherein, the three-dimensional joint coordinates characterize the spatial position of the hand joints; Based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates, human hand posture training data is determined.

2. The method according to claim 1, characterized in that, The step of calculating the static biomechanical characteristics of each of the target three-dimensional hand postures based on the target object's three-dimensional hand postures includes: For each of the target three-dimensional hand poses of the target object, obtain the root bone vectors of the palm corresponding to the target three-dimensional hand pose; The Euclidean distance is calculated based on each of the root bone vectors to obtain the root bone length corresponding to the three-dimensional hand posture of the target. The angle between the root vectors of adjacent root bones is calculated by using the inverse cosine function to obtain the angle between the root bones corresponding to the three-dimensional hand posture of the target; wherein, the number of angles between root bones is less than the number of root vectors; Based on the root bone vectors of adjacent root bones, a basis vector is determined, and the palm curvature corresponding to the three-dimensional hand posture of the target is determined based on the basis vector; wherein, the basis vector represents the orthogonal vector of adjacent root bone vectors.

3. The method according to claim 1, characterized in that, The acquisition of the target object's three-dimensional hand poses includes: Obtain the initial 3D hand poses of the target object; For each initial three-dimensional hand pose, an orthogonal reference coordinate system corresponding to the initial three-dimensional hand pose is obtained; the orthogonal reference coordinate system is a coordinate system with the palm center joint of each initial three-dimensional hand pose as the origin and the palm normal vector as the target direction; Each initial three-dimensional hand pose is transformed and rotated at the root node in the corresponding orthogonal reference coordinate system to obtain the target three-dimensional hand pose.

4. The method according to claim 1, characterized in that, The dynamic biomechanical features include the angles of flexion / extension and abduction of the finger bone vectors; obtaining the dynamic biomechanical features corresponding to each of the target object's three-dimensional hand postures includes: For each of the target three-dimensional hand poses of the target object, obtain the bone vectors of each finger corresponding to the target three-dimensional hand pose and the local reference coordinate system of each finger; wherein, the local reference coordinate system is determined based on the reference plane where the root bone vector of the palm is located; For each finger, the angles of flexion-extension and abduction of the finger bone vector are obtained in the local reference coordinate system where each finger is located; wherein, the angle of each flexion-extension angle is within the flexion-extension angle reference range; and the angle of each abduction angle is within the abduction angle reference range.

5. The method according to claim 1, characterized in that, The reference static biomechanical features include reference calcaneal length, the angle between reference calcaneal bones, and reference palmar surface curvature; the dynamic biomechanical features include the flexion-extension angle and abduction angle of the finger bone vectors; based on the reference static biomechanical features and the dynamic biomechanical features, the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates are determined, including: Based on the angles of flexion and extension of each finger bone vector corresponding to the target three-dimensional hand posture, the angles of abduction of each finger bone vector, the length of the reference root bone of the palm corresponding to the target three-dimensional hand posture, the angle between the reference root bones, the curvature of the reference palm surface, and the surface direction constraints, the vector encoding is determined. Based on the vector encoding, three-dimensional joint coordinates are determined; wherein, the three-dimensional joint coordinates represent the spatial position of the joint points; By performing data augmentation on the three-dimensional joint coordinates through affine transformation, different types of three-dimensional hand poses can be obtained; By projecting the viewpoint onto each type of 3D hand pose, the 2D joint coordinates of each type of 3D hand pose are obtained.

6. A method for training a pose estimation model, characterized in that, include: Obtain human hand posture training data generated by the method described in any one of claims 1-5; The three-dimensional pose estimation model is trained using the aforementioned hand pose training data; in; The three-dimensional pose estimation model is used to process the two-dimensional joint coordinates in the human hand pose training data to obtain the three-dimensional joint coordinates.

7. A biomechanical modeling and normalized training data generation device for human hand posture estimation, characterized in that, The device includes: The first acquisition module is used to acquire the three-dimensional hand posture data of each target object; The second acquisition module is used to acquire, for each target object, the dynamic biomechanical features corresponding to each of the target three-dimensional hand postures of the target object; calculate the static biomechanical features of each of the target three-dimensional hand postures of the target object based on each of the target three-dimensional hand postures of the target object; the static biomechanical features include the length of the calcaneus, the angle between the calcaneus, and the palmar surface curvature; determine the reference static biomechanical features corresponding to the target object based on the length of the calcaneus, the angle between the calcaneus, and the palmar surface curvature of each of the target three-dimensional hand postures of the target object; wherein, the reference static biomechanical features correspond one-to-one with the target object, and the reference static biomechanical features are determined by normalizing the static biomechanical features of each of the target three-dimensional hand postures of the target object; the static biomechanical features are used to characterize the palm features corresponding to the target three-dimensional hand posture; the dynamic biomechanical features are used to characterize the spatial angles of the fingers corresponding to the target three-dimensional hand posture; The first determining module is used to determine, for each target three-dimensional hand posture, the three-dimensional joint coordinates of the target three-dimensional hand posture and the corresponding two-dimensional joint coordinates of the three-dimensional joint coordinates based on the reference static biomechanical characteristics and the dynamic biomechanical characteristics; wherein, the three-dimensional joint coordinates characterize the spatial position of the hand joint points; The second determining module is used to determine human hand posture training data based on the three-dimensional joint coordinates and the corresponding two-dimensional joint coordinates.

8. The apparatus according to claim 7, characterized in that, The second acquisition module is further configured to acquire, for each of the target three-dimensional hand poses of the target object, the root bone vectors of the palm corresponding to the target three-dimensional hand pose; calculate the Euclidean distance based on each root bone vector to obtain the root bone length corresponding to the target three-dimensional hand pose; calculate the angle between the root bone vectors of adjacent root bones using an inverse cosine function to obtain the angle between root bones corresponding to the target three-dimensional hand pose; wherein the number of angles between root bones is less than the number of root bone vectors; determine the basis vectors based on the root bone vectors of adjacent root bones, and determine the palm curvature corresponding to the target three-dimensional hand pose based on the basis vectors; wherein the basis vectors represent the orthogonal vectors of adjacent root bone vectors.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images