Digital human hand posture generation method and device, storage medium and computer program product

Abstract

The application provides a digital human hand posture generation method and device, a storage medium and a computer program product. The method comprises the following steps: obtaining a rotation amount corresponding to each target finger joint of each finger of a digital human hand, wherein the target finger joint is obtained by preselecting a plurality of joints contained in each finger; determining a weight value of each target finger joint of each finger based on the rotation amount; and generating a target hand posture of the digital human based on the weight value and a preset hand posture.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, device, storage medium, and computer program product for generating digital human hand gestures. Background Technology

[0002] In the field of digital human motion capture, monocular cameras are one of the commonly used devices. They capture image information to recognize finger movements and postures, thereby driving the movement of the digital human's fingers. However, due to the limited image precision of monocular cameras, finger movements are prone to problems such as tremors, interlacing, and incoordination during the capture process. These problems directly affect the accuracy and naturalness of motion capture.

[0003] In related technologies, a common solution is to use post-processing to fine-tune and beautify the finger bones along specified axes after the digital human finger animation is generated. Specifically, this involves fine-tuning certain axes of the finger bones after the animation is generated to improve the appearance of finger movements. For example, when finger movements are uncoordinated, the rotation angle of the finger bones can be adjusted to make the fingers look more natural. However, this fine-tuning and beautification of the finger bones along specified axes has a fixed axial compensation and cannot be dynamically adjusted according to different driving scenarios, resulting in uncoordinated finger movements still occurring. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, device, storage medium, and computer program product for generating digital human hand postures, which can solve the problem of uncoordinated finger movements in digital humans.

[0005] The technical solution of this application is implemented as follows: In a first aspect, this application provides a method for generating digital human hand poses, the method comprising: The rotation amount corresponding to each target finger joint of each finger of a digital human hand is obtained, wherein the target finger joint is selected in advance from multiple joints contained in each finger; Based on the rotation amount, the weight value of each target finger joint of each finger is determined; Based on the weight values ​​and preset hand poses, the target hand pose of the digital human is generated.

[0006] Secondly, this application provides a digital human hand pose generation device, the digital human hand pose generation device comprising: The acquisition unit is used to acquire the rotation amount corresponding to each target finger joint of each finger of the digital human hand, wherein the target finger joint is selected in advance from multiple joints contained in each finger; A determining unit is used to determine the weight value of each target finger joint of each finger based on the rotation amount; The generation unit is used to generate the target hand pose of the digital human based on the weight value and the preset hand pose.

[0007] Thirdly, this application provides a digital human hand pose generation device, the device comprising: a processor and a memory; the processor executes the running program stored in the memory to implement the above-described digital human hand pose generation method.

[0008] Fourthly, this application provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the above-described digital human hand pose generation method.

[0009] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described digital human hand pose generation method.

[0010] This application provides a method, device, storage medium, and computer program product for generating digital human hand poses. The method includes: acquiring the rotation amount corresponding to each target finger joint of each finger of the digital human hand, wherein the target finger joints are pre-selected from multiple joints contained in each finger; determining the weight value of each target finger joint of each finger based on the rotation amount; and generating the target hand pose of the digital human based on the weight value and a preset hand pose. Using the above implementation scheme, for each finger of the digital human hand, the rotation amount of each target finger joint is first acquired. The rotation amount of each target finger joint is converted into a weight value for each target finger joint. The target hand pose is then generated by combining the weight value of each target finger joint with the preset hand pose. This scheme converts the specific rotation amount of each finger joint into a weight value of the preset hand pose. Because the weight value can ignore the error value caused by the rotation angle, the combination of the weight value and the preset hand pose results in more coordinated digital human finger movements. Furthermore, this method does not rely on fixed axial adjustment, making the adjustment method more flexible. Attached Figure Description

[0011] Figure 1 A schematic flowchart of a digital human hand pose generation method provided in an embodiment of this application; Figure 2 A schematic diagram of three finger joints of a finger provided for an embodiment of this application; Figure 3 A schematic diagram of a finger gripping posture provided in an embodiment of this application; Figure 4 A schematic diagram of a finger extended and closed posture provided in an embodiment of this application; Figure 5 A schematic diagram of a finger extended and spread posture provided in an embodiment of this application; Figure 6 A schematic diagram of the overall process of a digital human hand pose generation method provided in an embodiment of this application; Figure 7 A schematic diagram of the composition structure of a digital human hand pose generation device provided in this application embodiment. Figure 1 ; Figure 8 A schematic diagram of the composition structure of a digital human hand pose generation device provided in this application embodiment. Figure 2 . Detailed Implementation

[0012] To gain a more detailed understanding of the features and technical content of the embodiments of this application, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and specific embodiments. The accompanying drawings are for reference only and are not intended to limit the embodiments of this application.

[0013] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0014] In the following description, references to "some embodiments" refer to a subset of all possible embodiments. It is understood that "some embodiments" may be the same or different subsets of all possible embodiments and may be combined with each other without conflict. It should also be noted that the terms "first / second / third" used in the embodiments of this application are merely for distinguishing similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0015] In related technologies, when digital human finger movements become uncoordinated, the rotation angle of the finger bones is adjusted to make the digital human's fingers appear more natural. However, this approach has significant limitations: Post-processing fine-tunes and beautifies finger bones along specific axes, but its axial compensation is fixed and cannot be dynamically adjusted according to different driving scenarios. For example, in situations where finger movements change rapidly or the scene is complex, fixed axial compensation may not effectively improve the incoordination of finger movements.

[0016] Furthermore, among related technologies, finger motion capture has a high degree of freedom (e.g., a single hand of a digital human contains 15 joints, each with 3 rotational axes, totaling 45 degrees of freedom), resulting in a large workload for post-processing optimization. At the same time, too many degrees of freedom can also introduce interference factors, affecting the accuracy and naturalness of motion capture.

[0017] Therefore, in related technologies, finger movements are prone to interweaving and incoordination during the capture process, and current post-processing methods cannot fundamentally solve this problem.

[0018] This application provides a finger-driven post-processing optimization scheme to address the problems of tremor, interlacing, incoordination, and poor adaptability in finger motion capture using monocular cameras in related technologies. This application utilizes core technologies such as degree-of-freedom optimization (i.e., extracting only core degrees of freedom), inter-frame interpolation smoothing, and pose fusion to effectively solve these problems. By reducing degrees of freedom, computational resource consumption is lowered. Simultaneously, inter-frame interpolation smoothing and pose fusion improve the smoothness and naturalness of the motion. Therefore, this application not only effectively solves the aforementioned shortcomings but also significantly improves the accuracy and naturalness of motion capture while reducing computational resource consumption and labor costs. Furthermore, this application enhances the adaptability of the technology, automatically adapting to updates of different driving digital humans and detection SDKs without frequent optimization, thus reducing labor costs. This not only improves technical performance but also significantly enhances user experience and system versatility.

[0019] The detailed technical solution of this application is as follows: This application provides a method for generating digital human hand poses, such as... Figure 1 As shown, this method aims to address issues such as micro-tremors, interlacing, incoordination, and poor adaptability in finger motion capture caused by the limited image accuracy of a monocular camera. The method may include: S101. Obtain the rotation amount corresponding to each target finger joint of each finger of the digital human hand, wherein the target finger joint is selected in advance from multiple joints contained in each finger.

[0020] In this embodiment, the amount of rotation can be represented by an angle value.

[0021] In this embodiment, each finger has three joints from the palm to the fingertip (except for the thumb, which contains only two joints), namely the metacarpophalangeal joint (MCP), the proximal interphalangeal joint (PIP), and the distal interphalangeal joint (DIP). Figure 2 As shown.

[0022] The metacarpophalangeal joint (MCP) is the joint on the finger closest to the palm, connecting the metacarpal bones and the proximal phalanges. It gives the finger a wide range of movement capabilities; whether it's making a fist, extending the fingers, or other fine motor skills, it's indispensable.

[0023] Proximal interphalangeal joint (PIP): Located between the distal interphalangeal joint and the metacarpophalangeal joint, it connects the middle phalanx and the proximal phalanx. It also plays an important role in finger movement; for example, when we bend our fingers to make an "OK" gesture, this joint is involved.

[0024] The distal interphalangeal joint (DIP) is the joint closest to the fingertip, connecting the distal and middle phalanges. It is primarily responsible for finger flexion and extension, such as when we type on a keyboard or grasp objects; this joint is involved in fine motor skills.

[0025] The situation is slightly different for the thumb. Generally, only two joints are considered: the metacarpophalangeal joint and the interphalangeal joint.

[0026] The above finger joints and their corresponding range of motion are shown in Table 1 below: Table 1

[0027] Normally, the regular movements of the fingers mainly consist of the bending and swinging behaviors of each finger. To simplify the analysis, the key degrees of freedom of finger movement can be summarized as follows: For bending movements such as clenching a fist or half-clenching a fist, the bending angle is mainly described by the bending angle of the proximal interphalangeal joint (PIP) and metacarpophalangeal joint (MCP).

[0028] The degree of finger opening and closing is mainly measured by the extension angle of the metacarpophalangeal joint (MCP).

[0029] Therefore, the bending direction of the MCP, the bending direction of the PIP, and the extension (or opening / closing) direction of the MCP can be used as core degrees of freedom for rotational extraction.

[0030] It should be noted that, typically, each finger has 3 joints, and each joint has 3 rotational degrees of freedom (direction) in 3 dimensions (such as X, Y, Z), resulting in 9 rotational degrees of freedom for each finger. Too many degrees of freedom can easily lead to uncontrollable results. Therefore, this step extracts only 3 core degrees of freedom for each finger, resulting in 15 core degrees of freedom for 5 fingers, which can greatly reduce the amount of computation.

[0031] In this embodiment, the movements and postures of the human body are first acquired. Based on these movements and postures, the limbs and hands of the digital human are driven. Two joints (MCP and PIP) are selected from the three joints of each finger in advance. The three degrees of freedom (rotation direction) of the MCP are simplified to two degrees of freedom, and the three degrees of freedom of the PIP are simplified to one degree of freedom. The bending direction and extension direction of the MCP and the bending direction of the PIP are extracted as the three core degrees of freedom of each finger. That is, by reading the current posture information of the digital human's hand, the bending amount of each finger of the digital human can be detected. The bending amount includes the bending angle of the MCP and the bending angle of the PIP. The opening and closing amount of each finger of the digital human, i.e., the extension angle of the MCP, is also detected.

[0032] In this embodiment of the application, through this simplification, the number of degrees of freedom of finger movement can be reduced from the original 45 to 15, thereby significantly reducing the complexity of subsequent calculations.

[0033] It should be noted that the degrees of freedom described above refer to the directions in which each joint can rotate. In 3D, each joint can rotate in three directions: X, Y, and Z.

[0034] S102. Based on the rotation amount, determine the weight value of each target finger joint for each finger.

[0035] In this embodiment, after obtaining the bending and extension angles of the MCP and the bending angle of the PIP using the above method, the rotation data (rotation amount) of each finger joint needs to be normalized to obtain the weight values ​​for hand pose fusion. Simultaneously, considering that insufficient image precision may prevent fingers from fully extending or clenching, the original data range can be remapped to correct the original finger pose. After correction, the bending and extension angles of the MCP and the bending angle of the PIP are weighted to obtain the weight values ​​corresponding to the MCP and PIP for each finger.

[0036] In this embodiment of the application, the weight value of each finger joint is determined based on the rotation amount, which can be achieved in the following way: a first rotation amount is determined based on the rotation amount and a preset first rotation amount; the weight value of each finger joint is determined based on the first rotation amount and a preset rotation range value.

[0037] In the embodiments of this application, the rotation amount includes the bending angle and extension angle of the MCP and the bending angle of the PIP.

[0038] In this embodiment of the application, the first rotation amount is preset to be the minimum angle value of MCP or the minimum angle value of PIP.

[0039] In this embodiment, the preset rotation range value is the angle range of MCP or the angle range of PIP.

[0040] In this embodiment of the application, the weight transformation formula can be implemented using the following formula (1): (1) Among them, DetectAngle is the detected current joint angle, which includes the angle data of the three core degrees of freedom of each finger extracted earlier, such as the bending angle and extension angle (opening and closing angle) of MCP, and the bending angle of PIP.

[0041] DetectMinAngle is the minimum angle detected at the joint (the lower limit used to define the range), including the lower limit of the bending angle of the MCP, the lower limit of the bending angle of the PIP, and the lower limit of the extension angle of the MCP.

[0042] DetectAngleRange is the range of angles detected in the joint (i.e., the difference between the maximum and minimum angles). Similarly, it also includes the flexion angle range and extension angle range corresponding to MCP and PIP, respectively.

[0043] It should be noted that DetectMinAngle and DetectAngleRange are presets and can be adjusted according to the version of the algorithm model. When the version of the algorithm model is fixed, the values ​​of DetectMinAngle and DetectAngleRange are fixed.

[0044] Clamp01 is a function that restricts the weight results to the interval [0,1], ensuring that the weight values ​​are always within a reasonable range.

[0045] Using the above formula (1), the angle data of each target finger joint can be normalized to a weight value in the interval [0,1], thereby effectively solving the problem of joint angle deviation caused by insufficient image precision.

[0046] It should be noted that the above formula (1) can be called the interval transformation process.

[0047] It should be noted that the weight values ​​corresponding to the two joints of the thumb are also calculated using the above formula (1).

[0048] It's important to note that finger detection involves first detecting the hand bounding box in the portrait image, then separately detecting the fingers within the hand region. However, this initial hand region image resolution is too low, resulting in insufficient image accuracy. Furthermore, since the rotation angles of multiple fingers are detected independently, even with all five fingers extended, the MCP (Mean Collateral Chance Point) of some fingers may not be entirely zero at the bending angle, leading to unsightly overlapping of fingers. Normalizing the specific joint rotation angles into preset animation rotation weights allows for smoothing out minor differences among multiple weights, making the weights of adjacent fingers more consistent, reducing small differences between adjacent fingers, and improving the aesthetics of fingers in digital human-driven animation.

[0049] In this embodiment, the bending weight value and extension weight value of the MCP and the bending weight value of the PIP for each finger can be calculated using the above formula (1). However, since the weight values ​​are calculated independently for each finger, the finger movements are completely independent, making gestures such as grasping appear unnatural and unattractive. To improve this problem, the weights of the fingers other than the thumb can be statistically analyzed, and the fingers with smaller weight deviations can be grouped together. Within each group, the weights are evenly distributed to make the finger movements more coordinated.

[0050] To achieve the above-mentioned technical effects, in this embodiment of the application, after determining the weight values ​​of each target finger joint of each finger based on the rotation amount, the following process can also be performed: For the first weight value among multiple weight values, a weight deviation value is determined based on the first weight value corresponding to the first target finger joint of each finger; the first target finger joints of all fingers are grouped based on the weight deviation value to obtain one or more groups of target finger joints; for each group of target finger joints, the updated weight value of the first target finger joint of each finger is determined based on the first weight value corresponding to each first target finger joint in each group of target finger joints.

[0051] In this embodiment, actual bending and opening / closing occur simultaneously, and the bending of each finger is determined by the bending angle of the MCP plus the bending angle of the PIP. To ensure that all fingers are aesthetically pleasing with the same degree of bending and to smooth out small angle differences, the above-mentioned technical effect can be achieved by weighting. When performing weighting calculation, this embodiment selects to distribute the weight value of the first target finger joint (i.e., MCP). Since multiple weight values ​​are calculated for MCP, including the bending weight value and the extension weight value, this embodiment considers distributing the bending weight value of MCP (i.e., the first weight value).

[0052] In this embodiment of the application, the specific implementation method for determining the weight deviation is as follows: In a computational model that calculates weights independently for each finger, each finger receives a scalar. (That is, the MCP bending weight value of each finger, which usually represents the contribution of that finger to the current posture or gesture).

[0053] If the calculated MCP flexion weight values ​​for the five fingers are respectively The average of these 5 weight values ​​is calculated using the following formula (2): (2) The weighted deviation value corresponding to the MCP of the i-th finger is defined as shown in the following formula (3): (3) in, This represents the bending weight value of the MCP of the i-th finger. The smaller the value, the closer the weight of that finger is to the overall average level; The larger it is, the more "unconventional" it is, and the more it deviates from the overall average level.

[0054] In this embodiment, the grip appears unnatural because of the movement of certain fingers. The large size of these components leads to their independent operation; therefore, in this embodiment, further based on... The values ​​are grouped among all fingers, with the thumb kept separately during the grouping. You can choose not to participate in the grouping, and use the other four fingers to press. Sort in ascending order and set grouping thresholds. (If empirical values ​​are used,) Take 0.05).

[0055] In the embodiments of this application, if All less than If so, then all four fingers are grouped together and denoted as G.

[0056] like or Greater than or equal to Then The smallest first k fingers ( ) is assigned to G, and the rest remain independent.

[0057] For example, =0.18, =0.20, =0.19, =0.21, =0.195, then =0.015, =0.005, =0.005, =0.015. It can be seen that... All less than =0.05, and finally, all four fingers are assigned to G.

[0058] In this embodiment of the application, after grouping, the bending weight value of the MCP in each group of target finger joints is spread out to determine the new weight value of each MCP. Specifically, the first weight value corresponding to each of the multiple first target finger joints in each group is processed to obtain the second weight value. The second weight value is then determined as the updated weight value of the first target finger joint of each finger.

[0059] In this embodiment of the application, after completing the above grouping, the bending weight value of MCP in each group can be weighted evenly, and the specific implementation method is as follows: Assume there are m fingers in group G, and the original MCP bending weights for the m fingers are... .

[0060] The within-group mean is calculated as follows: , where m is the number of MCP bending weight values ​​in each group.

[0061] It should be noted that the subscript of w in this formula can simply be the group number.

[0062] Continuing with the numerical examples in the preceding embodiments: G contains 4 fingers, then The updated bending weight value of MCP is .

[0063] The final MCP bending weight value for each finger is then: .

[0064] For example, the original MCP bending weight values ​​are: [0.30, 0.18, 0.20, 0.19, 0.21]. Obviously, there are significant differences between the fingers.

[0065] The flattened MCP bending weight values ​​are: [0.30, 0.195, 0.195, 0.195, 0.195]. This allows for synchronized four-finger gripping, resulting in a more natural grasping motion.

[0066] In this embodiment, the bending weight value of the MCP is determined based on the bending angle of the MCP. After the above processing, the averaged MCP bending weight value is obtained. The calculated averaged MCP bending weight value can replace the originally calculated MCP bending weight value and be used in the subsequent process of fusing with the preset finger posture to generate the target gesture posture.

[0067] It should be noted that after calculating the bending weight of the PIP, the actual bending weight of the PIP can be determined by multiplying the calculated bending weight of the PIP by the bending weight of the MCP, while the bending weight of the DIP can directly use the actual bending weight of the PIP. Since each finger independently manages opening and closing (i.e., left and right swinging), if they are complementary, it will affect the aesthetics. Therefore, it is not necessary to distribute the extension weight value of the MCP.

[0068] In this embodiment, to alleviate the hand gesture recognition jitter problem caused by insufficient image accuracy, a time-series-based smoothing mechanism can be introduced based on the aforementioned embodiments. Specifically, by combining the weight values ​​of each target finger joint calculated from the rotation amount of the current frame with the weight values ​​of the previous frame, a weighted average is used to smooth the weight values, thereby reducing instantaneous fluctuations caused by image noise or detection accuracy issues, and making hand gesture changes smoother and more natural.

[0069] In this embodiment of the application, the time-series-based smoothing mechanism is specifically implemented as follows: the weight value is calculated by the rotation amount of each target finger joint in the current image frame. After determining the weight value of each target finger joint of each finger based on the rotation amount, the method further includes: obtaining the historical weight value of each target finger joint of each finger in the previous image frame of the current image frame, wherein the historical weight value is obtained by smoothing the weight value of each target finger joint of each finger in the previous image frame; and determining the smoothing weight value of each target finger joint of each finger based on the historical weight value, the weight value, the first weight factor corresponding to the historical weight value, and the second weight factor corresponding to the weight value.

[0070] In this embodiment, the first weighting factor and the second weighting factor can be preset. Specifically, they can be selected according to the actual situation. No specific limitation is made in this embodiment.

[0071] In this embodiment, after calculating the bending weight value and extension weight value of MCP, and the bending weight value of PIP, the weight values ​​can be smoothed based on a time series smoothing mechanism. The specific smoothing method can be implemented using the following formula (4): (4) Wherein, CurrentFrameWeight: The weight value calculated for the rotation of each target finger joint in the current frame.

[0072] LastFrameWeight: The weight values ​​of each target finger joint after smoothing in the previous frame.

[0073] SmoothParam: The smoothing parameter, also known as the weight factor, has a value range of [0,1]. This parameter determines the proportion of the current frame's weight value in the smoothing result. The closer the value is to 1, the greater the influence of the current frame and the weaker the smoothing effect; the closer the value is to 0, the greater the influence of the previous frame and the stronger the smoothing effect.

[0074] It should be noted that, This is the first weighting factor in the embodiments of this application. This is the second weighting factor in the embodiments of this application.

[0075] In this embodiment of the application, formula (4) can be used to calculate the bending weight value of MCP, the stretching weight value, and the weight value of PIP after smoothing.

[0076] For example, the current frame is the 2nd frame, and the bending weight value of the MCP in this frame is a. a can be the original bending weight value calculated. Then, the bending weight value of the MCP after smoothing in the 1st frame is obtained. The bending weight value of the MCP after smoothing in the 2nd frame can be calculated using the above formula (4). Similarly, the stretching weight value of the MCP after smoothing in the 2nd frame and the bending weight value of the PIP after smoothing can be calculated.

[0077] It should be noted that if it is the first frame of the image, the weight values ​​after smoothing can be directly replaced by the weight values ​​corresponding to the calculated MCP and PIP of the first frame.

[0078] In this embodiment of the application, if the bending weight value of the MCP is the weight value after weight amortization, then when the above formula (4) is used for calculation, the bending weight value of the MCP in the current frame is the weight value after weight amortization. The bending weight value of the smoothed MCP in the previous frame can be the bending weight value of the smoothed and weight-amortized MCP.

[0079] It should be noted that other weight values ​​are similar and will not be elaborated on here.

[0080] In this embodiment, by introducing temporal smoothing, we can effectively suppress the gesture weight jitter problem caused by insufficient image precision. Specifically, the weight of each frame is no longer an independently calculated result, but a weighted average of the weights of the current frame and the previous frame. This processing method retains the information of the current frame while using the weight of the previous frame as a reference, thereby smoothing the weight changes over time and making the gesture movements more fluid and natural.

[0081] In this embodiment of the application, the smoothed weight value can be used to fuse with a preset hand pose to generate the target hand pose of the digital human.

[0082] S103. Based on the weight values ​​and preset hand poses, generate the target hand pose of the digital human.

[0083] In this embodiment of the application, in order to generate the final hand pose for each style of digital human, the art team provided the following three key preset hand poses for each style: fingers clenched, such as... Figure 3 As shown; fingers straight and closed, as Figure 4 As shown; fingers straight and spread, as Figure 5 As shown.

[0084] In this embodiment of the application, there are multiple preset hand poses. Based on the weight values ​​and preset hand poses, the target hand pose of the digital human is generated. Specifically, the target hand pose of the digital human can be generated by fusing multiple preset hand poses based on the weight values ​​corresponding to each target finger joint of each finger.

[0085] In this embodiment of the application, based on the previously calculated finger bending weight values ​​(including the bending weight values ​​of MCP and PIP) and extension (opening and closing) weight values ​​(including the extension weight value of MCP), the preset finger clenching, finger straightening and closing, and finger straightening and opening posture information are fused to obtain the final hand pose.

[0086] In the embodiments of this application, the bending weight value and extension weight value of MCP and the bending weight value of PIP can be obtained by directly calculating the weight value. In addition, the bending weight value of MCP can also be the weight value of MCP after weight amortization and smoothing. The bending weight value of PIP can also be the bending weight value of PIP after smoothing the actual bending weight value. The extension weight value of MCP can also be the extension weight value of MCP after smoothing.

[0087] It should be noted that the fusion method is the same regardless of whether the weight values ​​obtained by direct calculation are used to fuse the three preset hand poses or the weight values ​​obtained by smoothing are used to fuse the three preset hand poses. The following embodiment only uses the bending weight value and extension weight value of MCP and the bending weight value of PIP obtained by direct calculation to fuse and generate the target hand pose as an example.

[0088] In this embodiment, there are multiple target finger joints, each target finger joint corresponds to one or more rotation amounts, and each rotation amount corresponds to a weight value. Based on the weight values ​​corresponding to each target finger joint of each finger, multiple preset hand poses are fused to generate the target hand pose of the digital human. Specifically, this can be achieved as follows: based on the third weight value corresponding to the first rotation amount of the first target finger joint, multiple first preset hand poses are fused to obtain a fused pose; based on the fourth weight value corresponding to the second rotation amount of the first target finger joint and the fifth weight value corresponding to the second rotation amount of the second target finger joint, the fused pose and the second preset hand pose are fused to generate the target hand pose of the digital human.

[0089] In this embodiment of the application, the target finger joint includes MCP and PIP. MCP corresponds to the bending angle and extension angle, and PIP corresponds to the bending angle. The bending angle of MCP corresponds to the bending weight value, the extension angle of MCP corresponds to the extension weight value, and the bending angle of PIP corresponds to the bending weight value.

[0090] In this embodiment, the first target finger joint is MCP, the first rotation amount is the extension angle, and therefore the third weight value is the extension weight value.

[0091] In this embodiment of the application, the second rotation amount is the bending angle, and the fourth weight value is the bending weight value of MCP.

[0092] In this embodiment of the application, the second target finger joint is PIP, and the fifth weight value is the bending weight value of PIP.

[0093] In this embodiment of the application, when performing specific fusion, the extension fusion pose is generated first. Specifically, the two preset hand poses, "finger straight and closed" and "finger straight and open", are fused through extension (opening and closing) weights (i.e., the extension weights of MCP) to obtain the intermediate extension fusion pose, wherein the extension (opening and closing) weights reflect the degree of the fingers from closed to open. The formula for obtaining the extension fusion pose is shown in the following formula (5): FingerStraightQuat=Slerp(FingerStraightAndClosedQuat,FingerStraightAndSpreadQuat,SpreadWeight)(5) Here, Slerp represents the quaternion spherical interpolation linear interpolation function.

[0094] FingerStraightQuat represents the rotation quaternion of the finger bones after fusion. Since the fusion involves the closed finger pose and the open finger pose, the fingers only have extended and open states.

[0095] FingerStraightAndClosedQuat represents the rotation quaternion of each bone in the preset straight and closed pose.

[0096] FingerStraightAndSpreadQuat represents the rotation quaternion of each bone in the preset straight and spread pose.

[0097] SpreadWeight represents the opening and closing weights of the MCP calculated above.

[0098] In this embodiment, after fusing the two preset hand postures "finger extended and closed" and "finger extended and open" using the above formula (5), the final target hand posture is further generated. Specifically, the generated extended fusion pose and the preset hand posture "finger clenched" are fused using a bending weight value to obtain the final target hand pose, where the bending weight determines the degree of bending of the fingers from extended to clenched. Specifically, it can be calculated using the following formula (6): FingerFinalQuat=Slerp(FingerStraightQuat,FingerClenchedQuat,ClenchedWeight) (6) Wherein, Slerp: quaternion spherical interpolation linear interpolation function.

[0099] FingerFinalQuat: The rotational quaternion of the fused finger bones, which now possesses opening and bending states.

[0100] FingerStraightQuat: The rotational quaternion for each bone in the Stretch Fusion Pose.

[0101] FingerClenchedQuat: Preset quaternion for the rotation of the corresponding bone in the finger gripping pose.

[0102] ClenchedWeight: Finger flexion weight value, including MCP flexion weight value and PIP flexion weight value.

[0103] After the above fusion process, the output FingerFinalQuat is the final target hand pose.

[0104] In this embodiment of the application, by using this step-by-step fusion method, the posture of the fingers can be precisely controlled by utilizing the extension weight value and the bending weight value, thereby achieving a smooth transition from full extension to full grip and generating the expected final posture.

[0105] It should be noted that the preset finger pose input in this embodiment is fixed and aesthetically pleasing. Interpolation between two preset finger poses yields an intermediate state that is also aesthetically pleasing; for example, five fingers extended and five fingers closed can be mixed with a weight of 0.5 to achieve a half-open / half-closed state. The same principle applies to the sequential merging of three preset hand poses. Furthermore, reducing the degrees of freedom satisfies most scenarios while avoiding small-angle shifts and aesthetic degradation caused by insufficient algorithm data precision.

[0106] It is understood that the digital human hand pose generation method provided in this application, for each finger of the digital human hand, first obtains the rotation amount of each target finger joint of each finger, converts the rotation amount of each target finger joint into a weight value of each target finger joint, and generates the target hand pose by combining the weight value of each target finger joint with a preset hand pose. This scheme converts the specific rotation amount of each finger joint into a weight value of the preset hand pose. Because the weight value can ignore the error value caused by the rotation angle, the combination of the weight value and the preset hand pose results in more coordinated digital human finger movements; moreover, this method does not rely on fixed axis adjustment, making the adjustment method more flexible.

[0107] Based on the above embodiments, this application also provides a schematic diagram of the overall process of a digital human hand pose generation method, as shown below. Figure 6 As shown, the specific steps include: S1, Image Detection.

[0108] S2. Based on image detection of hand pose, drive the movement of limbs and fingers of digital humans.

[0109] S3. Extract the three core degrees of freedom of each target finger joint of each finger in the digital human hand.

[0110] S4. Detect the amount (angle) of finger bending based on core degrees of freedom.

[0111] S5. Perform interval transformation, weight amortization, and inter-frame smoothing on the finger bending amount to obtain the smoothed bending weight value.

[0112] S6. Detect finger extension (angle) based on core degrees of freedom.

[0113] S7. Perform interval conversion and inter-frame smoothing on the finger extension amount to obtain the smoothed extension weight value.

[0114] It should be noted that S4-S5 and S6-S7 are executed in parallel.

[0115] After S7, S8 is executed, which involves fusing the extended and closed fingers with the fingers extended and closed based on the smoothed extension weight values ​​to obtain the fused posture.

[0116] S9. The fused posture and the five-finger clenched posture are fused based on the smoothed bending weight value to obtain the target hand posture.

[0117] The optimized result (i.e., the target finger pose) is output after obtaining the above optimization result.

[0118] It should be noted that steps S1 and S2 can be implemented with reference to relevant technologies, and will not be elaborated here.

[0119] Based on the above embodiments, the key technologies in this application can be summarized as follows: 1. Degrees of freedom optimization This application's embodiments extract the most representative core degrees of freedom by deeply analyzing the motion characteristics of finger movements. For example, the bending angle (opening angle) of the first joint closest to the palm for each finger can represent the activity information of each finger. This optimization not only preserves the key information of finger movements but also significantly simplifies the model complexity.

[0120] 2. Inter-frame interpolation smoothing technique This application's embodiments introduce inter-frame interpolation smoothing technology, combined with interval constraints, to dynamically calculate the hybrid weights of core joints, effectively reducing finger tremors and making movements smoother and more natural.

[0121] 3. Pose fusion technology This application's embodiments generate more natural finger movements by blending three built-in finger poses (five fingers open, five fingers closed, and five fingers clenched into a fist). By dynamically calculating the blending weights, finger overlap and incoordination are avoided.

[0122] Based on the above embodiments, another embodiment of this application provides a digital human hand pose generation device 1, such as... Figure 7 As shown, the digital human hand pose generation device 1 includes: The acquisition unit 10 is used to acquire the rotation amount corresponding to each target finger joint of each finger of the digital human hand, wherein the target finger joint is selected in advance from multiple joints contained in each finger.

[0123] The determining unit 11 is used to determine the weight value of each target finger joint of each finger based on the rotation amount.

[0124] The generation unit 12 is used to generate the target hand posture of the digital human based on the weight value and the preset hand posture.

[0125] In one embodiment, the determining unit 11 is further configured to determine a first rotation amount based on the rotation amount and a preset first rotation amount.

[0126] The determining unit 11 is also used to determine the weight value of each finger joint based on the first rotation amount and the preset rotation range value.

[0127] In one embodiment, the target finger joint includes a first target finger joint, which corresponds to multiple weight values.

[0128] The determining unit 11 is further configured to, for a first weight value among multiple weight values, determine a weight deviation value based on the first weight value corresponding to the first target finger joint of each finger; group the first target finger joints of all fingers based on the weight deviation value to obtain one or more groups of target finger joints; and for each group of target finger joints, determine the updated weight value of the first target finger joint of each finger based on the first weight value corresponding to each first target finger joint in each group of target finger joints.

[0129] In one embodiment, the determining unit 11 is further configured to perform calculations on the first weight values ​​corresponding to the plurality of first target finger joints in each group to obtain a second weight value; and to determine the second weight value as the updated weight value of the first target finger joint of each finger.

[0130] In one embodiment, the number of preset hand gestures is multiple.

[0131] The generation unit 12 is also used to perform fusion processing on multiple preset hand postures based on the weight values ​​corresponding to each target finger joint of each finger, so as to generate the target hand posture of the digital human.

[0132] In one embodiment, there are multiple target finger joints, each target finger joint corresponds to one or more rotation amounts, and each rotation amount corresponds to a weight value.

[0133] The generation unit 12 is further configured to fuse multiple first preset hand postures based on the third weight value corresponding to the first rotation amount of the first target finger joint to obtain a fused posture; and to fuse the fused posture and the second preset hand posture based on the fourth weight value corresponding to the second rotation amount of the first target finger joint and the fifth weight value corresponding to the second rotation amount of the second target finger joint to generate the target hand posture of the digital human.

[0134] In one embodiment, the weight value is calculated by measuring the rotation of each target finger joint in the current image frame.

[0135] The acquisition unit 10 is further configured to acquire the historical weight values ​​of each target finger joint of each finger in the previous image frame of the current image frame, wherein the historical weight values ​​are obtained by smoothing the weight values ​​of each target finger joint of each finger in the previous image frame.

[0136] The determining unit 11 is further configured to determine the smoothing weight value of each target finger joint of each finger based on the historical weight value, the weight value, the first weight factor corresponding to the historical weight value, and the second weight factor corresponding to the weight value.

[0137] This application provides a digital human hand pose generation device. It acquires the rotational amount of each target finger joint of each finger in a digital human hand, wherein the target finger joints are pre-selected from multiple joints contained in each finger. Based on the rotational amount, a weight value is determined for each target finger joint of each finger. Based on the weight values ​​and a preset hand pose, the target hand pose of the digital human is generated. Therefore, this digital human hand pose generation device, for each finger of a digital human hand, first acquires the rotational amount of each target finger joint, converts the rotational amount of each target finger joint into a weight value, and generates the target hand pose by combining the weight values ​​of each target finger joint with a preset hand pose. This solution converts the specific rotational amount of each finger joint into a weight value of the preset hand pose. Because the weight value can ignore the error value caused by the rotation angle, the combination of the weight value and the preset hand pose results in more coordinated digital human finger movements. Furthermore, this method does not rely on fixed axial adjustment, making the adjustment method more flexible.

[0138] Figure 8 This is a schematic diagram of the composition structure of a digital human hand pose generation device 1 provided in an embodiment of this application. In practical applications, based on the same disclosed concept of the above embodiments, such as...Figure 8 As shown, the digital human hand pose generation device 1 of this application embodiment includes a processor 13, a memory 14 and a communication bus 15.

[0139] In specific embodiments, the acquisition unit 10, determination unit 11, and generation unit 12 described above can be implemented by a processor 13 located on the digital human hand posture generation device 1. The processor 13 can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), CPU, controller, microcontroller, and microprocessor. It is understood that for different devices, the electronic device used to implement the above processor functions can also be other types, and this application embodiment does not impose specific limitations.

[0140] In this embodiment, the communication bus 15 is used to establish a connection between the processor 13 and the memory 14; when the processor 13 executes the running program stored in the memory 14, it implements the following digital human hand pose generation method: The rotation amount corresponding to each target finger joint of each finger of the digital human hand is obtained, wherein the target finger joint is selected in advance from multiple joints contained in each finger; based on the rotation amount, the weight value of each target finger joint of each finger is determined; based on the weight value and the preset hand posture, the target hand posture of the digital human is generated.

[0141] In one embodiment, the processor 13 is further configured to determine a first rotation amount based on the rotation amount and a preset first rotation amount; and to determine the weight value of each finger joint based on the first rotation amount and a preset rotation range value.

[0142] In one embodiment, the target finger joint includes a first target finger joint, which corresponds to multiple weight values.

[0143] The processor 13 described above is further configured to determine a weight deviation value based on the first weight value corresponding to the first target finger joint of each finger for a first weight value among multiple weight values; group the first target finger joints of all fingers based on the weight deviation value to obtain one or more groups of target finger joints; and for each group of target finger joints, determine the updated weight value of the first target finger joint of each finger based on the first weight value corresponding to each first target finger joint in each group of target finger joints.

[0144] In one embodiment, the processor 13 is further configured to perform calculations on the first weight values ​​corresponding to the plurality of first target finger joints in each group to obtain second weight values; and to determine the second weight values ​​as the updated weight values ​​for the first target finger joints of each finger.

[0145] In one embodiment, the number of preset hand gestures is multiple.

[0146] The processor 13 is further configured to perform fusion processing on multiple preset hand poses based on the weight values ​​corresponding to each target finger joint of each finger, and generate the target hand pose of the digital human.

[0147] In one embodiment, there are multiple target finger joints, each target finger joint corresponds to one or more rotation amounts, and each rotation amount corresponds to a weight value.

[0148] The processor 13 is further configured to fuse multiple first preset hand postures based on a third weight value corresponding to a first rotation amount of a first target finger joint to obtain a fused posture; and to fuse the fused posture and the second preset hand posture based on a fourth weight value corresponding to a second rotation amount of the first target finger joint and a fifth weight value corresponding to a second rotation amount of the second target finger joint to generate the target hand posture of the digital human.

[0149] In one embodiment, the weight value is calculated by measuring the rotation of each target finger joint in the current image frame.

[0150] The processor 13 is further configured to obtain the historical weight values ​​of each target finger joint of each finger in the previous image frame of the current image frame, wherein the historical weight values ​​are obtained by smoothing the weight values ​​of each target finger joint of each finger in the previous image frame; and determine the smoothed weight values ​​of each target finger joint of each finger based on the historical weight values, the weight values, the first weight factor corresponding to the historical weight values, and the second weight factor corresponding to the weight values.

[0151] Based on the above embodiments, this application provides a storage medium storing a computer program thereon. The computer-readable storage medium stores one or more programs, which can be executed by one or more processors and applied in a digital human hand pose generation device. The computer program implements the digital human hand pose generation method as described above.

[0152] Based on the above embodiments, this application provides a computer program product, including a computer program that can be executed by one or more processors and applied in a digital human hand pose generation device. The computer program implements the digital human hand pose generation method as described above.

[0153] It should be noted that, in the embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the embodiments of this application.

[0155] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for generating digital human hand poses, characterized in that, The method includes: The rotation amount corresponding to each target finger joint of each finger of a digital human hand is obtained, wherein the target finger joint is selected in advance from multiple joints contained in each finger; Based on the rotation amount, the weight value of each target finger joint of each finger is determined; Based on the weight values ​​and preset hand poses, the target hand pose of the digital human is generated.

2. The method according to claim 1, characterized in that, Determining the weight value of each finger joint based on the rotation amount includes: The first rotation amount is determined based on the rotation amount and the preset first rotation amount; Based on the first rotation amount and the preset rotation range value, the weight value of each finger joint is determined.

3. The method according to claim 1, characterized in that, The target finger joint includes a first target finger joint, which corresponds to multiple weight values. After determining the weight values ​​of each target finger joint for each finger based on the rotation amount, the method further includes: For the first weight value among multiple weight values, a weight deviation value is determined based on the first weight value corresponding to the first target finger joint of each finger; Based on the weighted deviation value, the first target finger joints of all fingers are grouped to obtain one or more groups of target finger joints; For each group of target finger joints, based on the first weight value corresponding to each first target finger joint in each group of target finger joints, the updated weight value of the first target finger joint of each finger is determined.

4. The method according to claim 3, characterized in that, The step of determining the updated weight value of the first target finger joint for each finger based on the first weight value corresponding to each first target finger joint in each group of target finger joints includes: The first weight values ​​corresponding to multiple first target finger joints in each group are calculated and processed to obtain the second weight values; The second weight value is determined as the updated weight value of the first target finger joint for each finger.

5. The method according to claim 1, characterized in that, The number of preset hand poses is multiple, and the generation of the target hand pose of the digital human based on the weight value and the preset hand poses includes: Based on the weight values ​​corresponding to each target finger joint of each finger, multiple preset hand poses are fused to generate the target hand pose of the digital human.

6. The method according to claim 5, characterized in that, The number of target finger joints is multiple, each target finger joint corresponds to one or more rotation amounts, and each rotation amount corresponds to a weight value; the process of fusing multiple preset hand poses based on the weight values ​​corresponding to each target finger joint of each finger to generate the target hand pose of the digital human includes: Based on the third weight value corresponding to the first rotation amount of the first target finger joint, multiple first preset hand postures are fused to obtain a fused posture; Based on the fourth weight value corresponding to the second rotation amount of the first target finger joint and the fifth weight value corresponding to the second rotation amount of the second target finger joint, the fused posture and the second preset hand posture are fused to generate the target hand posture of the digital human.

7. The method according to claim 1, characterized in that, The weight values ​​are calculated by taking the rotation of each target finger joint in the current image frame. After determining the weight values ​​of each target finger joint for each finger based on the rotation, the method further includes: Obtain the historical weight values ​​of each target finger joint of each finger in the previous image frame of the current image frame, wherein the historical weight values ​​are obtained after smoothing the weight values ​​of each target finger joint of each finger in the previous image frame. Based on the historical weight values, the weight values, the first weight factor corresponding to the historical weight values, and the second weight factor corresponding to the weight values, the smoothing weight values ​​of each target finger joint of each finger are determined.

8. A digital human hand pose generation device, characterized in that, The digital human hand pose generation device includes a processor and a memory; the processor executes the running program stored in the memory to implement the method as described in any one of claims 1 to 7.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1 to 7.

Images