Gesture processing method and device

Through the rotation and scaling of quaternions and hand scale coefficients, combined with reverse kinematics, the joint position of virtual gestures is adjusted, and the problems of abnormal and unnatural gestures in the existing technology are solved, realizing the natural presentation and action transition of virtual gestures smooth.

CN118915904APending Publication Date: 2024-11-08NANCHANG VIRTUAL REALITY RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410110272.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-01-26
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing three-dimensional gesture estimation method based on vision causes abnormal three-dimensional gesture estimation when the hand occlusion or two-dimensional joint node detection accuracy, resulting in unnatural VR virtual gestures and unsmooth movement transitions.

Method used

By determining the quaternion and hand scale coefficient between the virtual gesture palm and the actual gesture palm, rotating and scaling, combined with the reverse kinematics method, each joint of the virtual gesture is adjusted until the target fingertip position is reached. Make sure that the virtual gestures are consistent with the actual gestures.

Benefits of technology

The natural presentation of virtual gestures and smoothing of motion transitions is achieved, and the problems of abnormal and unnatural gestures in the prior art are solved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915904A_ABST
    Figure CN118915904A_ABST
Patent Text Reader

Abstract

According to the gesture processing method and device provided by the embodiment of the invention, rotation and scale transformation are performed on the virtual gesture according to the actual gesture, and meanwhile, an inverse kinematics method is adopted to adjust each joint of the virtual gesture from the root joint until the target fingertip pose is reached; and then the pose and joint axis angle information of the virtual gesture is updated for driving the virtual gesture, so that the virtual hand shape is more natural, and the action transition is smooth.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of virtual gesture processing technology, and more particularly to a gesture posture processing method and device. Background Art

[0002] Existing vision-based 3D hand gesture estimation methods generally use hand gesture 2D joint points to estimate 3D hand gesture joint points. However, when the hand is occluded or the detection accuracy of some 2D joint points is not high, the reconstructed 3D hand gesture estimation may be abnormal, and thus the joint axis angles may also be abnormal. Abnormal joint data drives VR virtual hand gestures, causing VR virtual hand gestures to appear unnatural and the movement transition to be unsmooth. Summary of the invention

[0003] In order to solve or alleviate the problems in the prior art, in a first aspect, an embodiment of the present application provides a gesture processing method, comprising:

[0004] Determine a first quaternion between the virtual gesture palm and the actual gesture palm;

[0005] Calculating a hand shape scale coefficient between the virtual gesture palm and the actual gesture palm;

[0006] Scaling the actual gesture by the hand scale coefficient to obtain the target fingertip position, and rotating the virtual gesture according to the first quaternion so that the virtual gesture palm is in the same direction as the actual gesture palm;

[0007] Calculating a wrist joint translation vector between a virtual gesture and an actual gesture, and moving the entire virtual gesture to the wrist joint of the actual gesture according to the wrist joint translation vector;

[0008] Rotate the second quaternion corresponding to each joint of each finger of the virtual gesture starting from the finger root joint in turn, and each rotation requires judging whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint complies with the rotation axis and rotation angle constraints of the human hand skeleton; if all comply, stop rotating the next-level joint of the current finger, and then obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if all do not comply, continue rotating the next-level joint of the current finger.

[0009] Compared with the prior art, the embodiment of the present application provides a gesture posture processing method, which rotates and scales the virtual gesture according to the actual gesture, and adopts the inverse kinematics method to adjust each joint of the virtual gesture starting from the root joint until the target fingertip posture is reached, and then updates the posture and joint axis angle information of the virtual gesture to drive the virtual gesture, so that the virtual hand shape appears more natural and the movement transition is smooth.

[0010] In a second aspect, an embodiment of the present application further provides a gesture posture processing device, including:

[0011] A determination module, used for determining a first quaternion between a virtual gesture palm and an actual gesture palm;

[0012] A calculation module, used for calculating the hand shape scale coefficient between the virtual gesture palm and the actual gesture palm;

[0013] The rotation module is further used to scale the actual gesture by the hand scale coefficient to obtain the target fingertip position, and rotate the virtual gesture according to the first quaternion so that the virtual gesture palm is in the same direction as the actual gesture palm;

[0014] The calculation module is further used to calculate the wrist joint point translation vector between the virtual gesture and the actual gesture, and move the virtual gesture as a whole to the wrist joint of the actual gesture according to the wrist joint point translation vector;

[0015] The rotation module is also used to rotate the second quaternion corresponding to each joint of each finger of the virtual gesture in turn starting from the finger root joint, and each rotation needs to determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint complies with the rotation axis and rotation angle constraints of the human hand skeleton; if all are met, stop rotating the next level joint of the current finger, and then obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if all are not met, continue rotating the next level joint of the current finger.

[0016] Compared with the prior art, the beneficial effects of the gesture posture processing device provided in the embodiment of the present application are the same as the beneficial effects of the technical solution provided in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. Some specific embodiments of the present application will be described in detail in an illustrative and non-restrictive manner with reference to the drawings. The same reference numerals in the drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the drawings:

[0018] Figure 1 It is a flowchart of a gesture posture processing method provided in an embodiment of the present application;

[0019] Figure 2 It is a schematic diagram of the structure of the inverse kinematics iteration process provided in an embodiment of the present application;

[0020] Figure 3 This is a schematic diagram of the joint numbering of a human hand provided in an embodiment of the present application;

[0021] Figure 4 It is a structural schematic diagram of a gesture posture processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0023] The embodiments of the present application are mainly used for virtual reality technology (abbreviated as: VR). By processing the three-dimensional coordinates of actual gesture joints and optimizing the joint posture information, the VR virtual gesture is driven by the joint information so that the posture of the VR virtual gesture is consistent with the actual gesture. This can make the VR virtual hand shape appear more natural, especially the transition of the occluding gesture action is smoother.

[0024] First, as Figure 1 As shown, the embodiment of the present application provides a gesture posture processing method, including:

[0025] Step S01, calculating the hand scale coefficient between the virtual gesture palm and the actual gesture palm, and scaling the actual gesture by the hand scale coefficient to obtain the target fingertip position;

[0026] Step S01 specifically includes:

[0027] According to the four first line segments formed between the root joints of the four fingers except the thumb and the corresponding wrist joints of the actual gesture palm and the four second line segments formed between the root joints of the four fingers of the virtual gesture palm and the corresponding wrist joints, the hand shape scale coefficient between the virtual gesture palm and the actual gesture palm is determined, and the hand shape scale coefficient is used to represent the degree of correlation between the first line segments of the actual gesture and the second line segments of the virtual gesture.

[0028] The hand shape scale coefficient s is calculated in the following way:

[0029]

[0030] Among them, l1, l2, l3, and l4 are respectively the lengths of the four first line segments formed between the four finger root joints of the actual gesture palm except the thumb and their corresponding wrist joints, L1, L2, L3, and L4 are respectively the lengths of the four second line segments formed between the four finger root joints of the virtual gesture palm and their corresponding wrist joints, and w1, w2, w3, and w4 are respectively the weights of each line segment.

[0031] Step S02, determining a first quaternion between the virtual gesture palm and the actual gesture palm, and rotating the virtual gesture according to the first quaternion so that the virtual gesture palm and the actual gesture palm face the same direction;

[0032] Step S02 specifically includes:

[0033] A mapping relationship is established between the first joint point of the virtual gesture palm and the second joint point of the actual gesture palm to obtain the first quaternion between the virtual gesture palm and the actual gesture palm, wherein the first joint point is the wrist joint point of the virtual gesture palm and the four finger root joint points except the thumb, and the second joint point is the actual gesture palm and the four finger root joint points except the thumb.

[0034] It should be noted that the virtual gesture and the actual gesture need to be aligned first. Before adjusting the orientation, the first quaternion between the virtual gesture palm and the actual gesture palm needs to be determined. The quaternion is a simple hypercomplex number, which is composed of a real number plus an imaginary unit i, where i 2 =-1. Similarly, quaternions are composed of real numbers plus three imaginary units i, j and k, and they have the following relationship: i 2 =j2=k2=-1, i°=j°=k°=1, each quaternion is a linear combination of 1, i, j and k, that is, the quaternion can be generally expressed as a+bi+cj+dk, where a, b, c, d are real numbers. The geometric meaning of i, j and k themselves can be understood as a rotation, where i rotation represents the rotation of the positive direction of the Z axis to the positive direction of the Y axis in the plane where the Z axis intersects the Y axis, j rotation represents the rotation of the positive direction of the X axis to the positive direction of the Z axis in the plane where the X axis intersects the Z axis, and k rotation represents the rotation of the positive direction of the Y axis to the positive direction of the X axis in the plane where the Y axis intersects the X axis. -i, -j, and -k represent the reverse rotations of the i, j, and k rotations, respectively.

[0035] In this application, the first quaternion between the virtual gesture palm and the actual gesture palm needs to be determined, because only the virtual gesture palm and the actual gesture palm have the same orientation, and other fingers of the virtual gesture will also rotate with the rotation of the virtual gesture palm.

[0036] It should be noted that, the virtual gesture is rotated according to the first quaternion obtained in step S02, so that the palm of the virtual gesture is oriented in the same direction as the palm of the actual gesture; the hand scale coefficient obtained in step S01 is used to scale each joint point of the actual gesture to obtain the target fingertip position.

[0037] Step S03, calculating a wrist joint translation vector between the virtual gesture and the actual gesture, and moving the entire virtual gesture to the wrist joint of the actual gesture according to the wrist joint translation vector;

[0038] It should be noted that, in the world coordinate system, the difference in wrist joint position between the virtual gesture and the actual gesture is t, where t is the translation vector. By calculating the wrist joint translation vector between the virtual gesture and the actual gesture, and moving the virtual gesture as a whole to the wrist joint of the actual gesture according to the wrist joint translation vector, the actual gesture and the virtual gesture can be made consistent in position at the wrist joint, and the only difference between the two is the finger shape.

[0039] In an embodiment of the present application, all joint points of the actual gesture are converted into the coordinate system of their respective parent joint points, which can facilitate subsequent operations.

[0040] Step S04, rotate the second quaternion corresponding to each joint of each finger of the virtual gesture starting from the finger root joint in turn, and each rotation needs to determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint complies with the rotation axis and rotation angle constraints of the human hand skeleton; if all comply, stop rotating the next level joint of the current finger, and then obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if all do not comply, continue rotating the next level joint of the current finger.

[0041] It should be noted that step S05 mainly updates the joint points of the virtual gesture by adopting the inverse kinematics method, adjusting each joint of the virtual gesture from the root joint until it reaches the target fingertip position, and then updating the posture and joint axis angle of the initial gesture template, so that the virtual hand shape appears more natural and the movement transition is smooth.

[0042] The step of rotating each finger of the virtual gesture in sequence from the root joint of the finger and the second quaternion corresponding to each joint includes:

[0043] The second quaternion is rotated from the root joint of the virtual gesture so that the root joint faces the fingertip posture of the actual gesture after scaling by the hand scale coefficient, and other joints on the same finger as the root joint are updated in sequence by rotating the root joint of the virtual gesture, and it is determined whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range; if so, the rotation of the next level joint of the current finger is stopped, if not, it is necessary to determine whether the total number of rotations of each finger joint exceeds a set threshold to determine whether to continue rotating the next level joint of the current finger.

[0044] The specific process of step S05 is described in detail below by taking three joints of a finger as an example:

[0045] In the embodiment of the present application, the virtual gesture after the first quaternion is rotated and translated by t is divided into 5 groups according to the fingers, and each finger has 3 joints and 4 joints. The following operations are performed on each finger:

[0046] Traditional inverse kinematics methods generally start from the end joint point to iterate and optimize, but when a human hand bends, it first rotates from the base joint of the finger, then adjusts the middle joint of the finger, and finally rotates the fingertip joint. The rotation of the non-fingertip joints will be transmitted to the fingertip, thereby driving the rotation of the fingertip joint.

[0047] Therefore, the present application adopts iterative optimization starting from the rotation of the finger root joints to prevent the abnormal hand shape in which the terminal joints are directly bent and the front joints are immobile, which is more in line with the physiological rotation requirements of the human hand.

[0048] First, it is necessary to determine the fingertip posture P3 obtained after each rotation. The fingertip posture error e obtained after each rotation is determined by the following method:

[0049] e=‖P t -P3‖2

[0050] Among them, P t is the fingertip pose of the actual gesture after scaling by the hand scale coefficient, and P3 is the fingertip pose obtained after each rotation of the virtual gesture.

[0051] Define the fingertip position of the actual gesture after scaling by the scale factor as Figure 2 Medium target.

[0052] like Figure 2 As shown, a finger has three joints, namely joint 0, joint 1 and joint 2. There are 3 fingertip joints. Starting from the root joint, the positions of each joint are: P0, P1, P2, P3.

[0053] Step 1: Rotate the direction of joint 0 so that it faces forward. The actual gesture is scaled by the scale factor. The fingertip position target is set.30 =P3-P0,P t0 =P t -P0, rotate the second quaternion Q0 by vector P 30 , P t0 and the rotation of P0 in the world coordinate system Calculation, the rotation of the second quaternion Q0 needs to meet the rotation axis and rotation angle constraints of the human hand skeleton, and then smoothed by the corresponding weighted quaternion spherical interpolation, and updated P1, P2, P3, Determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint conforms to the rotation axis and rotation angle constraints of the human hand skeleton; if both are met, stop rotating the next-level joint of the current finger to obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if both are not met, continue rotating the next-level joint of the current finger.

[0054] Step 2: Rotate the direction of joint 1 so that it faces forward. The actual gesture is scaled by the scale factor. The fingertip position target is set. 31 =P3-P1,P t1 =P t -P1, rotate the second quaternion Q1 by vector P 31 , P t1 and the rotation of P1 in the world coordinate system Rotation relative to the parent joint Calculation, the rotation of the second quaternion Q1 needs to meet the rotation axis and rotation angle constraints of the human hand skeleton, and then smoothed by corresponding weighted spherical interpolation, and updated P2, P3, Determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint conforms to the rotation axis and rotation angle constraints of the human hand skeleton; if both are met, stop rotating the next-level joint of the current finger to obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if both are not met, continue rotating the next-level joint of the current finger.

[0055] Step 3: Rotate the direction of joint 2 so that it faces forward. The actual gesture is scaled by the scale factor. The fingertip position target is set. 32 =P3-P2, P t2 =P t -P2, rotate the second quaternion Q2 by vector P 32 , P t2 and the rotation of P2 in the world coordinate system Rotation relative to the parent joint Calculation, the rotation of the second quaternion Q2 needs to meet the rotation axis and rotation angle constraints of the human hand skeleton, and then smoothed by corresponding weighted spherical interpolation, and P3 is updated in sequence through the transfer method. Determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint conforms to the rotation axis and rotation angle constraints of the human hand skeleton; if both are met, stop rotating the next-level joint of the current finger to obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if both are not met, continue rotating the next-level joint of the current finger.

[0056] In the embodiment of the present application, the rotation weights increase from the root joint point to the fingertip joint points.

[0057] Specifically, in order to prevent the fingertip joints of the virtual gesture from missing the target posture (the fingertip position of the actual gesture after scaling by the scale factor), and the rotation of the non-fingertip joints will be transmitted to the fingertip joints step by step, resulting in a large change in the posture of the fingertip joints, this application adopts different rotation weights for each joint. In this application, Figure 2 As shown in the figure, the rotation weight of joint 0 is k1, the rotation weight of joint 1 is k2, the rotation weight of joint 2 is k3, and joint 3 is the fingertip joint point, regardless of its rotation direction. The rotation weight processing adopts the four-element spherical interpolation method, where k1 <k2<k3。

[0058] In a preferred embodiment of the present application, whether the rotation of each of the joints complies with the rotation axis and rotation angle constraints of the human hand skeleton includes:

[0059] Determine whether the root joint of each finger of the virtual gesture has two rotation axes, whether the remaining joints have one rotation axis and whether hinges are formed between the joints; if so, update the finger posture of the entire joint chain of the virtual gesture until the fingertip joint of the virtual gesture reaches the fingertip posture of the actual gesture after being scaled by the hand scale coefficient or the number of times the finger posture of the entire joint chain of the virtual gesture is updated reaches a set threshold.

[0060] Specifically, Figure 3 As shown in the figure, each time a joint rotates during the iteration process, it is determined whether the axis angle of the joint rotation meets the rotation axis and rotation angle constraints of the human hand skeleton. The root joints of the thumb, index finger, middle finger, ring finger, and little finger have two rotation axes (joint numbers 1, 5, 9, 13, and 17 in the figure); the remaining joints of these four fingers have only one rotation axis (joint numbers 2, 3, 6, 7, 10, 11, 14, 15, 18, and 19 in the figure), which need to meet the hinge constraint conditions.

[0061] In an embodiment of the present application, when the fingertip joint has reached the target posture (the fingertip position after the actual gesture is scaled by the scale coefficient), it is necessary to check whether the entire finger joint chain meets the rotation axis and rotation angle constraints of the human hand skeleton. For the finger joint chain that does not meet the requirements, the corresponding joint rotation angle is updated, and then the finger posture of the entire joint chain is updated again. The specific updating method is described in detail in step S05, until the fingertip joint reaches the target position again or the number of iterations reaches the set threshold N.

[0062] During the gesture interaction process, each time step S05 is used to estimate the joint posture of the virtual gesture model, the posture of the virtual gesture can be the last virtual gesture posture, and the posture of each joint point of the virtual gesture at the last moment needs to be converted to the corresponding parent joint coordinate system, which can reduce the hand shape jump and make the movement transition natural.

[0063] It should be noted that if the corresponding finger joints of the virtual gesture have undergone a round of rotation and the virtual gesture fingertip has not reached the target position, then continue to rotate from the finger root node in the above manner, and iterate in a cycle until the fingertip joints of the virtual gesture reach the fingertip posture of the actual gesture after scaling by the hand scale coefficient or the number of times the finger posture of the entire joint chain of the virtual gesture is updated reaches the set threshold.

[0064] Second, as Figure 4 As shown, the embodiment of the present application also provides a gesture posture processing device, including:

[0065] A calculation module 21, used for calculating the hand shape scale coefficient between the virtual gesture palm and the actual gesture palm;

[0066] A determination module 22, used to determine a first quaternion between the virtual gesture palm and the actual gesture palm;

[0067] The rotation module 23 is further used to scale the actual gesture by the hand scale coefficient to obtain the target fingertip position, and rotate the virtual gesture according to the first quaternion so that the virtual gesture palm is in the same direction as the actual gesture palm;

[0068] The calculation module 22 is further used to calculate the wrist joint point translation vector between the virtual gesture and the actual gesture, and move the entire virtual gesture to the wrist joint of the actual gesture according to the wrist joint point translation vector;

[0069] The rotation module 23 is also used to rotate the second quaternion corresponding to each joint of each finger of the virtual gesture in turn starting from the finger root joint, and each rotation needs to determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint complies with the rotation axis and rotation angle constraints of the human hand skeleton; if all are met, stop rotating the next level joint of the current finger, and then obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if all are not met, continue rotating the next level joint of the current finger.

[0070] Compared with the prior art, the beneficial effects of the gesture posture processing device provided in the embodiment of the present application are the same as the beneficial effects of the technical solution provided in the first aspect, and will not be repeated here.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A gesture processing method, characterized in that: include: Calculating a hand scale coefficient between the virtual gesture palm and the actual gesture palm, and scaling the actual gesture by the hand scale coefficient to obtain a target fingertip position; Determine the first quaternion between the virtual gesture palm and the actual gesture palm, and rotating the virtual gesture according to the first quaternion so that the virtual gesture palm and the actual gesture palm are in the same direction; Calculating a wrist joint translation vector between a virtual gesture and an actual gesture, and moving the entire virtual gesture to the wrist joint of the actual gesture according to the wrist joint translation vector; Rotate the second quaternion corresponding to each joint of each finger of the virtual gesture starting from the finger root joint in turn, and each rotation requires judging whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint complies with the rotation axis and rotation angle constraints of the human hand skeleton; if all comply, stop rotating the next-level joint of the current finger, and then obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if all do not comply, continue rotating the next-level joint of the current finger.

2. A method for processing hand gestures as claimed in claim 1, characterized in that: The determining of the first quaternion between the virtual gesture palm and the actual gesture palm comprises: Establishing a mapping relationship between a first joint point of the virtual gesture palm and a second joint point of the actual gesture palm, and obtaining a first quaternion between the virtual gesture palm and the actual gesture palm; Among them, the first joint points are the wrist joint points of the virtual gesture palm and the root joint points of four fingers except the thumb, and the second joint points are the actual gesture palm and the root joint points of four fingers except the thumb.

3. A method for processing hand gestures as claimed in claim 2, characterized in that: The calculating of the hand shape scale coefficient between the virtual gesture palm and the actual gesture palm includes: The hand shape scale coefficient between the virtual gesture palm and the actual gesture palm is determined based on four first line segments formed between the four finger root joints of the actual gesture palm except the thumb and their corresponding wrist joints and four second line segments formed between the four finger root joints of the virtual gesture palm and their corresponding wrist joints.

4. A method for processing hand gestures as claimed in claim 3, characterized in that: The hand shape scale coefficient s is calculated in the following way: Among them, l1, l2, l3, and l4 are respectively the lengths of the four first line segments formed between the four finger root joints of the actual gesture palm and their corresponding wrist joints, L1, L2, L3, and L4 are respectively the lengths of the four second line segments formed between the four finger root joints of the virtual hand model palm and their corresponding wrist joints, and w1, w2, w3, and w4 are respectively the weights of each line segment.

5. A method for processing hand gestures as claimed in claim 1, characterized in that: The method further comprises: Convert all joint points of the actual gesture to the coordinate system of their respective parent joint points.

6. A method for processing hand gestures as claimed in claim 1, characterized in that: The fingertip position error e obtained after each rotation of the virtual gesture is determined by the following method: e=‖P t -P3‖2 Among them, P t is the fingertip pose of the actual gesture after scaling by the hand scale coefficient, and P3 is the fingertip pose obtained after each rotation of the virtual gesture.

7. A method for processing hand gestures as claimed in claim 6, characterized in that: The step of rotating each finger of the virtual gesture in sequence from the root joint of the finger and the second quaternion corresponding to each joint includes: The second quaternion is rotated from the root joint of the virtual gesture so that the root joint faces the fingertip posture of the actual gesture after scaling by the hand scale coefficient, and other joints on the same finger as the root joint are updated in sequence by rotating the root joint of the virtual gesture, and it is determined whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range; if so, the rotation of the next level joint of the current finger is stopped, if not, it is necessary to determine whether the total number of rotations of each finger joint exceeds a set threshold to determine whether to continue rotating the next level joint of the current finger.

8. A method for processing hand gestures as claimed in claim 7, characterized in that: The rotation weights increase from the root joint point of the virtual gesture to the fingertip joint points.

9. A method for processing hand gestures as claimed in claim 7, characterized in that: Whether the rotation of each of the joints complies with the rotation axis and rotation angle constraints of the human hand skeleton includes: Determine whether the root joint of each finger of the virtual gesture has two rotation axes, whether the remaining joints have one rotation axis and whether hinges are formed between the joints; if so, update the finger posture of the entire joint chain of the virtual gesture until the fingertip joint of the virtual gesture reaches the fingertip posture of the actual gesture after being scaled by the hand scale coefficient or the number of times the finger posture of the entire joint chain of the virtual gesture is updated reaches a set threshold.

10. A gesture processing device, characterized in that: include: A determination module, used for determining a first quaternion between a virtual gesture palm and an actual gesture palm; A calculation module, used for calculating the hand shape scale coefficient between the virtual gesture palm and the actual gesture palm; The rotation module is further used to scale the actual gesture by the hand scale coefficient to obtain the target fingertip position, and rotate the virtual gesture according to the first quaternion so that the virtual gesture palm is in the same direction as the actual gesture palm; The calculation module is further used to calculate the wrist joint point translation vector between the virtual gesture and the actual gesture, and move the virtual gesture as a whole to the wrist joint of the actual gesture according to the wrist joint point translation vector; The rotation module is also used to rotate the second quaternion corresponding to each joint of each finger of the virtual gesture in turn starting from the finger root joint, and each rotation needs to determine whether the fingertip posture error obtained after each rotation of the virtual gesture is within the target error range and whether the rotation of each joint complies with the rotation axis and rotation angle constraints of the human hand skeleton; if all are met, stop rotating the next level joint of the current finger, and then obtain a virtual gesture consistent with the corresponding finger of the actual gesture; if all are not met, continue rotating the next level joint of the current finger.