Method and device for generating 3D gesture data

Abstract

The invention relates to the technical field of gesture interaction, provides a 3D gesture data generation method and device, and is used for improving the reasonability and diversity of gesture data. According to the method, aiming at an arm, a wrist and a hand, based on a forward kinematics theory, a 3D gesture data set is generated by adopting three different modes, namely a random continuous function-based mode, a random walk-based angle change-based mode and an existing hand motion capture data set-based mode, so that the generated 3D gesture data has relatively high randomness; according to the method, the hand gesture data set can be acquired, the unreachable hand gesture in the real world cannot be acquired, the diversity and rationality of the gesture data set are improved, and the action data of the arm, the wrist and the hand are generated in an asynchronous mode, so that the generation efficiency is relatively high.

Description

Technical Field

[0001] The present application relates to the field of gesture interaction technology and provides a method and device for generating 3D gesture data. Background Art

[0002] Virtual Reality (VR) technology provides users with an immersive gaming experience and is currently used in various fields. At present, gesture interaction is one of the important ways for users to interact with the virtual world. With the development of deep learning, 3D gesture estimation algorithms based on deep learning are the main technology for realizing gesture interaction. As a data-based estimation algorithm, deep learning has a huge and indispensable demand for rich and high-precision gesture data. Therefore, high-quality and rich 3D gesture data plays a vital role in gesture interaction.

[0003] The main method of generating 3D gesture data is to collect hand images with the help of a multi-camera acquisition system, and reconstruct the 3D hand posture data through annotation. However, due to the problems of cumbersome acquisition process, poor mobility, high structural strength requirements, high camera calibration accuracy requirements, and high data production costs of the multi-camera acquisition system, getting rid of excessive dependence on the multi-camera acquisition system and using synthesis to generate 3D gesture data has become the main research trend in the industry.

[0004] However, the publicly available 3D gesture synthesis data may not match the usage scenarios of VR products. In order to synthesize gesture data that conforms to the usage scenarios of VR products, related technologies have built a motion capture system. The motion capture system obtains the 3D gesture data made by the user when using VR products, and then synthesizes it with the existing 3D gesture data. However, the motion capture system still faces problems such as high construction cost and large space occupation.

[0005] Therefore, reducing the limitations on generating 3D gesture data while ensuring the quality and amount of 3D gesture data has become an urgent problem to be solved in gesture interaction scenarios. Summary of the invention

[0006] The embodiments of the present application provide a method and device for generating 3D gesture data, which are used to improve the rationality and diversity of gesture data and reduce the cost of generating gesture data.

[0007] On the one hand, an embodiment of the present application provides a method for generating 3D gesture data, including:

[0008] Randomly generate N continuous functions, and generate an arm motion data set according to the N continuous functions; wherein each continuous function represents a movement of the arm, and the end point of the previous continuous function is used as the starting point of the next continuous function to ensure the continuity of the arm motion;

[0009] A random walk method is used to adjust the rotation angle of the wrist joint, and a wrist motion data set is generated according to the rotation angle;

[0010] Based on the existing hand motion capture dataset, synthesize the hand motion dataset;

[0011] The arm motion data set, the wrist motion data set and the hand motion data set are combined to generate a 3D gesture data set; wherein the different motion data sets are generated in an asynchronous manner.

[0012] On the other hand, an embodiment of the present application provides an electronic device, including a processor, a memory, and a communication interface, wherein the communication interface, the memory, and the processor are connected via a bus;

[0013] The communication interface is used to communicate with the terminal device and the server;

[0014] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0015] Randomly generate N continuous functions, and generate an arm motion data set according to the N continuous functions; wherein each continuous function represents a movement of the arm, and the end point of the previous continuous function is used as the starting point of the next continuous function to ensure the continuity of the arm motion;

[0016] A random walk method is used to adjust the rotation angle of the wrist joint, and a wrist motion data set is generated according to the rotation angle;

[0017] Based on the existing hand motion capture dataset, synthesize the hand motion dataset;

[0018] The arm motion data set, the wrist motion data set and the hand motion data set are combined to generate a 3D gesture data set; wherein the different motion data sets are generated in an asynchronous manner.

[0019] Optionally, the arm includes an upper arm and a lower arm, and the upper arm and the lower arm are in the same coordinate system;

[0020] The processor randomly generates N continuous functions, and generates an arm motion data set according to the N continuous functions. The specific operations are:

[0021] Randomly generate N first continuous functions, and generate the upper arm motion data set according to the N first continuous functions;

[0022] Generate a random number that follows a uniform distribution within a preset interval, and determine whether the random number is greater than a preset threshold;

[0023] If it is greater than, N second continuous functions are randomly generated, and the motion data set of the lower arm is generated according to the N second continuous functions; otherwise, the rotation angle of the elbow joint around the lower arm is adjusted by random walk, and the motion data set of the lower arm is generated according to the rotation angle.

[0024] Optionally, the processor randomly generates N first continuous functions, and generates the upper arm motion data set according to the N first continuous functions, and the specific operation is:

[0025] Initialize the orientation of the shoulder joint of the upper arm in the three-dimensional space in the T-pose state, wherein the front orientation of the human body is the positive direction of the Z axis in the three-dimensional space, the standing direction of the human body is the positive direction of the Y axis in the three-dimensional space, and the positive direction of the X axis in the three-dimensional space and the positive directions of the Z axis and the Y axis satisfy the right-hand rule;

[0026] A starting point is randomly generated in the XOY plane, and the starting point is used as the starting point of the first first continuous function among the N first continuous functions, and a forward kinematics solution method is used to calculate a set of first plane point sets of the shoulder joint in the three-dimensional space after the first movement;

[0027] Taking the end point in the first plane point set as the starting point of the next first continuous function, repeating the forward kinematics solution calculation until the N first continuous functions are traversed;

[0028] The motion data set of the upper arm is determined according to the calculated N groups of first plane point sets of the shoulder joint.

[0029] Optionally, the process of the processor calculating a set of first plane point sets of the upper arm in the three-dimensional space according to each first continuous function is:

[0030] According to the parity of the number of movements corresponding to the current first continuous function, n sampling points are sampled equidistantly on the X-axis; wherein, when the number of movements is an odd number, the sampling direction of the X-axis is from left to right, and when the number of movements is an even number, the sampling direction of the X-axis is from right to left;

[0031] According to the characteristics of standard normal distribution, the coefficients and periods of Fourier series are randomly generated;

[0032] According to the coefficient and the period, the function value corresponding to each sampling point is determined respectively to obtain a set of first plane point sets.

[0033] Optionally, the processor determines the motion data set of the upper arm according to the calculated N groups of first plane point sets of the shoulder joint, and the specific operation is:

[0034] Using vector outer product, calculate the first unit rotation axis of the shoulder joint after initialization and rotate it to each first plane point in the N groups of first plane points;

[0035] Using vector inner product, calculate the first rotation angle of the shoulder joint after initialization to each first plane point in the N groups of first plane points;

[0036] A motion data set of the upper arm is obtained according to each first unit rotation axis and each first rotation angle.

[0037] Optionally, the processor randomly generates N second continuous functions, and generates the motion data set of the lower arm according to the N second continuous functions, and the specific operation is:

[0038] Initialize the rotation matrix of the elbow joint around which the lower arm moves in three-dimensional space;

[0039] Randomly generate an initial angle, and use the initial angle as the starting point of the first second continuous function among the N second continuous functions, and use a forward kinematics solution method to calculate a set of second plane point sets of the elbow joint in the three-dimensional space after the first movement;

[0040] Taking the rotation angle corresponding to the end point in the second plane point set as the starting point of the next second continuous function, repeating the forward kinematics solution calculation until the N second continuous functions are traversed;

[0041] The motion data set of the lower arm is determined according to the calculated N groups of second plane point sets of the elbow joint.

[0042] Optionally, the process of the processor calculating a set of second plane point sets of the elbow joint in the three-dimensional space according to each second continuous function is:

[0043] Randomly sample a value in the preset interval, if the value is less than the midpoint of the preset interval, increase the initial angle to obtain an upper limit of the angle, otherwise, decrease the initial angle to obtain a lower limit of the angle;

[0044] Determine a sampling interval according to one of the angle upper limit and the angle lower limit and the initial angle, and sample n sampling points equidistantly in the sampling interval;

[0045] The center and radius of the elbow joint movement are randomly generated, and the coordinates of each sampling point in the XOY plane are determined according to the center and the radius to obtain a set of second plane point sets.

[0046] Optionally, the processor determines the motion data set of the lower arm according to the calculated N sets of second plane point sets of the elbow joint, and the specific operation is:

[0047] Using the vector outer product, calculate the second unit rotation axis of the shoulder joint after the initialization rotation to each second plane point in the N groups of second plane point sets;

[0048] Using vector inner product, calculate the second rotation angle of each second plane point in the N sets of second plane point sets after the shoulder joint is initialized;

[0049] According to each second unit rotation axis and each second rotation angle, in combination with the rotation matrix, a motion data set of the lower arm is obtained.

[0050] Optionally, for the joints to be rotated in the wrist joint and the elbow joint, the processor generates a motion data set of the wrist or lower arm according to the rotation angle in the following manner:

[0051] According to a preset rotation angle interval, determine the probability that the joint to be rotated moves toward the upper limit and the lower limit of the rotation angle interval at the current moment, and randomly determine the movement direction of the joint to be rotated according to the two probabilities;

[0052] Determine an angle sampling interval according to the movement direction and the rotation angle at the current moment;

[0053] Sampling n sampling points equidistantly in the angle sampling interval, and calculating the sampling probability of each sampling point using a gamma function;

[0054] According to each sampling probability, an angle sampling point is selected as the rotation angle at the next moment, and in combination with a preset data length, equidistant sampling is performed between the rotation angle at the current moment and the rotation angle at the next moment to obtain an arithmetic progression, and the arithmetic progression is used as a set of action data of the joint to be rotated;

[0055] The rotation angle at the next moment is used as the rotation angle at the current moment, and the above process is repeated N times to obtain multiple sets of motion data of the joint to be rotated;

[0056] The multiple groups of motion data of the joints to be rotated are used as motion data sets of corresponding parts.

[0057] Optionally, the processor synthesizes a hand motion dataset based on an existing hand motion capture dataset, and the specific operation is:

[0058] Cluster the existing hand motion capture dataset to obtain multiple subsets, where the cluster center of each subset represents a type of hand motion;

[0059] Extracting a preset number of hand motion capture data from subsets of different categories respectively, splicing the hand motion capture data belonging to different categories, and synthesizing a plurality of new hand motion capture data;

[0060] The synthesized new hand motion capture data and the old hand motion capture data before synthesis are interpolated by quaternion to obtain a hand motion data set.

[0061] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer device to execute the steps of a method for generating 3D gesture data provided in an embodiment of the present application.

[0062] In a method and device for generating 3D gesture data provided by an embodiment of the present application, motion data of the arm, wrist and hand are generated asynchronously, thereby improving the generation efficiency of the gesture data set; wherein, the first motion data of the arm is generated by using N randomly generated continuous functions that characterize the arm movement, and the end point of the previous continuous function is used as the starting point of the next continuous function to ensure the continuity of the arm movement; the second motion data of the wrist is generated by adjusting the rotation angle of the wrist joint in a random walk manner; the third motion data of the hand is synthesized based on the existing hand motion capture data set; and by using three different methods to generate a 3D gesture data set, the diversity of the 3D gesture data is ensured, and the three methods comply with the principle of forward kinematics, thereby avoiding the appearance of postures that are unattainable in the real world and improving the accuracy of the 3D gesture data set.

[0063] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0065] Figure 1 Provide a gesture data screening schematic diagram for an embodiment of the present application;

[0066] Figure 2 A flow chart of a method for generating 3D gesture data provided in an embodiment of the present application;

[0067] Figure 3 A flow chart of a method for generating an arm motion data set provided in an embodiment of the present application;

[0068] Figure 4 A flow chart of a method for generating an upper arm motion data set provided in an embodiment of the present application;

[0069] Figure 5 A schematic diagram of the motion trajectory of the upper arm provided in an embodiment of the present application;

[0070] Figure 6 A schematic diagram of the motion trajectory of the lower arm provided in an embodiment of the present application;

[0071] Figure 7 A flow chart of a method for generating a lower arm motion data set provided in an embodiment of the present application;

[0072] Figure 8 A flow chart of another method for generating a lower arm motion data set provided in an embodiment of the present application;

[0073] Fig. 9 A flow chart of a method for generating a wrist motion data set provided in an embodiment of the present application;

[0074] Fig.10 A flow chart of a method for generating a hand motion data set provided in an embodiment of the present application;

[0075] Fig.11 A clustering effect diagram of hand motion capture data provided in an embodiment of the present application;

[0076] Fig.12 A synthetic effect diagram of the hand motion capture data provided in an embodiment of the present application;

[0077] Fig.13 An overall architecture diagram for generating a 3D gesture data set provided in an embodiment of the present application;

[0078] Fig.14 The structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0080] At present, there are three main methods for generating gesture data: the generation method based on animation data in the model animation library, the generation method based on robot trajectory planning, and the generation method based on hand motion capture. To ensure the accuracy of gesture interaction, the generated gesture data generally needs to meet two conditions: one is that the data volume is large and diverse, and the other is that it needs to be ergonomic, that is, there should be no gestures that are impossible for the hands to make in the real world.

[0081] The method for generating animation data based on the model animation library can meet the second condition well thanks to the special hand movement design. However, due to the high production cost of the diversity of model animations and the lack of animation data specially designed for several common gesture movements in AR / VR interaction scenarios, the gesture data generated by this method is generally difficult to meet the first condition.

[0082] The generation method based on robot trajectory planning randomly sets the hand endpoint coordinates and adopts the inverse kinematics solution method to ensure the diversity of gesture data. However, due to the limitations of different inverse kinematics solution methods and the need to meet the requirements of specific gesture postures in AR / VR interaction scenarios, the trajectory of the same endpoint may obtain different gesture data, such as Figure 1 As shown in , therefore, it takes a relatively time-consuming manual verification to screen out reasonable gesture data, such as Figure 1 The left image is more likely to be selected as reasonable gesture data. Since there is currently no simple way to replace manual verification, the gesture data generated by this method is difficult to meet the second condition. In addition, for the trajectory planning of fingers and the interaction between fingers, it is also necessary to design a reasonable trajectory method, which is time-consuming.

[0083] The generation method based on hand motion capture only requires different movements of the human hand in the real world to generate a variety of gesture data that meet the AR / VR interaction scenarios. Therefore, this method can well meet the above two conditions. However, due to the long time period, high cost, and large manpower investment in the construction of the motion capture system, the promotion and application of this method is limited.

[0084] In view of this, an embodiment of the present application provides a method for generating 3D gesture data, which can generate rich and high-precision gesture data. 3D gestures generally include body (such as: arm) postures and hand (such as: wrist and fingers) postures. Therefore, when generating a 3D gesture data set, the method uses three different asynchronous generation methods based on forward kinematics for the motion data of the arm, wrist and hand. Among them, a method for generating motion data based on random continuous functions is designed for the arm. The arm motion trajectory represented by continuous functions is used to simulate the possible motion process of the user's arm during AR / VR experience, thereby generating arm motion data; for the motion data of rotating parts such as wrists and elbows, a method for generating angle changes based on random walks is designed. The rotation angle of the joint to be rotated at the next moment (such as wrist joints and elbow joints) is determined by gamma distribution, thereby generating motion data of wrists, elbows and other parts; for the hand, a method for generating motion data based on random splicing is designed. By randomly sampling different types of hand motion capture data and randomly splicing the sampling results, more diverse gestures or specific gestures are generated. Finally, the motion data of the arm and hand are combined to obtain a complete 3D gesture data set. The 3D gesture data generated by this method has high randomness, can meet the requirements of gesture diversity, and will not appear in the real world. Unreachable gestures and postures have high generation accuracy.

[0085] Compared with the generation method based on the animation data in the model animation library, the method provided in the embodiment of the present application generates a 3D gesture data set with stronger randomness and can produce richer hand movements. In addition, the method provided in the embodiment of the present application can generate 3D gesture data from the first-person perspective, which can be used in the first-person interaction scene, which is a feature lacking in the animation data in the model animation library. On the other hand, compared with the generation method based on robot trajectory planning, the method provided in the embodiment of the present application, from the perspective of forward kinematics, can better limit the range of motion of the shoulder joint, elbow joint and wrist joint while ensuring randomness, thereby forming a more reasonable interactive gesture. On the other hand, compared with the generation method based on hand motion capture, the method provided in the embodiment of the present application avoids the problems of high cost and large equipment space occupied by building a motion capture system, while ensuring the posture diversity of the synthetic data and better simulating the interactive actions that users may make in real scenes.

[0086] It should be noted that the methods for generating left and right gestures are the same, so the embodiments of the present application do not make any distinction, that is, the following embodiment methods are applicable to the left and right hands respectively.

[0087] See also Figure 2 , is a flow chart of a method for generating 3D gesture data provided in an embodiment of the present application, and the process mainly includes the following steps:

[0088] S201: randomly generate N continuous functions, and generate an arm motion data set according to the N continuous functions.

[0089] The embodiment of the present application designs a method of generating arm motion data based on a random continuous function, and simulates the possible arm motion process of the user during the AR / VR experience through the arm motion trajectory represented by the continuous function, thereby generating arm motion data. Among them, each continuous function represents a movement of the arm, and the end point of the previous continuous function is used as the starting point of the next continuous function to ensure the continuity of the arm movement.

[0090] The generation process of the arm motion dataset is detailed in Figure 3 , mainly includes the following steps:

[0091] S2011: randomly generate N first continuous functions, and generate an upper arm motion data set according to the N first continuous functions.

[0092] Generally, the arm is divided into an upper arm and a lower arm. The upper arm moves around the shoulder joint, and the lower arm moves around the elbow joint. Due to the different joints, different continuous functions are used to generate the action data sets of the upper arm and the lower arm. To distinguish the descriptions, the N continuous functions randomly generated for the upper arm are recorded as the first continuous function, and the N continuous functions randomly generated for the lower arm are recorded as the second continuous function.

[0093] The upper arm moves back and forth around the shoulder joint N times by randomly generating N first continuous functions. The process of generating the upper arm motion data set based on the random N first continuous functions is shown in Figure 4 , mainly includes the following steps:

[0094] S2011_1: Initializes the orientation of the shoulder joint in three-dimensional space around which the upper arm moves in the T-pose state.

[0095] Taking the left arm as an example, Figure 5 As shown, it is a schematic diagram of the motion trajectory of the upper arm around the shoulder joint along the first continuous function in the three-dimensional space, wherein the front direction of the human body is the positive direction of the Z axis in the three-dimensional space, the standing direction of the human body is the positive direction of the Y axis in the three-dimensional space, and the positive direction of the X axis in the three-dimensional space satisfies the right-hand rule as do the positive directions of the Z axis and the Y axis.

[0096] Based on the above three-dimensional space, the shoulder joint around which the upper arm moves is initialized, the left arm is set to a T-pose state, and the direction of the shoulder joint is set to (0, 0, 1).

[0097] S2011_2: A starting point is randomly generated in the XOY plane, and the starting point is used as the starting point of the first first continuous function among N first continuous functions. The forward kinematics solution method is used to calculate a set of first plane point sets of the shoulder joint in three-dimensional space after the first movement.

[0098] Generate a random point on the XOY plane, denoted as p start =(x0, y0), point p start As the starting point of the first first continuous function among the N first continuous functions, a forward kinematics solution method is used to calculate a set of first plane point sets of the shoulder joint in the three-dimensional space after the first movement.

[0099] In one example, first, according to the parity of the number of movements corresponding to the current first continuous function, n sampling points are sampled equidistantly on the X axis, denoted as x i =[x1, x2, ..., x n ], where when the number of movements is an odd number, the sampling direction of the X-axis is from left to right, and when the number of movements is an even number, the sampling direction of the X-axis is from right to left. Then, according to the standard normal distribution characteristics, the coefficients and periods of the Fourier series are randomly generated. Finally, according to the coefficients and periods of the Fourier series, the function values ​​corresponding to each sampling point are determined respectively to obtain a set of first plane point sets

[0100] For example, for the first movement, the movement number corresponding to the first continuous function is an odd number, and sampling is performed equidistantly from left to right on the x-axis to obtain χ1=[x1, x2, ..., x n ], for the second movement, the number of movements corresponding to the second first continuous function is an even number, and sampling is performed equidistantly from right to left on the X-axis to obtain χ2 = [x1, x2, ..., x n ].

[0101] Assume that the coefficients of the Fourier series are denoted by a j and b j (j=0,1,2,...,K), the period is T, and the calculation formula of the function value of each sampling point is as follows:

[0102]

[0103] Among them, x i represents the i-th sampling point, y i Represents the sampling point x i The corresponding function value, K is the preset value, (x i ,y i ) represents the i-th first plane point.

[0104] S2011_3: Taking the end point in the first plane point set as the starting point of the next first continuous function, repeat the forward kinematics solution calculation until N first continuous functions are traversed.

[0105] For the next first continuous function, the last first plane point (i.e., the end point) in the first plane point set corresponding to the previous first continuous function (x n ,y n ) as the starting point p start , repeat the forward kinematics solution process in S2011_2 until N first continuous functions are traversed, thereby obtaining N groups of first plane point sets.

[0106] It should be noted that the number of sampling points corresponding to each first continuous function may be the same or different.

[0107] S2011_4: Determine the motion data set of the upper arm based on the calculated N groups of first plane point sets of the shoulder joint.

[0108] In one example, the vector outer product is used to calculate the first unit rotation axis of the shoulder joint after it is initialized and rotated to each first plane point in the N groups of first plane point sets, and the vector inner product is used to calculate the first rotation angle of the shoulder joint after it is initialized and rotated to each first plane point in the N groups of first plane point sets, thereby obtaining a motion data set of the upper arm based on each first unit rotation axis and each first rotation angle.

[0109] For example, assuming that the shoulder joint is initialized to face (0, 0, 1), the vector outer product is used to calculate the rotation from (0, 0, 1) to the shoulder joint (x i ,y i , 1) the unit rotation axis , and rotate to the shoulder joint (x) by calculating the vector inner product (0, 0, 1) i ,y i , 1) the rotation angle θ i , thus obtaining the axis rotation set , that is, the action data set Θ of the upper arm moving continuously N times shoulder , where m is the total number of N groups of first plane points.

[0110] S2012-S2013: Generate a random number that follows a uniform distribution within a preset interval, and determine whether the random number is greater than a preset threshold. If so, execute S2014; if not, execute S2015.

[0111] In one example, a random number α that follows a uniform distribution in the interval [0, 1) is randomly generated, and it is determined whether α is greater than a preset threshold, and according to the judgment result, different methods are used to generate the lower arm action data set to improve the diversity of the 3D gesture data set. The preset threshold ∈ (0, 1).

[0112] It should be noted that the threshold value can be set according to actual needs, and the embodiments of the present application do not impose any restrictive requirements.

[0113] S2014: Randomly generate N second continuous functions, and generate a motion data set of the lower arm according to the N second continuous functions.

[0114] Still taking the left arm as an example, Figure 6 The figure shows the motion trajectory of the lower arm around the elbow joint along the second continuous function in three-dimensional space, where the upper arm and the lower arm are in the same coordinate system. When α is greater than the preset threshold, the action data set of the lower arm is generated by using N random second continuous functions similar to those of the upper arm. For the specific process, see Figure 7 , mainly includes the following steps:

[0115] S2014_1: Initialize the rotation matrix of the elbow joint around which the lower arm moves in three-dimensional space.

[0116] In one example, the rotation matrix R of the elbow joint is initialized start =Rodrigues(0,0,-π / 2).

[0117] S2014_2: Randomly generate an initial angle and use the initial angle as the starting point of the first second continuous function among N second continuous functions. Use the forward kinematics solution method to calculate a set of second plane point sets of the elbow joint in three-dimensional space after the first movement.

[0118] In one example, first, a value is randomly sampled in a preset interval. If the value is less than the midpoint of the preset interval, the initial angle is increased to obtain the upper limit of the angle, otherwise, the initial angle is reduced to obtain the lower limit of the angle; then, a sampling interval is determined according to one of the upper limit and the lower limit of the angle and the initial angle, and n sampling points are sampled equidistantly in the sampling interval; finally, the center and radius of the elbow joint movement are randomly generated, and according to the center and radius, the coordinates of each sampling point in the XOY plane are determined respectively to obtain a set of second plane point sets.

[0119] Taking the first second continuous function as an example, assuming that the preset interval is [0, 1), the sampling value is u, and the initial angle θ start =θ0, the randomly generated radius is r, and the center is (c x , c y ), at this time, when u<0.5, θ start Increase Δθ to obtain the upper limit of the angle θ start +Δθ, otherwise, θ start Reduce Δθ to obtain the lower limit of angle θ start -Δθ. At θ start and θstart In the ±Δθ sampling interval, n sampling points are sampled equidistantly to obtain a set of angles Φ=(θ1=θ start ,θ2,θ3,…,θ n =θ start ±Δθ). According to the radius r and the center (c x , c y ), calculate the coordinates of each sampling point in the angle Φ in the XOY plane, and obtain a set of second plane point sets

[0120] S2014_3: Taking the rotation angle corresponding to the end point in the second plane point set as the starting point of the next second continuous function, repeat the forward kinematics solution calculation until N second continuous functions are traversed.

[0121] For the next second continuous function, the last second plane point (i.e., the end point) in the first plane point set corresponding to the previous second continuous function (x n ,y n ) corresponds to the rotation angle θ n As the starting point θ start , repeat the forward kinematics solution process in S2014_2 until N second continuous functions are traversed, thereby obtaining N sets of second plane point sets.

[0122] It should be noted that the number of sampling points corresponding to each second continuous function may be the same or different.

[0123] S2014_4: Determine the motion data set of the lower arm based on the calculated N sets of second plane point sets of the elbow joint.

[0124] In one example, the vector outer product is used to calculate the second unit rotation axis of each second plane point in the N sets of second plane point sets after the shoulder joint is initialized, and the vector inner product is used to calculate the second rotation angle of each second plane point in the N sets of second plane point sets after the shoulder joint is initialized, so that according to each second unit rotation axis and each second rotation angle, the rotation matrix R is combined start , obtain the lower arm motion dataset.

[0125] For example, assuming that the shoulder joint is initialized to face (0, 0, 1), the vector outer product is used to calculate the rotation from (0, 0, 1) to the elbow joint (x i ,y i , 1) the unit rotation axis , and rotate to the elbow joint (x i ,y i , 1) the rotation angle θ i , combined with the rotation matrix R start , thus obtaining the rotation matrix set , that is, the action data set Θ of the lower arm moving continuously N times elbow , where m is the total number of N groups of second plane points.

[0126] It should be noted that, in the process of generating the upper arm and lower arm motion data sets, the same variable represents the same meaning but has different values.

[0127] S2015: A random walk method is used to adjust the rotation angle of the elbow joint around the lower arm, and a motion dataset of the lower arm is generated according to the rotation angle.

[0128] When α is less than or equal to the preset threshold, a different method from that used to generate the upper arm motion data set can be used to generate the lower arm motion data set, namely, a random walk-based angle change generation method. The main idea of ​​the random walk-based angle change generation method is to pre-set the upper and lower limits of the angle change and move back and forth between the upper and lower limits with a certain probability.

[0129] Since the lower arm moves around the elbow joint, the process of obtaining the lower arm motion data set can be obtained by simulating the back-and-forth motion of the elbow joint. For the specific process, see Figure 8 , mainly includes the following steps:

[0130] S2015_1: According to a preset rotation angle interval, determine the probability that the elbow joint moves to the upper limit and the lower limit of the rotation angle interval at the current moment, and randomly determine the movement direction of the elbow joint according to the two probabilities.

[0131] Assume that the rotation angle interval is [θ lower1 ,θ upper1 ], the current rotation angle of the elbow joint is θ curr1 , we can get the rotation angle range Δθ1=θ upper1 -θ lowei1 , at this time, the elbow joint moves to the upper limit θ upper1 Probability of wandering Elbow joint lower limitθ lowei1 Probability of wandering And according to P1(upper) and P1(lower), the movement direction of the elbow joint is randomly determined.

[0132] S2015_2: Determine an angle sampling interval according to the movement direction and the rotation angle at the current moment.

[0133] When the elbow joint moves to the upper limit, the angle sampling interval is [θ curr1 ,θ upper1 ], when the elbow joint moves to the lower limit, the angle sampling interval is [θ lower1 ,θ curr1 ].

[0134] S2015_3: Sample n sampling points equidistantly in the angle sampling interval, and use the gamma function to calculate the sampling probability of each sampling point.

[0135] Taking the elbow joint moving toward the upper limit as an example, in the angle sampling interval [θ curr1 ,θ upper1 ] equidistantly sample n sampling points {θ i , i = 1, 2, ..., n} (including the upper limit θ upper1 ), for each sampling point θ i , the calculation formula of the sampling probability is as follows:

[0136]

[0137] Among them, x i ∈[0.25, 10.25], each x i Corresponding to a θ i , that is, x i are n points sampled equally between [0.25, 10.25], Γ(.) is the gamma function, P i is θ i The corresponding sampling probability.

[0138] S2015_4: Select an angle sampling point as the rotation angle of the next moment according to each sampling probability, and perform equidistant sampling between the rotation angle of the current moment and the rotation angle of the next moment in combination with the preset data length to obtain an arithmetic sequence, and use the arithmetic sequence as a set of motion data of the elbow joint.

[0139] According to the sampling probability {P i ,i=1,2,...,n}, in {θ i , i = 1, 2, ..., n} and randomly select a θ i* As the rotation angle of the next moment, and then according to the preset data length, in [θ curr1 ,θ i* ] to perform equidistant sampling between the two points to obtain an arithmetic progression of rotation angles that is continuous in time, and the obtained arithmetic progression is used as a set of motion data of the elbow joint.

[0140] S2015_5: The rotation angle at the next moment is used as the rotation angle at the current moment, and the above process is repeated N times to obtain multiple sets of motion data of the elbow joint.

[0141] Let θ curr1 =θ i* ,After repeating the process from S2015_1 to S2015_4 N times, N groups of action data sets of the elbow joint can be obtained.

[0142] S2015_6: Multiple sets of elbow joint motion data as lower arm motion data set Θ elbow .

[0143] In the embodiment of the present application, by setting the random number α and the threshold, a plurality of methods are adopted to generate the motion data set of the lower arm, thereby increasing the diversity of the 3D gesture data.

[0144] The 3D gesture data generated by the embodiment of the present application includes the posture (including position coordinates and angle information) of the arm (including the upper arm and the lower arm) and the posture of the hand (including the wrist and the fingers). Since the arm and the hand are independent of each other, and the connection between the hand motion data and the arm motion data is weak, the embodiment of the present application adopts an asynchronous method to generate not only the arm motion data set, but also the hand motion data set to improve the generation efficiency of the 3D gesture data set.

[0145] Considering that the wrist and fingers can move independently, the embodiments of the present application use different methods to generate a hand motion data set, see S202 to S203.

[0146] S202: The rotation angle of the wrist joint is adjusted by a random walk method, and a wrist motion data set is generated according to the rotation angle.

[0147] Similar to the elbow joint, the hand can move around the wrist joint. Therefore, the angle change generation method based on random walk can also be used to generate the wrist motion dataset. For the specific process, see Fig. 9 , mainly includes the following steps:

[0148] S2021: According to a preset rotation angle interval, determine the probability that the wrist joint moves toward the upper limit and the lower limit of the rotation angle interval at the current moment, and randomly determine the movement direction of the wrist joint according to the two probabilities.

[0149] Assume that the rotation angle interval is [θ lower2 ,θ upper2 ], the rotation angle of the wrist joint at the current moment is θ curr2 , we can get the rotation angle range Δθ2=θ upper2 -θ lower2 At this time, the wrist joint moves to the upper limit θ upper2 Probability of wandering Wrist joint lower limit θ lower1 Probability of wandering And according to P2(upper) and P2(lower), the movement direction of the wrist joint is randomly determined.

[0150] S2022: Determine an angle sampling interval according to the movement direction and the rotation angle at the current moment.

[0151] When the wrist joint moves toward the upper limit, the angle sampling interval is [θ curr2 ,θ upper2 ], when the wrist joint moves to the lower limit, the angle sampling interval is [θ lower2 ,θ curr2 ].

[0152] S2023: Sample n sampling points equidistantly in the angle sampling interval, and use a gamma function to calculate the sampling probability of each sampling point.

[0153] Taking the wrist joint moving toward the upper limit as an example, in the angle sampling interval [θ curr2 ,θ upper2 ] equidistantly sample n sampling points {θ i , i = 1, 2, ..., n} (including the upper limit θ upper2 ), for each sampling point θ i , the calculation formula of the sampling probability is as follows:

[0154]

[0155] Among them, x i ∈[0.25, 10.25], each x i Corresponding to a θ i , that is, x i are n points sampled equally between [0.25, 10.25], Γ(.) is the gamma function, P i is θ i The corresponding sampling probability.

[0156] S2024: Select an angle sampling point as the rotation angle at the next moment according to each sampling probability, and perform equidistant sampling between the rotation angle at the current moment and the rotation angle at the next moment in combination with the preset data length to obtain an arithmetic sequence, and use the arithmetic sequence as a set of motion data of the wrist joint.

[0157] According to the sampling probability {P i ,i=1,2,...,n}, in {θ i , i = 1, 2, ..., n} and randomly select a θ i* As the rotation angle of the next moment, and then according to the preset data length, in [θ curr1 ,θ i* ] to perform equidistant sampling between the two points to obtain an arithmetic progression of rotation angles that is continuous in time, and use the obtained arithmetic progression as a set of motion data of the wrist joint.

[0158] S2025: Taking the rotation angle at the next moment as the rotation angle at the current moment, repeat the above process N times to obtain multiple sets of motion data of the wrist joint.

[0159] Let θcurr2 =θ i* , after repeating the process from S2021 to S2024 N times, N groups of action data sets of the wrist joint can be obtained.

[0160] S2026: Multiple sets of wrist joint motion data as wrist motion data set Θ wrist .

[0161] The diversity of hand gestures determines the diversity of 3D gesture datasets. Compared with the situation where arm gestures are basically controllable, the frequent interactions between fingers and the need for specific gestures make it difficult for the generation methods based on random continuous functions and random walks to meet the needs of hand gesture diversity. Therefore, the embodiment of the present application designs a hand gesture generation method based on random splicing based on the existing hand motion capture dataset (where each hand motion capture data contains the rotation, translation and other information required for the forward dynamics of the hand), see S203.

[0162] S203: Synthesizing a hand motion dataset based on an existing hand motion capture dataset.

[0163] In one example, the synthesis process of the hand motion dataset is shown in Fig.10 , mainly includes the following steps:

[0164] S2031: Cluster the existing hand motion capture dataset to obtain multiple subsets.

[0165] like Fig.11 As shown in the figure, it is a schematic diagram of the clustering results of the hand motion capture dataset, wherein each cluster center represents a type of hand motion, such as (a) represents "OK motion", (b) represents "fist motion", (c) represents "T-pose motion", (d) represents "Y-hand motion", (e) represents "single-finger motion", and (f) represents "grasping motion".

[0166] It should be noted that the type of hand movement represented by each cluster center can be automatically determined by a supervised clustering algorithm, or an unsupervised or semi-supervised clustering algorithm can be used to manually add the type of hand movement to the clustering result.

[0167] It should be noted that the embodiments of the present application do not impose restrictive requirements on clustering algorithms, including but not limited to K-means algorithm, Gaussian mixture clustering algorithm, Mean Shift algorithm, etc.

[0168] S2032: extracting a preset number of hand motion capture data from subsets of different categories respectively, splicing the hand motion capture data belonging to different categories, and synthesizing a plurality of new hand motion capture data.

[0169] In one example, 2 to 5 hand motion capture data are randomly extracted from subsets of different categories, and the hand motion capture data belonging to different categories are spliced ​​into new hand motion capture data.

[0170] In one example, according to different specific gesture requirements, a certain amount of hand motion capture data can be randomly extracted from a subset of a specified hand category and spliced ​​into a specific 3D gesture according to certain rules.

[0171] For example, Fig.12 As shown, after splicing the hand motion capture data of the "T-pose action" type and the hand motion capture data of the "Fist action" type, the "Rock&Roll" type of hand motion capture data is obtained.

[0172] S2033: Perform quaternion interpolation on the synthesized new hand motion capture data and the old hand motion capture data before the synthesis to obtain a hand motion data set.

[0173] In order to ensure the continuity of gesture changes, the old hand motion capture data g old And the synthesized new hand motion capture data g new , using the quaternion interpolation method, in g old and g new Interpolate between to generate a smooth sequence of gesture transformations and obtain the hand action dataset Θ gesture .

[0174] S204: Combine the arm motion dataset, the wrist motion dataset, and the hand motion dataset to generate a 3D gesture dataset.

[0175] The arm motion dataset, wrist motion dataset and hand motion dataset generated by three different random methods based on forward kinematics are combined together to obtain the final 3D gesture dataset.

[0176] It should be noted that the generation methods of the motion data sets of the upper arm, lower arm, wrist and hand provided in the embodiment of the present application are independent of each other, and therefore, are generated in an asynchronous manner. In the generation process of the motion data sets of different parts, the values ​​of the same variable are different. For example, the rotation angle of the wrist joint is different from the rotation angle of the elbow joint, but both are represented by θ i express.

[0177] See also Fig.13, which is an overall architecture diagram for generating a 3D gesture data set provided in an embodiment of the present application, adopts three different forward kinematics methods, namely, a random continuous function generation method, a random walk-based angle change generation method, and a splicing method based on an existing hand motion capture data set, to generate motion data sets of the upper arm, wrist, and hand, respectively, so that the motion data sets have both high randomness and no unreachable gestures, meeting the diversity and accuracy of the gesture data sets. At the same time, in order to improve the randomness of the lower arm motion data set, a random number that obeys a uniform distribution is generated within a preset interval. Through the comparison result of the random number with a preset threshold, a random continuous function generation method and a random walk-based angle change generation method are adopted to generate the motion data set of the lower arm. Finally, the motion data sets of the upper arm, lower arm, wrist, and hand are combined to obtain a complete 3D gesture data set.

[0178] In a method for generating 3D gesture data provided in an embodiment of the present application, a method for generating 3D gesture data based on a random continuous function, a method for generating angle changes based on random walks, and a method for splicing existing hand motion capture data sets are used to asynchronously generate arm, wrist and hand motion data, thereby improving the diversity of gesture data sets and the efficiency of generating gesture data sets; and these three generating methods are based on the principle of forward kinematics and can avoid the appearance of hand gestures that are unreachable in the display world, so that the generated 3D gesture data set meets ergonomics and has higher accuracy. On the other hand, it avoids dependence on multi-camera acquisition systems and motion capture systems to a certain extent, and can generate reasonable gesture data that meets the use scenarios of AR / VR products at a lower cost.

[0179] Based on the same technical concept, an embodiment of the present application provides an electronic device, which can be a terminal device, such as a laptop computer, a desktop computer, etc., or a server, such as a server cluster, a cloud server, etc. The electronic device can implement the steps of the above-mentioned method for generating a 3D gesture data set and can achieve the same technical effect.

[0180] See also Fig.14 The electronic device includes a processor 1401, a memory 1402 and a communication interface 1403, wherein the communication interface 1403, the memory 1402 and the processor 1401 are connected via a bus;

[0181] The memory 1402 may be a volatile memory, such as a random-access memory (RAM); the memory 1402 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1402 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1402 may be a combination of the above memories;

[0182] The processor 1401 may include one or more central processing units (CPU) or a digital processing unit, etc.;

[0183] The communication interface 1403 is used to communicate with other terminal devices and other servers;

[0184] The memory 1402 stores a computer program, and the processor 1401 performs the following operations according to the computer program:

[0185] Randomly generate N continuous functions, and generate arm motion data sets based on the N continuous functions; each continuous function represents a movement of the arm, and the end point of the previous continuous function is used as the starting point of the next continuous function to ensure the continuity of the arm motion;

[0186] The rotation angle of the wrist joint is adjusted by random walk, and the wrist motion data set is generated according to the rotation angle;

[0187] Based on the existing hand motion capture dataset, synthesize the hand motion dataset;

[0188] The arm motion dataset, the wrist motion dataset and the hand motion dataset are combined to generate a 3D gesture dataset; wherein the different motion datasets are generated in an asynchronous manner.

[0189] Optionally, the arm includes an upper arm and a lower arm, and the upper arm and the lower arm are in the same coordinate system;

[0190] The processor 1401 randomly generates N continuous functions, and generates an arm motion data set according to the N continuous functions. The specific operations are as follows:

[0191] Randomly generate N first continuous functions, and generate an upper arm motion data set according to the N first continuous functions;

[0192] Generate a random number that follows a uniform distribution within a preset interval, and determine whether the random number is greater than a preset threshold;

[0193] If it is greater than, N second continuous functions are randomly generated, and the motion data set of the lower arm is generated according to the N second continuous functions; otherwise, the rotation angle of the elbow joint around the lower arm is adjusted by random walk, and the motion data set of the lower arm is generated according to the rotation angle.

[0194] Optionally, the processor 1401 randomly generates N first continuous functions, and generates an upper arm motion data set according to the N first continuous functions, and the specific operation is:

[0195] Initialize the orientation of the shoulder joint of the upper arm in the T-pose state in the three-dimensional space, where the front orientation of the human body is the positive direction of the ZZ axis in the three-dimensional space, the standing direction of the human body is the positive direction of the Y axis in the three-dimensional space, and the positive direction of the X axis in the three-dimensional space and the positive directions of the Z axis and the Y axis satisfy the right-hand rule;

[0196] A starting point is randomly generated in the XOY plane, and the starting point is used as the starting point of the first first continuous function among the N first continuous functions, and a set of first plane point sets of the shoulder joint in the three-dimensional space after the first movement is calculated by using the forward kinematics solution method;

[0197] The end point in the first plane point set is used as the starting point of the next first continuous function, and the forward kinematics solution calculation is repeated until N first continuous functions are traversed;

[0198] The motion data set of the upper arm is determined based on the calculated N groups of first plane point sets of the shoulder joint.

[0199] Optionally, the process of the processor 1401 calculating a set of first plane point sets of the upper arm in the three-dimensional space according to each first continuous function is:

[0200] According to the parity of the number of movements corresponding to the current first continuous function, n sampling points are sampled equidistantly on the X-axis; wherein, when the number of movements is an odd number, the sampling direction of the X-axis is from left to right, and when the number of movements is an even number, the sampling direction of the X-axis is from right to left;

[0201] According to the characteristics of standard normal distribution, the coefficients and periods of Fourier series are randomly generated;

[0202] According to the coefficient and the period, the function value corresponding to each sampling point is determined respectively to obtain a set of first plane point sets.

[0203] Optionally, the processor 1401 determines the motion data set of the upper arm according to the calculated N sets of first plane point sets of the shoulder joint, and the specific operation is:

[0204] Using vector outer product, calculate the first unit rotation axis of each first plane point in the N sets of first plane points after the shoulder joint is initialized;

[0205] Using vector inner product, calculate the first rotation angle of each first plane point in the N sets of first plane points after the shoulder joint is initialized;

[0206] A motion data set of the upper arm is obtained according to each first unit rotation axis and each first rotation angle.

[0207] Optionally, the processor 1401 randomly generates N second continuous functions, and generates a motion data set of the lower arm according to the N second continuous functions. The specific operation is:

[0208] Initialize the rotation matrix of the elbow joint around which the lower arm moves in three-dimensional space;

[0209] An initial angle is randomly generated and used as the starting point of the first second continuous function among N second continuous functions. A forward kinematics solution method is used to calculate a set of second plane point sets of the elbow joint in three-dimensional space after the first movement.

[0210] The rotation angle corresponding to the end point in the second plane point set is used as the starting point of the next second continuous function, and the forward kinematics solution calculation is repeated until N second continuous functions are traversed;

[0211] The motion data set of the lower arm is determined based on the calculated N sets of second plane point sets of the elbow joint.

[0212] Optionally, the process of the processor 1401 calculating a set of second plane point sets of the elbow joint in the three-dimensional space according to each second continuous function is:

[0213] Randomly sample a value in the preset interval. If the value is less than the midpoint of the preset interval, increase the initial angle to obtain the upper limit of the angle. Otherwise, decrease the initial angle to obtain the lower limit of the angle.

[0214] According to one of the upper limit and the lower limit of the angle and the initial angle, a sampling interval is determined, and n sampling points are sampled at equal intervals in the sampling interval;

[0215] The center and radius of the elbow joint movement are randomly generated, and the coordinates of each sampling point in the XOY plane are determined according to the center and radius to obtain a set of second plane point sets.

[0216] Optionally, the processor 1401 determines the motion data set of the lower arm according to the calculated N sets of second plane point sets of the elbow joint, and the specific operation is:

[0217] Using the vector outer product, calculate the second unit rotation axis of each second plane point in the N sets of second plane point sets after the shoulder joint is initialized;

[0218] Using vector inner product, calculate the second rotation angle of each second plane point in the N sets of second plane point sets after the shoulder joint is initialized;

[0219] According to each second unit rotation axis and each second rotation angle, in combination with the rotation matrix, a motion data set of the lower arm is obtained.

[0220] Optionally, for the joints to be rotated in the wrist joint and the elbow joint, the processor 1401 generates a motion data set of the wrist or lower arm according to the rotation angle in the following manner:

[0221] According to the preset rotation angle interval, the probability that the joint to be rotated moves to the upper limit and the lower limit of the rotation angle interval at the current moment is determined, and the movement direction of the joint to be rotated is randomly determined according to the two probabilities;

[0222] According to the direction of movement and the rotation angle at the current moment, an angle sampling interval is determined;

[0223] Sample n sampling points equidistantly in the angle sampling interval, and use the gamma function to calculate the sampling probability of each sampling point;

[0224] According to each sampling probability, an angle sampling point is selected as the rotation angle of the next moment, and combined with the preset data length, equidistant sampling is performed between the rotation angle of the current moment and the rotation angle of the next moment to obtain an arithmetic progression, and the arithmetic progression is used as a set of action data of the joint to be rotated;

[0225] The rotation angle at the next moment is used as the rotation angle at the current moment, and the above process is repeated N times to obtain multiple sets of motion data of the joint to be rotated;

[0226] Multiple groups of motion data of the joints to be rotated are used as motion data sets of corresponding parts.

[0227] Optionally, the processor 1401 synthesizes a hand motion dataset according to an existing hand motion capture dataset, and the specific operations are as follows:

[0228] Cluster the existing hand motion capture dataset to obtain multiple subsets, where the cluster center of each subset represents a type of hand motion;

[0229] Extracting a preset number of hand motion capture data from subsets of different categories respectively, splicing the hand motion capture data belonging to different categories, and synthesizing a plurality of new hand motion capture data;

[0230] The synthesized new hand motion capture data and the old hand motion capture data before synthesis are interpolated by quaternion to obtain a hand motion data set.

[0231] It should be noted that Fig.14 This is just an example, and provides the necessary hardware for an electronic device to execute the steps of a method for generating a 3D gesture data set provided in an embodiment of the present application. If not shown, the electronic device may also include conventional hardware such as a speaker, a pickup, a communication interface, and a power supply.

[0232] Embodiments of the present application Fig.14 The processor involved may be a central processing unit (CPU), a general-purpose processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.

[0233] An embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete a method for generating a 3D gesture data set in the aforementioned embodiment.

[0234] An embodiment of the present application further provides a computer program product for storing a computer program, wherein the computer program is used to execute a method for generating a 3D gesture data set in the aforementioned embodiment.

[0235] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0236] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0237] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0238] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0239] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating 3D gesture data, characterized in that: include: Randomly generate N continuous functions, and generate an arm motion data set according to the N continuous functions; wherein each continuous function represents a movement of the arm, and the end point of the previous continuous function is used as the starting point of the next continuous function to ensure the continuity of the arm motion; A random walk method is used to adjust the rotation angle of the wrist joint, and a wrist motion data set is generated according to the rotation angle; Based on the existing hand motion capture dataset, synthesize the hand motion dataset; The arm motion data set, the wrist motion data set and the hand motion data set are combined to generate a 3D gesture data set; wherein the different motion data sets are generated in an asynchronous manner.

2. The method according to claim 1, characterized in that The arm comprises an upper arm and a lower arm, and the upper arm and the lower arm are in the same coordinate system; The randomly generating N continuous functions and generating the arm motion data set according to the N continuous functions comprises: Randomly generate N first continuous functions, and generate the upper arm motion data set according to the N first continuous functions; Generate a random number that follows a uniform distribution within a preset interval, and determine whether the random number is greater than a preset threshold; If it is greater than, N second continuous functions are randomly generated, and the motion data set of the lower arm is generated according to the N second continuous functions; otherwise, the rotation angle of the elbow joint around the lower arm is adjusted by random walk, and the motion data set of the lower arm is generated according to the rotation angle.

3. The method according to claim 2, characterized in that The randomly generating N first continuous functions and generating the upper arm motion data set according to the N first continuous functions includes: Initialize the orientation of the shoulder joint of the upper arm in the three-dimensional space in the T-pose state, wherein the front orientation of the human body is the positive direction of the Z axis in the three-dimensional space, the standing direction of the human body is the positive direction of the Y axis in the three-dimensional space, and the positive direction of the X axis in the three-dimensional space and the positive directions of the Z axis and the Y axis satisfy the right-hand rule; A starting point is randomly generated in the XOY plane, and the starting point is used as the starting point of the first first continuous function among the N first continuous functions, and a forward kinematics solution method is used to calculate a set of first plane point sets of the shoulder joint in the three-dimensional space after the first movement; Taking the end point in the first plane point set as the starting point of the next first continuous function, repeating the forward kinematics solution calculation until the N first continuous functions are traversed; The motion data set of the upper arm is determined according to the calculated N groups of first plane point sets of the shoulder joint.

4. The method according to claim 3, characterized in that The process of calculating a set of first plane point sets of the upper arm in the three-dimensional space according to each first continuous function comprises: According to the parity of the number of movements corresponding to the current first continuous function, n sampling points are sampled equidistantly on the X-axis; wherein, when the number of movements is an odd number, the sampling direction of the X-axis is from left to right, and when the number of movements is an even number, the sampling direction of the X-axis is from right to left; According to the characteristics of standard normal distribution, the coefficients and periods of Fourier series are randomly generated; According to the coefficient and the period, the function value corresponding to each sampling point is determined respectively to obtain a set of first plane point sets.

5. The method according to claim 3 or 4, characterized in that Determining the motion data set of the upper arm according to the calculated N groups of first plane point sets of the shoulder joint includes: Using vector outer product, calculate the first unit rotation axis of the shoulder joint after initialization and rotate it to each first plane point in the N groups of first plane points; Using vector inner product, calculate the first rotation angle of the shoulder joint after initialization to each first plane point in the N groups of first plane points; A motion data set of the upper arm is obtained according to each first unit rotation axis and each first rotation angle.

6. The method according to claim 2, characterized in that The randomly generating N second continuous functions and generating the motion data set of the lower arm according to the N second continuous functions includes: Initialize the rotation matrix of the elbow joint around which the lower arm moves in three-dimensional space; Randomly generate an initial angle, and use the initial angle as the starting point of the first second continuous function among the N second continuous functions, and use a forward kinematics solution method to calculate a set of second plane point sets of the elbow joint in the three-dimensional space after the first movement; Taking the rotation angle corresponding to the end point in the second plane point set as the starting point of the next second continuous function, repeating the forward kinematics solution calculation until the N second continuous functions are traversed; The motion data set of the lower arm is determined according to the calculated N groups of second plane point sets of the elbow joint.

7. The method according to claim 6, characterized in that The process of calculating a set of second plane point sets of the elbow joint in the three-dimensional space according to each second continuous function comprises: Randomly sample a value in the preset interval, if the value is less than the midpoint of the preset interval, increase the initial angle to obtain an upper limit of the angle, otherwise, decrease the initial angle to obtain a lower limit of the angle; Determine a sampling interval according to one of the angle upper limit and the angle lower limit and the initial angle, and sample n sampling points equidistantly in the sampling interval; The center and radius of the elbow joint movement are randomly generated, and the coordinates of each sampling point in the XOY plane are determined according to the center and the radius to obtain a set of second plane point sets.

8. The method according to claim 6 or 7, characterized in that Determining the motion data set of the lower arm according to the calculated N sets of second plane point sets of the elbow joint comprises: Using the vector outer product, calculate the second unit rotation axis of the shoulder joint after the initialization rotation to each second plane point in the N groups of second plane point sets; Using vector inner product, calculate the second rotation angle of each second plane point in the N sets of second plane point sets after the shoulder joint is initialized; According to each second unit rotation axis and each second rotation angle, in combination with the rotation matrix, a motion data set of the lower arm is obtained.

9. The method according to claim 1 or 2, characterized in that: Methods for generating a wrist or lower arm motion data set according to the rotation angle include: For the joints to be rotated in the wrist and elbow, do the following: According to a preset rotation angle interval, determine the probability that the joint to be rotated moves toward the upper limit and the lower limit of the rotation angle interval at the current moment, and randomly determine the movement direction of the joint to be rotated according to the two probabilities; Determine an angle sampling interval according to the movement direction and the rotation angle at the current moment; Sampling n sampling points equidistantly in the angle sampling interval, and calculating the sampling probability of each sampling point using a gamma function; According to each sampling probability, an angle sampling point is selected as the rotation angle at the next moment, and in combination with a preset data length, equidistant sampling is performed between the rotation angle at the current moment and the rotation angle at the next moment to obtain an arithmetic progression, and the arithmetic progression is used as a set of action data of the joint to be rotated; The rotation angle at the next moment is used as the rotation angle at the current moment, and the above process is repeated N times to obtain multiple sets of motion data of the joint to be rotated; The multiple groups of motion data of the joints to be rotated are used as motion data sets of corresponding parts.

10. The method according to claim 1, characterized in that The step of synthesizing a hand motion dataset based on an existing hand motion capture dataset includes: Cluster the existing hand motion capture dataset to obtain multiple subsets, where the cluster center of each subset represents a type of hand motion; Extracting a preset number of hand motion capture data from subsets of different categories respectively, splicing the hand motion capture data belonging to different categories, and synthesizing a plurality of new hand motion capture data; The synthesized new hand motion capture data and the old hand motion capture data before synthesis are interpolated by quaternion to obtain a hand motion data set.

Images