A motion data optimization method, computer device and storage medium

Abstract

The present invention belongs to the technical field of motion data optimization, and specifically discloses a motion data optimization method, a computer device, and a storage medium. The present invention optimizes motion data by combining a stacked autoencoder with a convolutional autoencoder, thereby improving the position accuracy and smoothness of the motion data output by the network. By increasing the number of stacked autoencoders, the smoothness of the optimized motion data is gradually improved, and the position accuracy of the data generated by the motion data optimized by the convolutional autoencoder is improved. In addition, by adding a time attention mechanism inside the bidirectional cyclic autoencoder and the convolutional autoencoder, the motion data optimization model pays more attention to the hand motion information in the motion data, thereby improving the quality of sign language motion optimization. In addition, the present invention also adopts a smoothing algorithm based on finite differences, constructs a difference matrix to achieve smoothing, and solves the problem of reduced smoothness of motion data after the motion data passes through the convolutional autoencoder.

Description

Technical Field

[0001] The present invention belongs to the technical field of motion data optimization, and specifically relates to a motion data optimization method, computer equipment and storage medium, and is particularly suitable for application fields such as film and television animation, games, sports and medical rehabilitation. Background Art

[0002] Sports data is widely used in sports science, sports research, social interaction, game production, medical rehabilitation and other fields. In sports science and sports research, sports data analysis can help scientists and researchers evaluate athletes' physical fitness and sports efficiency, discover sports rules and optimize sports strategies. In the fields of film and television and game production, sports data gives virtual characters smooth movements, realistic scene effects, etc., greatly reducing production costs. Among them, sports data plays a vital role, so the acquisition and optimization of sports data has become the focus and difficulty.

[0003] At present, the mainstream motion data optimization methods can be mainly divided into optimization algorithms based on interpolation, filtering, dimensionality reduction, and deep learning. However, the optimization algorithm based on interpolation cannot handle scenarios where the data missing time is greater than 0.5s, and because the optimization algorithm based on interpolation needs to be reconstructed and estimated, it requires reliable motion data. When the missing data point information is unavailable, the interpolation optimization algorithm cannot optimize the motion data. The optimization algorithm based on filtering often relies on the accuracy of the model. If the model does not match the actual situation, it may lead to poor optimization results. In addition, human body movements are highly coordinated, the movements of various joints are not independent, and the degrees of freedom are highly correlated, so the use of filtering is not effective. The optimization algorithm based on dimensionality reduction may lose some important information during the dimensionality reduction process, thereby affecting the accuracy of subsequent optimization results.

[0004] Compared with motion data optimization methods based on interpolation, filtering, and dimensionality reduction, motion data optimization algorithms based on deep learning show more powerful data processing capabilities than traditional methods. However, when deep learning optimization algorithms process motion data, there are mainly the following problems: (1) The problem of motion generalization. The model may be difficult to generalize to motion types or individual differences that do not appear in the training data. Because of the diversity of scenes, the optimization effect of the model may be affected in different scenes; (2) The optimization model usually requires a lot of computing resources. The model may overfit on the training data, resulting in poor performance on new motion data; (3) Motion data is time series data. The model needs to be able to capture the dependencies in the time series. For some long motion sequences, the model may find it difficult to capture long-distance time dependencies. Summary of the invention

[0005] The purpose of the present invention is to propose a motion data optimization method, which optimizes motion data by combining a stacked autoencoder with a convolutional autoencoder, wherein the accuracy of posture estimation is improved by utilizing the bidirectional information transmission of the model, and the smoothness of the action is enhanced by utilizing the mapping process from motion data to latent variable space and then to motion data. By combining the stacked autoencoder with the convolutional autoencoder, the smoothing error and position error of the motion data are optimized, and the output maintains a high accuracy and has a reasonable skeleton structure to improve the detail level of the motion data.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] A motion data optimization method comprises the following steps:

[0008] Step 1. Preprocess the collected human motion data and construct a training data set;

[0009] Step 2. Build a motion data optimization model, which includes a smoothing optimization module, a position optimization module, and a matrix optimization module;

[0010] The smooth optimization module uses an improved stacked autoencoder to perform smooth optimization on the motion data; wherein the improved stacked autoencoder is composed of a plurality of bidirectional cyclic autoencoders; after the motion data passes through a bidirectional cyclic autoencoder, the optimized motion data is input into the next bidirectional cyclic autoencoder again, and the bidirectional cyclic autoencoder is repeatedly used for optimization to obtain a smooth optimization result; wherein a time attention mechanism is introduced between the encoder and the decoder of each bidirectional cyclic autoencoder, and when processing the motion data, more attention is paid to the motion details in the hand data sequence to improve the smoothness of the sign language motion optimization; the position optimization module uses an improved convolutional autoencoder; the smooth optimization result is input into the improved convolutional autoencoder for position error optimization to obtain the final optimized motion data; wherein a time attention mechanism is introduced between the encoder and the decoder of the convolutional autoencoder, and when processing the motion data, more attention is paid to the motion information of the hand data sequence part to improve the quality of the sign language motion position optimization; the matrix optimization module adopts a smoothing algorithm based on finite differences to construct a difference matrix to achieve the smoothing of the final optimized motion data to solve the problem of reduced smoothness of the motion data after the motion data passes through the convolutional autoencoder;

[0011] Step 3. Use the training data set constructed in step 1 to train the motion data optimization model constructed in step 2, and use the trained motion data optimization model to optimize the human motion data input to the model.

[0012] Based on the motion data optimization method, the present invention further proposes a computer device, which includes a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement the motion data optimization method mentioned above.

[0013] On the basis of the motion data optimization method, the present invention further proposes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the motion data optimization method mentioned above.

[0014] The present invention has the following advantages:

[0015] As described above, for the problem of optimizing human skeletal motion data, the present invention relates to a motion data optimization method, in which a motion data optimization model is built, and the model optimizes motion data by combining a stacked autoencoder with a convolutional autoencoder, improves the accuracy of posture estimation by using the bidirectional information transmission of the model, and enhances the smoothness of the action by using the mapping process from motion data to latent variable space and then to motion data. In order to ensure the optimization of sign language motion, a time attention mechanism is introduced between the encoder and decoder of each bidirectional cyclic autoencoder and convolutional autoencoder in the stacked autoencoder, so that the motion data optimization model pays more attention to the motion information of the hand data sequence part, and improves the optimization quality of sign language motion. The present invention combines the stacked autoencoder with the convolutional autoencoder to optimize the smoothing error and position error of the motion data, and applies mean square error constraints, bone length constraints and smoothness constraints to the output of each bidirectional cyclic autoencoder in the stacked autoencoder during the training process. Wherein the latent variable constraints are returned by the pre-trained perceptual autoencoder, which can keep the network output at a high precision and have a reasonable bone structure, so that the method of the present invention is suitable for improving the level of detail of motion data. The method of the present invention can optimize human skeletal motion data and can be applied to the fields of film and television animation, games, sports and medical rehabilitation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flow chart of a motion data optimization method in an embodiment of the present invention.

[0017] Figure 2 4 is a flowchart of data preprocessing in an embodiment of the present invention.

[0018] Figure 3 It is a structural block diagram of the motion data optimization model in an embodiment of the present invention.

[0019] Figure 4 This is a smoothing optimization flow chart in an embodiment of the present invention.

[0020] Figure 5This is a diagram of the improved bidirectional recurrent autoencoder network structure in an embodiment of the present invention.

[0021] Figure 6 This is a flowchart for implementing the attention mechanism in an embodiment of the present invention.

[0022] Figure 7 This is a flowchart of an improved bidirectional recurrent autoencoder network in an embodiment of the present invention.

[0023] Figure 8 This is a diagram of the stacked autoencoder network structure in an embodiment of the present invention.

[0024] Fig. 9 This is a flow chart of position optimization in an embodiment of the present invention.

[0025] Fig.10 This is a diagram of the improved convolutional autoencoder network structure in an embodiment of the present invention.

[0026] Fig.11 This is a flowchart of the improved convolutional autoencoder network in an embodiment of the present invention.

[0027] Fig.12 Matrix optimization flow chart in an embodiment of the present invention.

[0028] Fig.13 This is a diagram of the network structure of the perceptual autoencoder in an embodiment of the present invention.

[0029] Fig.14 Schematic diagram of the left hand joint coordinate system.

[0030] Fig.15 Rotate the graph for the joint point. DETAILED DESCRIPTION

[0031] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0032] Example 1

[0033] like Figure 1 As shown, for the problem of optimizing human skeleton motion data, this embodiment describes a motion data optimization method, which includes the following steps:

[0034] Step 1. Preprocess the collected human motion data and construct a training data set. The motion data is stored in a specific file format in the computer, such as BVH, ASF / AFC, TXT, etc. Among these storage formats, BVH and ASF / AFC formats are mostly output file formats of traditional motion acquisition devices, in which the rotation information of the motion data is recorded and saved in real time. The motion data in this embodiment refers to sign language motion data. The TXT format is an output file format of a new and convenient motion acquisition device, which saves the three-dimensional coordinate data of the joint points. The present invention uses DeepMotion for motion capture, and based on technologies such as Animate 3D, real-time body tracking, virtual reality tracking and Motion Brain, it captures and extracts motion data in sign language videos. The motion data obtained by DeepMotion is in BVH format, and the input data of the motion data optimization model of the present invention is often in global coordinate format, so it is necessary to convert the data representation method.

[0035] The collected human motion data is preprocessed as follows: Figure 2 As shown, the specific steps are as follows:

[0036] Step 1.1. In order to improve processing efficiency, reduce data volume, and ensure compatibility of animation with other data, all motion data needs to be downsampled and frequency converted, that is, the animation data is sampled with a step size of 2 and the frame rate is converted from the original 120fps to 60fps.

[0037] Step 1.2. Use the forward kinematics method to calculate the global position and perform coordinate transformation.

[0038] In order to convert the BVH file to the global coordinate format, this paper uses the forward kinematics method to calculate the global position and perform coordinate conversion, such as Fig.14 As shown, take the left hand joint point as an example and place it in the world coordinate system.

[0039] The first joint point (left shoulder joint point) can rotate along the Z axis, and the angle is expressed as Indicates that the second joint point (left arm joint point) rotates along the X axis, and the angle is expressed as Indicates that the third joint point (left forearm joint point) rotates along the X-axis, recorded as .

[0040] These three joints constitute the posture. Draw the local coordinate system of the first joint. Like the world coordinate system, red, green and blue represent , the joint point after rotation is Fig.15 shown.

[0041] The first joint point is equivalent to rotating around the Z axis. Angle, the change from world coordinates to this is written as a homogeneous matrix .

[0042] .

[0043] in, Indicates rotation around the Z axis The rotation matrix corresponding to the angle, T describes the overall translation of the left hand.

[0044] The second joint point is translated by L1 length in the Y direction relative to the first joint point, and along the X axis Angle, so the transformation matrix from world coordinates to the second joint point can be written as .

[0045] .

[0046] in, Represents the rotation matrix corresponding to the rotation of angle β around the X axis.

[0047] The third joint point is the second joint point rotated along the X axis degrees, the Y axis moves L2, and the third joint point to the fourth joint point (i.e. the left hand joint point) moves L3, so the transformation matrix from the world coordinates to the third joint point , the transformation matrix to the fourth joint point .

[0048] .

[0049] .

[0050] in, represents the rotation matrix corresponding to the rotation angle γ around the X axis, Represents the identity matrix.

[0051] Step 1.3. In order to ensure the realism and physical rationality of the character animation, it is necessary to obtain the foot contact information of the motion data and adjust the foot height. Obtaining foot contact information can detect the contact state and determine whether the character's feet are in contact with the ground. It is essential for realizing realistic animation behaviors (such as jumping, walking, running, etc.). Only when the feet are in contact with the ground can the character stand or move normally. By analyzing the contact information of the feet, the character's motion control strategy can be optimized. Foot height adjustment can ensure that the feet are in contact with the ground. By adjusting the height of the feet, the feet can be correctly in contact with the ground in the animation, avoiding the unnatural effect of the character "floating" in the air, so that the motion data can conform to the physical behavior in reality and adapt to different interactive scenarios.

[0052] Step 1.4. In order to improve the naturalness and fluency of the data, it is necessary to obtain the joint velocity and rotation velocity.

[0053] Joint velocity can help adjust the character's movement to make it smoother and more natural. Fast or sudden joint movements can cause jerky movements, and getting joint velocity can avoid this by interpolating and adjusting.

[0054] By analyzing the speed changes of joints, we can ensure that the transition between different animations of the character is more natural, and the speed changes can help achieve smoother action transitions. At the same time, in the character control system, obtaining joint speed and rotation speed helps calculate the motion state of the character, thereby achieving better control in dynamic scenes.

[0055] Step 1.5. Finally, the joint speed, rotation speed, and foot contact information are integrated, and the skeleton is normalized to obtain noisy motion data to ensure the coordination of various parts of the character when moving, produce more natural and realistic motion performance, avoid the character appearing suspended or in unnatural postures at inappropriate times, and enhance the authenticity of the action.

[0056] After preprocessing, the motion data is saved in a compressed array format. The processed motion segments are saved in a three-dimensional array, the first dimension represents different motion segments, the second dimension represents the number of frames in each segment, and the third dimension represents the feature vector of each frame. The training data set consists of two parts, including a precise human motion data set CMU and a data set collected by DeepMotion. The data collected by DeepMotion is used to train and test the model. The precise data set CMU is used to train the perceptual autoencoder as label data. Because the motion data is high-dimensional data containing human skeleton data and contains a large number of degrees of freedom, it is difficult to optimize in high-dimensional space. The present invention converts complex high-dimensional data into a global coordinate format through data preprocessing to facilitate subsequent optimization processing of motion data.

[0057] Step 2. Build a motion data optimization model, whose network structure is as follows Figure 3 As shown, the motion data optimization model includes a smoothing optimization module, a position optimization module and a matrix optimization module.

[0058] The smoothing optimization module uses an improved stacked autoencoder to perform smoothing optimization on motion data.

[0059] Among them, the improved stacked autoencoder consists of multiple bidirectional recurrent autoencoders composition.

[0060] After the motion data passes through a bidirectional cyclic autoencoder, the optimized motion data is input into the next bidirectional cyclic autoencoder again, and the bidirectional cyclic autoencoder is repeatedly used for optimization to obtain a smooth optimization result. .

[0061] A temporal attention mechanism is introduced between the encoder and decoder of each bidirectional recurrent autoencoder, so that the model can focus more on the motion details in the hand data sequence when processing motion data, thereby improving the smoothness of sign language motion optimization.

[0062] The position optimization module uses an improved convolutional autoencoder. The smooth optimization result of the improved stacked autoencoder output is Input into the improved convolutional autoencoder for position error optimization to obtain the final optimized motion data Y.

[0063] A temporal attention mechanism is introduced between the encoder and decoder of the convolutional autoencoder, so that the model pays more attention to the motion information of the hand data sequence to improve the quality of sign language motion position optimization.

[0064] The matrix optimization module adopts a smoothing algorithm based on finite differences to construct a difference matrix to achieve the final optimization of the smoothing of the motion data, so as to solve the problem of reduced smoothness of the motion data after passing through the convolutional autoencoder.

[0065] In the field of motion data processing, the mapping process from motion data to latent variable space and then back to motion data can obtain smooth and natural motion, mainly because the latent variable space can capture the potential structure and regularities in the data.

[0066] By mapping motion data to latent variable space, the present invention can transform high-dimensional and complex data into low-dimensional representation, remove noise and redundant information in the data, and thus better express the essential characteristics of the data.

[0067] In order to better optimize the smoothness of motion data, stacked autoencoders are used for optimization. The motion data can be repeatedly mapped to obtain smoother motion data. The smooth optimization process is as follows: Figure 4 shown.

[0068] like Figure 5 As shown, the bidirectional recurrent autoencoder includes an encoder, a temporal attention mechanism, and a decoder. After the motion data is preprocessed to obtain noisy motion data, the data is input into the bidirectional recurrent autoencoder for position optimization.

[0069] The encoder and decoder of the bidirectional recurrent autoencoder are symmetrical and both consist of three fully connected layers and a bidirectional long short-term memory unit. The fully connected layer is used for motion data prediction and defect repair.

[0070] Bidirectional long short-term memory units are used to process data with time series characteristics.

[0071] The temporal attention mechanism enables the model to pay more attention to certain time points in the sequence. When the model processes sequence data, a weight is assigned to each time point in the sequence. This weight indicates the importance of the time point to the model output.

[0072] The encoder of the bidirectional recurrent autoencoder first maps the noisy human skeletal motion data to a latent variable space, that is, outputs a latent variable space, which is usually a vector of fixed dimension that encodes a compressed representation of the input sequence.

[0073] The temporal attention mechanism provides a method for the decoder of the bidirectional recurrent autoencoder to dynamically select important parts of the input sequence based on the latent variable space, and generates a weighted context vector, which will be used as the input of the decoder to help the decoder focus on the sign language sequence part when generating the output.

[0074] The decoder of the bidirectional recurrent autoencoder maps the latent variables back to the motion data space to obtain the optimized motion data.

[0075] The role of the attention mechanism is to provide the decoder with a method to dynamically select important parts of the input sequence based on this latent variable space and generate a weighted context vector.

[0076] This context vector will be used as input to the decoder and will help the decoder focus on which parts of the sign language sequence when generating the output.

[0077] The present invention adopts the temporal attention mechanism, which enables the model to pay more attention to certain specific time points in the sequence. When the model processes sequence data, a weight is assigned to each time point in the sequence, which indicates the importance of the time point to the model output. The size of the weight determines the degree of attention that the model should pay to each time point when predicting.

[0078] like Figure 6 As shown in Figure 2, the specific processing process of the temporal attention mechanism is as follows:

[0079] First, the attention score is calculated.

[0080] Defining motion data sequences , calculates the correlation between the current time step and all other time steps , with a special focus on hand movement information; represents the motion data of the i-th time step, .

[0081] ;in is the score function, is the hidden state at the previous time step.

[0082] Secondly, calculate the normalized score; use the softmax function to normalize the scores between the hand motion features to obtain the normalized attention weight , to ensure that the weights of all time steps add up to 1.

[0083] .

[0084] Finally, a weighted sum is performed to obtain a weighted context vector; the normalized attention weights are applied to the hand motion features and a weighted sum is performed to obtain a weighted feature representation ; .

[0085] in, represents the motion data of the i-th time step, .

[0086] like Figure 7 A flow chart of an improved bidirectional recurrent autoencoder network in an embodiment of the present invention is shown.

[0087] The encoder of the bidirectional recurrent autoencoder includes three fully connected layers and one bidirectional long short-term memory unit, which are the fully connected layers FC1, FC2, FC3, and the first bidirectional long short-term memory unit.

[0088] The motion data input into the encoder of the bidirectional recurrent autoencoder is processed in sequence by the input layer, FC1, FC2, FC3, and the first bidirectional long short-term memory unit to obtain a latent variable space.

[0089] The sample size of the input layer of the bidirectional recurrent autoencoder is 3480×73, which means that each input sample contains 3480 time steps, each time step has 73 features, 3480 represents the length of the time series, and 73 represents the feature dimension of each time point.

[0090] The input sample sizes of FC1, FC2, and FC3 are 3840×128, 3840×256, and 3840×512, respectively, which means that the size of the input sample contains 3480 time steps, and each time step has 128, 256, and 512 features, respectively.

[0091] The sample size of the first bidirectional long short-term memory unit input is 16×240×512, which means that each sample is a time series with a dimension of 240, and each time point has a 512-dimensional feature vector, with a total of 16 samples.

[0092] The temporal attention mechanism provides a way for the decoder of the bidirectional recurrent autoencoder to dynamically select important parts of the input sequence based on the latent variable space and generate a weighted context vector.

[0093] Three fully connected layers and one bidirectional long short-term memory unit are defined in the decoder of the bidirectional recurrent autoencoder, which are the fully connected layers FC4, FC5, FC6, and the second bidirectional long short-term memory unit.

[0094] The weighted context vector is processed by the second bidirectional long short-term memory unit, FC4, FC5, and FC6 in sequence to obtain optimized motion data, which is then output through the output layer.

[0095] Among them, the sample size of the second bidirectional long short-term memory unit input is 16×240×512, which means that each sample is a time series with a dimension of 240, each time point has a 512-dimensional feature vector, and there are 16 samples in total.

[0096] The input sample sizes of FC4, FC5, and FC6 are 3840×512, 3840×256, and 3840×128, respectively; the corresponding input sample size contains 3480 time steps, and each time step has 512, 256, and 128 features, respectively.

[0097] The sample size of the output layer of the bidirectional recurrent autoencoder is 3480×73, which means that each input sample contains 3480 time steps, each time step has 73 features, 3480 represents the length of the time series, and 73 represents the feature dimension of each time point.

[0098] The bidirectional LSTM considers both past and future information at each time point.

[0099] Because the mapping process from motion data to latent variable space and then to motion data enhances the smoothness of the action, after a bidirectional recurrent autoencoder is optimized, the optimized result is input into the next bidirectional recurrent autoencoder again to form a stacked autoencoder. The stacked autoencoder is a bidirectional recurrent autoencoder ( ) are stacked and the mapping process is repeated to obtain smoother motion data. The network structure is shown in the figure below. Figure 8 shown.

[0100] In stacked autoencoders, .in, represents the result after optimization of the i-1th bidirectional cyclic autoencoder, represents the i-th encoder, represents the i-th decoder.

[0101] The first input to the bidirectional recurrent autoencoder is the noisy data , is the serial number of the bidirectional cyclic autoencoder, and the output of the last bidirectional cyclic autoencoder is used as the smoothed and optimized motion data .

[0102] In the process of smooth optimization, in order to supervise the learning process of the neural network, mean square error constraints, smoothness constraints and latent variable constraints are imposed on the output of each bidirectional recurrent autoencoder in the stacked autoencoder during the training process.

[0103] The latent variable constraints are returned by the perceptual autoencoder pre-trained using an accurate data set, which can enable the network output to maintain a high accuracy and have a reasonable skeleton structure, making the method of the present invention suitable for improving the detail level of motion data.

[0104] After stacking the autoencoder, the output is smoothed and optimized. , the smoothness of the motion data is improved, but the position error increases, so the smooth optimization result is input into the convolutional autoencoder for position optimization.

[0105] As the number of stacked layers of the stacked autoencoder increases, the position error and perception error of the network output data gradually increase. In order to solve the position error problem, the present invention proposes a method of combining the stacked autoencoder with the convolutional autoencoder, and uses the smoothed and optimized motion data as the input of the convolutional autoencoder again to optimize the position error of the motion data.

[0106] The convolutional autoencoder is used to optimize the position error of motion data, such as Fig. 9 shown.

[0107] First, we set the variable scope, control variable reuse, and reshape the input data to meet the model requirements of the convolutional autoencoder. We input the reshaped motion data into the convolutional autoencoder, and introduce the attention mechanism into the convolutional autoencoder to make the model pay more attention to the motion information of the hand data sequence, thus improving the quality of sign language motion position optimization.

[0108] Specifically, the improved convolutional autoencoder consists of three parts: encoder, attention layer attention mechanism and decoder.

[0109] Encoder It consists of convolutional layers, pooling layers, and activation layers.

[0110] The pooling layer is used to take the maximum value between adjacent frames to ensure the invariance of the time domain, speed up the calculation and prevent overfitting of training. The activation layer is to ensure that the network obtains a nonlinear manifold to better fit the motion.

[0111] A temporal attention mechanism is introduced between the encoder and decoder of the convolutional autoencoder to help the convolutional autoencoder better capture and utilize the encoded feature representation, provide the decoder of the convolutional autoencoder with a method to dynamically select the important part of the input sequence, and generate a weighted context vector as the input of the decoder of the convolutional autoencoder. A temporal attention mechanism is introduced between the encoder and decoder to help the convolutional autoencoder better capture and utilize the encoded feature representation, provide the decoder with a method to dynamically select the important part of the input sequence, and generate a weighted context vector as the input of the decoder.

[0112] Decoder It consists of an anti-pooling layer and a convolutional layer. The anti-pooling layer is used to evenly distribute the values ​​of the hidden variables on the two visible units so that the merged visible units remain consistent with those before pooling in the time dimension.

[0113] The network structure of the improved convolutional autoencoder is as follows Fig.10 As shown, the encoder and decoder process is expressed as:

[0114] ;

[0115] ;

[0116] In the formula For the encoder, For the decoder, represents the weight matrix of the convolution kernel in the convolution layer, Represents the motion data after smooth optimization, that is, the output characteristics of the smooth optimization module. represents the bias in the convolutional layer, Represents the maximum pooling function; in the decoder of the convolutional autoencoder As input, through the unpooling operation Subtract the bias , and finally use the weight matrix Perform convolution calculation to obtain the final position-optimized motion data .

[0117] like Fig.11 A network flow chart of an improved convolutional autoencoder in an embodiment of the present invention is shown.

[0118] Encoder There are five convolutional layers and five pooling layers. The five convolutional layers are defined as Conv1, Conv2, Conv3, Conv4, and Conv5, and the five pooling layers are defined as Pool1, Pool2, Pool3, Pool4, and Pool5. Input encoder The motion data features are processed by the input layer, convolution layer Conv1, pooling layer Pool1, convolution layer Conv2, pooling layer Pool2, convolution layer Conv3, pooling layer Pool3, convolution layer Conv4, pooling layer Pool4, convolution layer Conv5 and pooling layer Pool5 in turn to obtain the latent variable space. Among them, the sample size of the input layer is 16×240×73×1, the input size of the convolution layer Conv1 is 16×240×73×32, the input size of the pooling layer Pool1 is 16×240×37×32, the input size of the convolution layer Conv2 is 16×120×37×64, the input size of the pooling layer Pool1 is 16×60×19×64, the input size of the convolution layer Conv3 is 16×60×19×128, the input size of the pooling layer Pool3 is 16×30×10×128, the input size of the convolution layer Conv4 is 16×30×10×256, the input size of the pooling layer Pool4 is 16×15×5×256, the input size of the convolution layer Conv5 is 16×15×5×512, and the input size of the pooling layer Pool5 is 16×8×3×512.

[0119] The temporal attention mechanism provides a way for the decoder of the convolutional autoencoder to dynamically select important parts of the input sequence based on the latent variable space and generate a weighted context vector.

[0120] Decoder There are five convolutional layers and five anti-pooling layers, which are defined as Conv6, Conv7, Conv8, Conv9, and Conv10, and five anti-pooling layers are defined as UnPool1, UnPool2, UnPool3, UnPool4, and UnPool5. The weighted context vector is processed by anti-pooling layer UnPool1, convolution layer Conv6, anti-pooling layer UnPool2, convolution layer Conv7, anti-pooling layer UnPool3, convolution layer Conv8, anti-pooling layer UnPool4, convolution layer Conv9, anti-pooling layer UnPool5, and convolution layer Conv10 to obtain optimized motion data, which is then output through the output layer. Among them, the input size of the unpooling layer UnPool1 is 16×15×5×512, the input size of the convolution layer Conv6 is 16×15×5×256, the input size of the unpooling layer UnPool2 is 16×30×10×256, and the input size of the convolution layer Conv7 is 16×30×10×128; the input size of the unpooling layer UnPool3 is 16×60×19×128, and the input size of the convolution layer Conv The input size of 8 is 16×60×19×64, the input size of the unpooling layer UnPool4 is 16×120×37×64, and the input size of the convolution layer Conv9 is 16×120×37×32; the input size of the unpooling layer UnPool5 is 16×240×73×32, the input size of the convolution layer Conv10 is 16×240×73×1, and the size of the output layer is 16×240×73×1.

[0121] Define the size of the convolutional layer, pooling layer, and unpooling layer as a×b×c×d. Among them, a represents the batch size, which indicates the number of data samples processed by the network during a forward or backward propagation; b×c represents the size of each input or output feature map; d represents the depth of the feature map, that is, the dimension of the feature vector at each spatial position.

[0122] In the process of position optimization, mean square error constraints, smoothness constraints and latent variable constraints are also imposed on the output of the convolutional autoencoder to improve the level of detail of the motion data. After the convolutional autoencoder, the output position optimization result Y greatly reduces the position error of the motion data, but the smoothing error increases. Therefore, the smoothing matrix is ​​optimized to solve the problem of reduced smoothness of the motion data after the convolutional autoencoder.

[0123] Specifically, the present invention adopts a smoothing algorithm based on finite differences and constructs a difference matrix to achieve smoothing, solving the problem of reduced smoothness of motion data after passing through a convolutional autoencoder. By constructing and applying a smoothing matrix, the aim is to reduce noise in the input signal and improve the smoothness of the data. The matrix optimization process is as follows: Fig.12shown.

[0124] I. First, create a smoothing matrix that uses the finite difference method to smooth the signal; the weights of the main diagonal and adjacent diagonals of the smoothing matrix are set to different values ​​to construct a smoothing operation.

[0125] II. Set the main diagonal weight of the smoothing matrix to 2 and the adjacent diagonal weight to -1, and construct a discrete second-order differential operator to smooth the data by comparing the differences between adjacent data points. Define the finite difference matrix S as:

[0126] ;

[0127] Where S is a (n+4)×(n+4) matrix, where n is the length of the input sequence.

[0128] III. By setting the zero-order hold boundary conditions, at the beginning of the input sequence, the first element remains unchanged, and at the end of the input sequence, the last element remains unchanged, ensuring the validity of the smoothing operation at the beginning and end of the input sequence while keeping the values ​​of the boundary elements unchanged, and then converting the smoothing matrix into a three-dimensional tensor, matching the dimension of the input data.

[0129] IV. Expand the position optimization result Y, add two data points at the beginning of the input sequence, these two points are the same as the first data point of the original sequence, to achieve the zero-order hold boundary condition, that is, keep the data unchanged at the beginning of the input sequence; add two data points at the end of the input sequence, these two points are the same as the last data point of the input sequence, to achieve the data unchanged at the end of the input sequence, and finally get the expanded input sequence .

[0130] The input sequence Y is a three-dimensional tensor of size B×S×N, with additional data points added to its time dimension. is a B×1×N matrix, representing the data of the first time step of each batch; let is a B×1×N matrix, representing the data of the last time step of each batch. Then the expanded input data It is expressed as:

[0131] .

[0132] Among them, B represents the number of data samples contained in a batch, S represents the number of time steps or sequence length of each sample, and N represents the number of features in each time step.

[0133] V. Combine the smoothing matrix S with the expanded input data Multiplying together to get a smooth result, the matrix product R of S and Y is:

[0134] .

[0135] The present invention optimizes motion data by combining a stacked autoencoder with a convolutional autoencoder, thereby improving the position accuracy and smoothness of the motion data output by the network.

[0136] By increasing the number of stacked autoencoders, the smoothness of the optimized motion data is gradually improved, and the data position accuracy generated by the motion data optimized by the convolutional autoencoder is improved.

[0137] In addition, by adding a temporal attention mechanism inside the bidirectional recurrent autoencoder and the convolutional autoencoder, the motion data optimization model pays more attention to the hand motion information in the motion data, thereby improving the quality of sign language motion optimization.

[0138] In addition, the present invention also adopts a smoothing algorithm based on finite differences and constructs a difference matrix to achieve smoothing, thereby solving the problem of reduced smoothness of motion data after the motion data passes through the convolutional autoencoder.

[0139] Step 3. Use the training data set constructed in step 1 to train the motion data optimization model constructed in step 2, and use the trained motion data optimization model to optimize the human motion data input to the model.

[0140] Because human motion contains variability and differences, different individuals have different motion habits and motion state characteristics. Human motion data is a continuous time series data, and the previous and subsequent frames of data are correlated and coordinated. Therefore, when optimizing motion data, a loss function is used to perform optimization constraints to ensure that the motion frames remain correlated and smooth, improve the smoothness of the motion data and the accuracy of the position, and at the same time use perception constraints to avoid data that does not conform to the laws of human motion.

[0141] The main function of the loss function module is to supervise the learning process of the neural network. By comparing the prediction results of the neural network with the difference between the labels, the loss of the model is calculated, so that the model can better fit the data, thereby improving the prediction performance of the model.

[0142] During the training process, mean square error constraints, smoothness constraints, and latent variable constraints are imposed on the output of each bidirectional recurrent autoencoder in the stacked autoencoder. The latent variable constraints are returned by the pre-trained perceptual autoencoder, which can keep the network output high accuracy and have a reasonable skeleton structure, making the algorithm suitable for improving the level of detail of motion data.

[0143] Therefore, there are three loss functions in the present invention, namely, joint point position loss function, smoothness loss function, and latent variable loss function. The overall loss function of model training is obtained by weighting the above three loss functions.

[0144] 1. Joint point position loss function .

[0145] In the task of joint point position estimation, the commonly used loss function is mainly the mean squared error (MSE). In joint point position estimation, MSE evaluates the performance of the model by calculating the sum of the squares of the Euclidean distances between the predicted joint point positions and the actual joint point positions, optimizes the model parameters, and makes the prediction more accurate.

[0146] The motion data of the present invention are all displayed in 3D joint coordinates. The mean square error constraint is used during the training process to minimize the Euclidean distance between the joint point position and the actual joint point position. The expression is:

[0147] .

[0148] Among them, f represents the number of frames in the motion sequence, d represents the dimension of each frame of motion data, Represents the precise motion dataset.

[0149] 2. Smoothness loss function .

[0150] In order to make the motion data smoother, a smoothness loss function is used to ensure that the optimized motion data is smoothly transitioned between adjacent frames. The smoothing matrix O is:

[0151] .

[0152] For a given input motion sequence R, the smoothness loss function is calculated as:

[0153] .

[0154] Among them, f represents the number of frames in the motion sequence, d represents the dimension of each frame of motion data, The optimized motion data R is obtained by repeating the motion data of the first frame and the last frame at the first and last frames respectively.

[0155] 3. Hidden variable loss function .

[0156] After passing the joint position constraints and smoothness constraints, the position error and smoothness error of the motion data are reduced. However, the lack of human kinematic constraints may cause skeletal distortion of the data and produce movements that do not conform to human motion trajectories. Therefore, a hidden variable loss function is applied to improve the kinematic rationality of the network output data.

[0157] Convolutional autoencoders are used as the basic architecture of perceptual autoencoders to achieve in-depth understanding and effective processing of high-precision motion capture data. During the training process, high-precision motion capture data is used, which usually contains rich motion details and precise motion trajectories. By deep learning these data, convolutional autoencoders can learn the key features and inherent laws in motion data. Using perceptual loss as a constraint on latent variables can guide the autoencoder to focus not only on the numerical accuracy of the data but also on the perceptual quality of the data during the reconstruction process. This constraint helps the model better capture the subtle differences and dynamic changes in motion data. By optimizing the perceptual loss, the mapping of the motion data to be optimized to the high-precision motion capture data can be achieved, which improves the level of detail of the motion data. The optimized data is not only similar to the high-precision motion capture data in terms of general shape and trajectory, but also significantly improved in terms of motion details, fluency and naturalness.

[0158] The network structure of the autoencoder Fig.13 As shown, the perceptual autoencoder can be expressed as:

[0159] .

[0160] In the formula For the encoder, It is a CMU high-precision motion capture dataset. represents the weight matrix of the convolution kernel in the convolution layer, represents the bias in the convolutional layer, Represents the maximum pooling function.

[0161] The present invention is based on pre-trained perceptual autoencoder , the latent variable loss function is defined as:

[0162] .

[0163] in, is the motion data after passing through i bidirectional cyclic autoencoders, For precise motion datasets, represents the encoder, Indicates the frame number of the motion sequence.

[0164] Overall loss function for model training .

[0165] in, , , Respectively represent the weight coefficients corresponding to each loss function.

[0166] In addition, in order to verify the relationship between the number of stacking layers of the bidirectional recurrent autoencoder in the stacked autoencoder and the position error, smoothing error, and perceptual error, the network structures with stacking layers of 1 to 5 are compared, as shown in Table 1.

[0167] Table 1 Comparison of stacking layers and errors

[0168]

[0169] From the data in Table 1, we can see that as the number of stacking layers increases, the smoothing error of motion data gradually decreases, improving the smoothness of motion data, but the position error and perception error increase. Through the stacking layer experiment, it can be found that the stacking autoencoder method can solve the problem of large smoothing error of motion data.

[0170] In order to solve the problem of increasing position error as the number of stacked autoencoder layers increases, after stacking three bidirectional cyclic autoencoders, the optimization result is used as the input of the convolutional autoencoder to optimize the motion data. The experimental results are shown in Table 2.

[0171] Table 2 Stacked + Convolutional Autoencoder Error Comparison Table

[0172]

[0173] From the data in Table 2, we can see that as the number of stacked layers increases, the position error gradually increases. After passing through the convolutional autoencoder, the position error and perception error are greatly improved, but the smoothing error increases. Through experimental analysis, it can be found that the convolutional autoencoder can solve the problem of large position error of motion data.

[0174] It can be seen from experiments that after the motion data passes through the convolutional autoencoder, the position accuracy is improved, but the smoothing error increases. In order to reduce the smoothing error, the present invention adopts a smoothing algorithm based on the finite difference method, and realizes smoothing by constructing a difference matrix, in which the weights of the main diagonal and the adjacent diagonal of the matrix are set as the core of the smoothing operation.

[0175] This finite difference based smoothing method can be used to remove high frequency noise in the signal and make the data change smoother. Experimental results: The matrix optimization process is as follows: Fig.12 The final experimental results are shown in Table 3.

[0176] Table 3 Optimization matrix error comparison table

[0177]

[0178] It can be seen from Table 3 that after matrix optimization, the position error is optimized by 25.11%, the smoothing error is optimized by 34.14%, and the perception error is optimized by 9.94%, so it has a good optimization effect.

[0179] In order to verify the effect of different methods on motion data optimization, the present invention randomly selected 70% from the CMU database as training data and the remaining 30% as test data. The test data was added with Gaussian noise (SNR=20db) for testing. The test results are shown in Table 4. The present invention compares the performance of different models under the two indicators of position error and smoothing error.

[0180] Among them, the CAE method only contains a single convolutional autoencoder structure; the CAE-P method adds perceptual constraints on the basis of the CAE method; the BRA method only contains a single bidirectional recurrent autoencoder structure; the BRA-P method adds perceptual constraints on the basis of the BRA method; the DCH method adopts a dual CAE structure and imposes latent variable constraints at the same time; the DBH method adopts a dual BRA structure and imposes latent variable constraints at the same time; the BC method contains BRA and CAE structures; the BCP method adds perceptual constraints on the basis of the BC method; and the BCH method adds latent variable constraints on the basis of the BC method.

[0181] Table 4 Stacked + convolutional autoencoder error comparison table

[0182]

[0183] It is not difficult to see from the above Table 4 that the position error of the method of the present invention is 1.334cm, which is 50.62% better than the more traditional CAE algorithm, 32.69% better than the BRA algorithm, and 8.19% better than the BC algorithm that contains both BRA and CAE structures. However, after the BC algorithm imposes hidden variable constraints, the position is more accurate, and the method of the present invention has increased relatively, increasing by 0.67%, but the smoothing has been significantly improved on the basis of the BVH algorithm, and optimized by 22.11%. The smoothing error of the present invention is 0.296cm, which is 31.63% better than the more traditional CAE algorithm. After the CAE algorithm imposes perceptual constraints, the smoothness is well improved, and the smoothing error is 0.303cm, which is the best smoothness among other algorithms, and the method of the present invention is 2.31% better than the algorithm, which effectively shows that the method proposed by the present invention has good performance and ensures the position accuracy and smoothness of motion data.

[0184] The motion data optimization method proposed in the present invention aims at the optimization problem of noisy sign language motion data, and proposes a sign language motion optimization method based on an improved autoencoder. The improved stacked autoencoder and the improved convolutional autoencoder are connected in series, which combines the advantages of the two encoders, and can ensure both the position accuracy and smoothness of the optimized data. The attention mechanism is added to the bidirectional cyclic autoencoder and the convolutional autoencoder, so that the network can focus on the hand sequence, enhance the generalization ability of the model, and accelerate the convergence speed of the model. Finally, a smoothing algorithm based on finite differences is used to ensure that the motion data has a higher smoothness at the detail level, and finally complete the sign language motion optimization.

[0185] Example 2

[0186] This embodiment 2 describes a computer device, which includes a memory and one or more processors. The memory stores executable code. When the processor executes the executable code, it is used to implement the steps of the motion data optimization method in the above embodiment 1.

[0187] In this embodiment, the computer device is any device or apparatus with data processing capability, which will not be described in detail here.

[0188] Example 3

[0189] This embodiment 3 describes a computer-readable storage medium on which a program is stored. When the program is executed by a processor, it is used to implement the steps of the motion data optimization method in the above-mentioned embodiment 1.

[0190] The computer-readable storage medium may be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc., equipped on the device.

[0191] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.

Claims

1. A motion data optimization method, characterized in that: The steps include: Step 1. Preprocess the collected human motion data and construct a training data set; Step 2. Build a motion data optimization model, which includes a smoothing optimization module, a position optimization module, and a matrix optimization module; The smoothing optimization module uses an improved stacked autoencoder to perform smoothing optimization on the motion data; Among them, the improved stacked autoencoder is composed of multiple bidirectional recurrent autoencoders; After the motion data passes through a bidirectional cyclic autoencoder, the optimized motion data is input into the next bidirectional cyclic autoencoder again, and the bidirectional cyclic autoencoder is repeatedly used for optimization to obtain a smooth optimization result; A temporal attention mechanism is introduced between the encoder and decoder of each bidirectional recurrent autoencoder, which focuses more on the motion details in the hand data sequence when processing motion data, so as to improve the smoothness of sign language motion optimization; The position optimization module uses an improved convolutional autoencoder; The smooth optimization result is input into the improved convolutional autoencoder to optimize the position error and obtain the final optimized motion data; A temporal attention mechanism is introduced between the encoder and decoder of the convolutional autoencoder to pay more attention to the motion information of the hand data sequence when processing motion data, so as to improve the quality of sign language motion position optimization; The matrix optimization module uses a smoothing algorithm based on finite differences to construct a difference matrix to achieve the final optimization of the smoothing of the motion data, so as to solve the problem of reduced smoothness of the motion data after passing through the convolutional autoencoder; Step 3. Use the training data set constructed in step 1 to train the motion data optimization model constructed in step 2, and use the trained motion data optimization model to optimize the human motion data input to the model.

2. The motion data optimization method according to claim 1, characterized in that: The bidirectional recurrent autoencoder includes an encoder, a temporal attention mechanism, and a decoder; the encoder and decoder of the bidirectional recurrent autoencoder have symmetrical structures and are both composed of three fully connected layers and a bidirectional long short-term memory unit; The fully connected layer is used for motion data prediction and defect repair; Bidirectional long short-term memory units are used to process data with time series characteristics; The temporal attention mechanism enables the model to pay more attention to certain time points in the sequence. When the model processes sequence data, a weight is assigned to each time point in the sequence to indicate the importance of the time point to the model output. The encoder of the bidirectional recurrent autoencoder first maps the noisy human skeleton motion data into the latent variable space; The temporal attention mechanism provides a dynamic selection of important parts of the input sequence for the decoder of the bidirectional recurrent autoencoder based on the latent variable space, generating a weighted context vector; the weighted context vector will be used as the input of the decoder to help the decoder of the bidirectional recurrent autoencoder focus on the sign language sequence part when generating the output; The decoder of the bidirectional recurrent autoencoder maps the latent variables back to the motion data space to obtain the optimized motion data.

3. The motion data optimization method according to claim 2, characterized in that: The specific processing process of the temporal attention mechanism is as follows: First, the attention score is calculated; Defining motion data sequences , calculates the correlation between the current time step and all other time steps , with special attention to hand movement information, represents the motion data of the i-th time step, ; ； in, is the score function, is the hidden state of the previous time step; Secondly, the normalized score is calculated; Use the softmax function to normalize the scores between hand motion features to obtain the normalized attention weights , to ensure that the weights of all time steps add up to 1; ； Finally, a weighted sum is performed to obtain a weighted context vector; Apply the normalized attention weights to the hand motion features and perform weighted summation to obtain the weighted feature representation ; ； in, represents the motion data of the i-th time step, .

4. The motion data optimization method according to claim 2, characterized in that: Define three fully connected layers and one bidirectional long short-term memory unit in the encoder of the bidirectional recurrent autoencoder, which are the fully connected layers FC1, FC2, FC3, and the first bidirectional long short-term memory unit; The motion data in the encoder of the bidirectional recurrent autoencoder is processed in sequence by the input layer, FC1, FC2, FC3, and the first bidirectional long short-term memory unit to obtain a latent variable space; The sample size of the input layer of the bidirectional recurrent autoencoder is 3480×73, which means that each input sample contains 3480 time steps, each time step has 73 features, 3480 represents the length of the time series, and 73 represents the feature dimension of each time point; The input sample sizes of FC1, FC2, and FC3 are 3840×128, 3840×256, and 3840×512, respectively, which means that the size of the input sample contains 3480 time steps, and each time step has 128, 256, and 512 features, respectively; The sample size of the first bidirectional long short-term memory unit input is 16×240×512, which means that each sample is a time series with a dimension of 240, and each time point has a 512-dimensional feature vector, with a total of 16 samples; The temporal attention mechanism provides a way for the decoder of the bidirectional recurrent autoencoder to dynamically select important parts of the input sequence based on the latent variable space and generate a weighted context vector. Define three fully connected layers and one bidirectional long short-term memory unit in the decoder of the bidirectional recurrent autoencoder, which are the fully connected layers FC4, FC5, FC6, and the second bidirectional long short-term memory unit; The weighted context vector is processed by the second bidirectional long short-term memory unit, FC4, FC5, and FC6 in sequence to obtain optimized motion data, which is output through the output layer; The sample size of the second bidirectional long short-term memory unit input is 16×240×512, which means that each sample is a time series with a dimension of 240, each time point has a 512-dimensional feature vector, and there are 16 samples in total; The input sample sizes of FC4, FC5, and FC6 are 3840×512, 3840×256, and 3840×128, respectively; the corresponding input sample size contains 3480 time steps, and each time step has 512, 256, and 128 features, respectively; The sample size of the output layer of the bidirectional recurrent autoencoder is 3480×73, which means that each input sample contains 3480 time steps, each time step has 73 features, 3480 represents the length of the time series, and 73 represents the feature dimension of each time point.

5. The motion data optimization method according to claim 1, characterized in that: The improved convolutional autoencoder includes three parts: an encoder, a temporal attention mechanism and a decoder; Encoder It consists of convolutional layers, pooling layers, and activation layers; The pooling layer is used to take the maximum value between adjacent frames to ensure the invariance of the time domain, speed up the calculation and prevent overfitting of the training. The activation layer is to ensure that the network obtains a nonlinear manifold to better fit the motion. A temporal attention mechanism is introduced between the encoder and decoder of the convolutional autoencoder to help the convolutional autoencoder better capture and utilize the encoded feature representation, provide a dynamic selection of important parts of the input sequence for the decoder of the convolutional autoencoder, and generate a weighted context vector as the input of the decoder of the convolutional autoencoder; Decoder It consists of an anti-pooling layer and a convolutional layer. The anti-pooling layer is used to evenly distribute the value of the hidden variable on the two visible units so that the merged visible unit is consistent with the one before pooling in the time dimension. The encoder and decoder process of the improved convolutional autoencoder is expressed as: ； ； In the formula For the encoder, For the decoder, represents the weight matrix of the convolution kernel in the convolution layer, Represents the motion data after smooth optimization, that is, the output characteristics of the smooth optimization module. represents the bias in the convolutional layer, represents the maximum pooling function; In the decoder of the convolutional autoencoder, As input, through the unpooling operation Subtract the bias , and finally use the weight matrix Perform convolution calculation to obtain the final position-optimized motion data .

6. The motion data optimization method according to claim 5, characterized in that: The encoder There are five convolutional layers and five pooling layers in the network. The five convolutional layers are defined as Conv1, Conv2, Conv3, Conv4, and Conv5, and the five pooling layers are defined as Pool1, Pool2, Pool3, Pool4, and Pool5. Input encoder The motion data features are processed by the input layer, convolution layer Conv1, pooling layer Pool1, convolution layer Conv2, pooling layer Pool2, convolution layer Conv3, pooling layer Pool3, convolution layer Conv4, pooling layer Pool4, convolution layer Conv5 and pooling layer Pool5 in turn to obtain the latent variable space; Among them, the sample size of the input layer is 16×240×73×1, the input size of the convolution layer Conv1 is 16×240×73×32, the input size of the pooling layer Pool1 is 16×120×37×32, the input size of the convolution layer Conv2 is 16×120×37×64, the input size of the pooling layer Pool2 is 16×60×19×64, the input size of the convolution layer Conv3 is 16×60×19×128, the input size of the pooling layer Pool3 is 16×30×10×128, the input size of the convolution layer Conv4 is 16×30×10×256, the input size of the pooling layer Pool4 is 16×15×5×256, the input size of the convolution layer Conv5 is 16×15×5×512, and the input size of the pooling layer Pool5 is 16×8×3×512; The temporal attention mechanism provides a way for the decoder of the convolutional autoencoder to dynamically select important parts of the input sequence based on the latent variable space and generate a weighted context vector. Decoder There are five convolutional layers and five anti-pooling layers. The five convolutional layers are defined as Conv6, Conv7, Conv8, Conv9, and Conv10, and the five anti-pooling layers are defined as UnPool1, UnPool2, UnPool3, UnPool4, and UnPool5. The weighted context vector is processed by anti-pooling layer UnPool1, convolutional layer Conv6, anti-pooling layer UnPool2, convolutional layer Conv7, anti-pooling layer UnPool3, convolutional layer Conv8, anti-pooling layer UnPool4, convolutional layer Conv9, anti-pooling layer UnPool5, and convolutional layer Conv10 to obtain optimized motion data, which is then output through the output layer. Among them, the input size of the unpooling layer UnPool1 is 16×15×5×512, the input size of the convolution layer Conv6 is 16×15×5×256, the input size of the unpooling layer UnPool2 is 16×30×10×256, and the input size of the convolution layer Conv7 is 16×30×10×128; the input size of the unpooling layer UnPool3 is 16×60×19×128, and the input size of the convolution layer Conv The input size of 8 is 16×60×19×64, the input size of the unpooling layer UnPool4 is 16×120×37×64, and the input size of the convolution layer Conv9 is 16×120×37×32; the input size of the unpooling layer UnPool5 is 16×240×73×32, the input size of the convolution layer Conv10 is 16×240×73×1, and the size of the output layer is 16×240×73×1; Define the size of the convolutional layer, pooling layer, and unpooling layer as a×b×c×d; where a represents the batch size, which indicates the number of data samples processed by the network in one forward or backward propagation; b×c represents the size of each input or output feature map; and d represents the depth of the feature map, that is, the dimension of the feature vector at each spatial position.

7. The motion data optimization method according to claim 1, characterized in that: The processing flow of the matrix optimization module is as follows: I. First, a smoothing matrix is ​​created, which uses the finite difference method to smooth the signal; the weights of the main diagonal and adjacent diagonals of the smoothing matrix are set to different values ​​to construct a smoothing operation; II. Set the main diagonal weight of the smoothing matrix to 2 and the adjacent diagonal weight to -1, construct a discrete second-order differential operator, which aims to smooth the data by comparing the differences between adjacent data points; define the finite difference matrix S as: ； Where S is a (n+4)×(n+4) matrix, where n is the length of the input sequence; III. By setting zero-order hold boundary conditions, at the beginning of the input sequence, the first element remains unchanged, and at the end of the input sequence, the last element remains unchanged, ensuring the effectiveness of the smoothing operation at the beginning and end of the input sequence while keeping the values ​​of the boundary elements unchanged, and then converting the smoothing matrix into a three-dimensional tensor, matching the dimension of the input data; IV. Expand the position optimization result Y by adding two data points at the beginning of the sequence, which are the same as the first data point of the original sequence to achieve the zero-order hold boundary condition, that is, keep the data unchanged at the beginning of the input sequence; Add two data points at the end of the input sequence. These two data points are the same as the last data point of the input sequence to keep the data unchanged at the end of the input sequence. Finally, the expanded input sequence is obtained. ; Define the input sequence Y as a three-dimensional tensor of B×S×N, and add additional data points in its time dimension; is a B×1×N matrix, representing the data of the first time step of each batch; let is a B×1×N matrix, representing the data of the last time step of each batch; then the expanded input data It is expressed as: ； Among them, B represents the number of data samples contained in a batch, S represents the number of time steps or sequence length of each sample, and N represents the number of features in each time step; V. Combine the smoothing matrix S with the expanded input data Multiplying together to get a smooth result, the matrix product R of S and Y is: 。 8. The motion data optimization method according to claim 1, characterized in that: In step 1, the process of preprocessing the collected human motion data is specifically as follows: Step 1.

1. Downsample all motion data for frequency conversion, that is, sample the animation data with a step size of 2, that is, convert the frame rate from the original 120fps to 60fps; Step 1.

2. Use the forward kinematics method to calculate the global position, perform coordinate transformation, remove unnecessary joints, and save the joints that need to be retained into the joint index; Step 1.

3. Obtain foot contact information of motion data and adjust foot height; Step 1.

4. Get the joint velocity and rotation velocity; Step 1.

5. Finally, the joint velocity, rotation speed, and foot contact information are integrated, and the bones are normalized to obtain noisy motion data.

9. A computer device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it is used to implement the motion data optimization method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, it is used to implement the motion data optimization method as described in any one of claims 1 to 8.

Images