A speech-driven 3D human motion method based on a local style encoder

Abstract

This invention discloses a speech-driven 3D human motion method based on a local style encoder, involving general image data processing or generation. It includes a step of constructing a speech-driven stylized 3D human motion overall model and a step of outputting a 3D human motion network with a reference style. The speech-driven stylized 3D human motion overall model extracts local perceptual style features through a local style encoder and extracts semantic, prosodic, and emotional features from the speech. These multiple conditions are injected into a partial perceptual diffusion model, and a 3D human motion with a reference style is synthesized by combining the reference motion style and speech information. This invention can synthesize 3D human motion with a reference style, significantly improving the naturalness and expressiveness of the generated 3D motion sequences.

Description

Technical Field

[0001] This invention relates to general image data processing or generation, and more specifically, to a speech-driven three-dimensional human motion method based on a local style encoder. Background Technology

[0002] With the development of virtual reality (VR), film production, and artificial intelligence, voice-driven 3D human motion generation technology plays a crucial role in these applications. By converting language signals into corresponding dynamic body movements, voice-driven 3D human motion generation technology can bring more natural and realistic expression to virtual humans, enhancing the interactive experience in virtual environments.

[0003] In nonverbal communication, body language serves as a core medium, enhancing the effectiveness of verbal communication. Through body posture, gestures, and facial expressions, individuals can convey additional information, such as thoughts, emotions, and intentions, which can greatly supplement and enrich the content of language. In human interaction, body movements are not merely a supplement to language, but also a crucial tool for conveying emotions and intentions. Therefore, generating highly expressive three-dimensional human motion that can realistically convey emotions has become a major challenge in current technology.

[0004] However, numerous challenges remain in generating virtual human body movements. Particularly in the transformation of emotion, rhythm, and personalized movement styles, ensuring that virtual human movements are both realistic and reflect the emotional nuances and contextual background of the language remains a key challenge for technological development. Currently, existing style coding methods often oversimplify style diversity or ignore the differences in movement styles between different areas of the body (such as the upper and lower body), both of which limit the realism of the generated movements. Furthermore, movement styles should be able to dynamically adapt to changes in the rhythm and emotion of the language, but current methods often overlook this important dynamic characteristic.

[0005] Therefore, there is an urgent need for a new technology that can combine emotion, rhythm, and personalized style to generate three-dimensional human motion that can realistically reflect the emotional and rhythmic changes in speech while also possessing a personalized style. This would make the movement of virtual humans more natural and realistic, dynamically adapting to language rhythm, emotional changes, and personalized style, thereby providing a more immersive virtual interactive experience. Summary of the Invention

[0006] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a speech-driven 3D human motion method based on a local style encoder. This method extracts local perceptual style features through a local style encoder and extracts semantic, prosodic, and emotional features from speech. These multiple conditions are then injected into a partial perceptual diffusion model, and by referencing motion style and speech information, 3D human motion with a reference style is synthesized, thereby improving the naturalness and expressiveness of 3D human motion sequences.

[0007] The present invention discloses a speech-driven 3D human motion method based on a local style encoder, comprising a speech-driven stylized 3D human motion overall model construction step and a 3D human motion network output step with a reference style; wherein,

[0008] The steps for constructing a voice-driven stylized 3D human motion model include:

[0009] S1. Obtain three-dimensional human body mesh sequence samples and original speech, and divide them into training set and test set;

[0010] S2. Preprocess the original speech and extract semantic features, prosodic features and emotional features from the preprocessed original speech;

[0011] S3. Obtain a reference style motion sequence from the three-dimensional human body mesh sequence samples in the training set, and input the reference style motion sequence into the local style encoder for pre-training to obtain local perceptual style features.

[0012] S4. The local perceptual style features, semantic features, prosodic features and emotional features are injected into the partial perceptual diffusion model for training, so as to obtain a speech-driven three-dimensional human motion overall model.

[0013] S5. Measure the performance of the speech-driven stylized 3D human motion model based on the test set, and obtain the best-performing speech-driven stylized 3D human motion model.

[0014] The reference-style 3D human motion network output step is used to input the reference-style motion sequence and speech samples in the actual environment into the best-performing speech-driven stylized 3D human motion overall model, so as to output a reference-style 3D human motion network.

[0015] Preferably, in step S3, the local style encoder includes multiple local style encoders with the same structure.

[0016] Preferably, the local style encoder is constructed as follows:

[0017] S31. Divide the reference style motion sequence of the three-dimensional human body mesh sequence samples in the training set into multiple local motion sequences according to body parts;

[0018] S32. Construct positive and negative sample pairs for the local motion sequence;

[0019] S33. Construct a loss function between the positive and negative sample pairs, and continuously train and optimize the encoder using gradient descent to obtain a local style encoder that can represent local perceptual style features.

[0020] Preferably, the loss function constructed between the positive and negative sample pairs is:

[0021] ;

[0022] In the formula, The loss function constructed between positive and negative sample pairs; |k≠i| τ is the indicator function; τ is the temperature parameter; N s s represents the number of 3D human body mesh sequence samples; i For sample i; s j Let j be the sample.

[0023] Preferably, in step S2, the original speech is preprocessed based on the multi-dimensional information contained in the speech, and then input into the speech encoding module to extract the semantic features, prosodic features and emotional features.

[0024] Preferably, in step S4, the partial sensing diffusion model consists of a partial sensing latent denoiser, a motion encoder, and a motion decoder.

[0025] Preferably, the specific method for training local perceptual style features, semantic features, prosodic features, and emotional features using the partial perceptual diffusion model to obtain a speech-driven three-dimensional human motion overall model is as follows:

[0026] S41. Input the three-dimensional human body mesh sequence samples of the training set into the motion encoder to obtain the original potential motion features;

[0027] S42. Sample random noise and perform T-step noise addition processing on the original potential motion features to obtain noisy potential motion features during the diffusion process.

[0028] S43. Input the semantic features, prosodic features, emotional features, local perceptual style features and noisy latent motion features into a partially perceptual latent denoiser. After T-step denoising, the predicted latent motion features are obtained.

[0029] S44. Construct a loss function between the predicted potential motion features and the original potential motion features, and continuously train and optimize the diffusion model through gradient descent to obtain a speech-driven three-dimensional human motion overall model.

[0030] Preferably, in step S42, the expression for adding Gaussian noise to the original potential motion features is:

[0031] ;

[0032] In the formula, q ( Z t | Z t) This represents the conditional transition distribution for the forward diffusion process; Z t Let represent the potential noise vector at time t∈{1,2,...,T}; Z t-1 α represents the potential noise vector at time t-1 ∈ {1, 2, ..., T-1}; t Let be the diffusion coefficient at each step; This represents the standard Gaussian distribution function.

[0033] Preferably, in step S43, the specific steps are as follows:

[0034] The semantic features, local perceptual style features, and noisy latent motion features are concatenated to construct a joint latent feature. The joint latent feature, along with prosodic features and sentiment features, is input into a partially perceptual latent denoiser composed of multiple Transformer encoders. The denoiser passes sequentially through the self-attention module, prosodic cross-attention module, and sentiment cross-attention module of the Transformer encoders, and is fused with the prosodic features and sentiment features. The fused feature is then subjected to progressive denoising processing for T time steps to obtain the predicted latent motion features.

[0035] Preferably, in step S44, the expression for the construction loss function between the predicted latent motion features and the original latent motion features is:

[0036] ;

[0037] In the formula, L net Losses due to network reconstruction; L 1 smooth For smoothing L 1. Loss operator; Z 0 represents the original potential motion characteristic; These are the potential motion characteristics for prediction.

[0038] Beneficial effects

[0039] The advantages of this invention are as follows: By dividing the body into different regions to encode local motion styles, the model can capture fine-grained regional differences. Furthermore, this invention employs a partial perceptual diffusion model, allowing rhythmic and emotional cues to precisely guide each body region, thereby ensuring that the generated motion is consistent with changes in language rhythm and emotional state. In this way, this invention can synthesize 3D human motion with a reference style, significantly improving the naturalness and expressiveness of the generated 3D motion sequences. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the steps of a speech-driven 3D human motion method based on a local style encoder in an embodiment of the present invention.

[0041] Figure 2 This is a flowchart illustrating the training process of a voice-driven stylized 3D human motion model in an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the framework structure of the voice-driven stylized 3D human motion overall model in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to embodiments, but this does not constitute any limitation on the present invention. Any limited modifications made by any person within the scope of the claims of the present invention are still within the scope of the claims of the present invention.

[0044] Existing speech-driven 3D human motion methods suffer from problems such as simplification of style diversity and neglect of differences in motion styles across different regions of the human body. These issues limit the realism and diversity of generated motion. Furthermore, existing methods fail to fully consider the crucial characteristic that motion style should dynamically adapt to changes in the rhythm and emotion of speech. Based on this, this invention proposes a speech-driven 3D human motion method based on a local style encoder. This method extracts local perceptual style features through a local style encoder and extracts semantic, prosodic, and emotional features from speech. These multiple conditions are injected into a partial perceptual diffusion model, and a 3D human motion network with a reference style is synthesized using a reference style motion sequence and speech information, thereby improving the naturalness and expressiveness of the 3D human motion sequence.

[0045] In this embodiment, the speech-driven 3D human motion method based on a local style encoder mainly includes two steps: a speech-driven stylized 3D human motion overall model construction step and a 3D human motion network output step with a reference style. The speech-driven stylized 3D human motion overall model constructed in the speech-driven stylized 3D human motion overall model construction step serves as the main body for implementing the speech-driven 3D human motion method of this invention, and its specific construction method will be described below.

[0046] In this embodiment, the voice-driven stylized 3D human motion model mainly includes a local style encoder, a speech encoding module, a partial perceptual latent denoiser, a motion encoder, and a motion decoder. Based on its structure, refer to... Figure 1-2 The specific construction method of voice-driven stylized 3D human motion model includes the following steps:

[0047] S1. Obtain three-dimensional human body mesh sequence samples and original speech, i.e., audio files, and divide these samples into training set and test set.

[0048] Specifically, the obtained 3D human body mesh sequence samples include the database's speech modality and the 3D human motion sequence visual modality. Both the speech modality and the 3D human motion sequence visual modality are temporally ordered. These 3D human body mesh sequence samples can be used in the training of a speech-driven stylized 3D human motion overall model, and also in the synthesis process of a speech-driven stylized 3D human motion overall model onto a reference-styled 3D human motion network.

[0049] Taking a single sentence as an example, each set of sample data consists of the following parts:

[0050] (1) The audio is a WAV file, which is read to obtain the speech waveform;

[0051] (2) The 3D face animation sequence is an npy file. After reading it, the time sequence of the 3D spatial coordinates of 10,475 vertices in the corresponding face mesh is obtained. X= { x i | i= Let {1, 2, ..., n} be the number of time frames in the 3D face animation sequence. x i ( i ∈{1,2,...,n})∈ R 10475×3 This represents the 3D spatial coordinates of 10,475 vertices in a single-frame human body mesh, with K = 54 joints, including those of the neck, jaw, eyes, and fingers. X After rendering, a realistic human motion sequence can be obtained.

[0052] In this embodiment, the 3D human motion data uses the BEAT2 dataset. The BEAT2 dataset contains 60 hours of audio-3D human motion sequence mesh pairs from 30 subjects. For each subject, a 3D human motion sequence and corresponding speech with a sampling rate of 22000Hz were captured at 30fps. Each subject has 8 different emotional styles. The 3D human motion mesh consists of 10,475 vertices and K = 54 joints.

[0053] The dataset is divided into training and testing sets according to the speaker and sentiment labels, specifically in a ratio of 85%:15%.

[0054] S2. Based on the multi-dimensional information contained in the speech, the original speech is preprocessed and then input into the speech coding module to extract semantic features, prosodic features and emotional features.

[0055] Specifically, in step S2, semantic features in speech are extracted using the self-supervised pre-trained speech model Wav2Vec2; prosodic features in speech are extracted using the self-supervised pre-trained speech model BEATs; and emotional features in speech are extracted using the self-supervised pre-trained speech model Emotion2Vec.

[0056] S3. Obtain reference style motion sequences from the 3D human body mesh sequence samples in the training set, and input the reference style motion sequences into the local style encoder for pre-training to obtain local perceptual style features.

[0057] The reference style motion sequence is a motion sequence fragment. During the training phase, the first p frames are extracted from the training samples as reference style fragments. During the inference phase, the duration of the input reference style motion sequence is m seconds.

[0058] Furthermore, in this embodiment, during the training phase, the first 128 frames are extracted from the training samples as reference style segments; during the inference phase, the duration of the input reference style 3D human motion sequence is 4 seconds.

[0059] The local style encoder consists of three locally style encoders with the same structure, divided into an upper body style encoder, a hand style encoder, and a lower body style encoder according to body parts. The local style encoder for each body part has a Transformer encoder structure.

[0060] In this embodiment, the process for obtaining the local style encoder is as follows:

[0061] S31. Divide the reference style motion sequence into multiple local motion sequences according to body parts. Specifically, mask the reference style motion sequence according to the index of different body parts to obtain local motion sequences for the upper body, hands, and lower body.

[0062] S32. Construct positive and negative sample pairs for local motion sequences.

[0063] Based on the assumption that the style of the same person remains consistent across short, adjacent motion segments, this study enhances the discriminative power and robustness of style representation by bringing samples with the same style closer together and pushing samples with different styles further apart. Each training batch contains... N s There are 3 samples, and the length of each motion parameter sample is 2. T s Divide each sample in half to form N s 1 sample pair; for each positive sample pair, the rest in the same batch All samples are treated as negative samples.

[0064] S33. Construct a loss function between positive and negative sample pairs, and continuously train and optimize the encoder using gradient descent to obtain a local style encoder that can represent local perceptual style features.

[0065] Specifically, an implicit modeling method based on contrastive learning is employed to construct the NT-Xent loss function between positive and negative sample pairs:

[0066] .

[0067] Among them, l |k≠i| Let be the indicator function, and τ be the temperature parameter. For and The total loss for all pairs is calculated. After pre-training, the local style encoder serves as the style feature extractor in the latent diffusion model.

[0068] S4. Inject local perceptual style features, semantic features, prosodic features and emotional features into the partial perceptual diffusion model for training, so as to obtain a speech-driven three-dimensional human motion overall model.

[0069] The partially perceptual diffusion model consists of a partially perceptual latent denoiser, a motion encoder, and a motion decoder. Both the motion encoder and motion decoder are pre-trained RVQ-VAE models, and their training process is as follows:

[0070] T-frame 3D human body mesh sequence samples X= { x i | i=The inputs 1, 2, ..., n} are encoded in the encoder of the RVQ-VAE model to obtain the feature representation. Z= { z i | i= 1,2,...,n}. Then, through multi-level quantization, Z The corresponding quantization representations are obtained by mapping them sequentially to the codebook spaces at each level. Z v . Z v The RVQ-VAE model decoder decodes and reconstructs the predicted three-dimensional human motion sequence, outputting the result. In the three-dimensional human body mesh sequence sample X With the predicted three-dimensional human motion sequence A reconstruction loss is constructed between the original input and the reconstruction result, and the difference between the original input and the reconstruction result is reduced by minimizing the reconstruction loss. The gradient descent method is used to train the model end-to-end, optimize the parameters of the multi-level codebook, encoder and motion decoder, and finally obtain the codebook, motion encoder and motion decoder that are trained.

[0071] Regarding the aforementioned multi-level quantization, specifically: generating the initial residual r 1 = { z i | i= For the expression {1, 2, ..., n}, find the first residual r using the nearest neighbor dictionary. 1 Quantization is performed to obtain the quantized output Z. 1 = Q (r 1 ).in, Q (r 1 ) represents the quantization function. Calculate the next residual r. 2 = r 1 - Z 1 Then, quantization is performed through a second layer of residual quantization, and this process is repeated until all residuals are processed.

[0072] In this embodiment, residual quantization uses 6 layers, with each layer having a 512-dimensional codebook.

[0073] Regarding the minimization of the reconstruction loss mentioned above, the specific details are as follows:

[0074] .

[0075] In this embodiment, the specific process of step S4 is as follows:

[0076] S41. Input the three-dimensional human body mesh sequence sample into the motion encoder to obtain the original potential motion features.

[0077] S42. Sample random noise and perform T-step noise addition processing on the original potential motion features to obtain noisy potential motion features during the diffusion process.

[0078] Specifically, in the pre-trained motion potential space, the diffusion process gradually moves towards the original potential motion features. Z Add Gaussian noise to 0, transforming it into a standard Gaussian distribution through a Markov forward process. .

[0079] The mathematical expression for this process is:

[0080] .

[0081] in, Z t α represents the latent noise vector at time t∈{1,2,...,T}. t Control noise levels.

[0082] S43. Semantic features, prosodic features, emotional features, local perceptual style features, and noisy latent motion features are input together into a partial perceptual latent denoiser. After T steps of denoising, the predicted latent motion features are obtained. Where T is a natural number greater than or equal to 1.

[0083] In this embodiment, the partial perceptual latent denoiser consists of eight Transformer encoders with a latent dimension of 512. Each Transformer encoder includes a self-attention module, a prosodic cross-attention module, and an emotion cross-attention module.

[0084] Specifically, semantic features, local perceptual style features, and noisy latent motion features are concatenated to construct joint latent features for each part. The expression for the constructed joint latent features for each part is as follows:

[0085] .

[0086] in, u , l , h These represent the upper body, hands, and lower body, respectively. Z i Indicates the potential motion characteristics with noise; S i Indicates local perceptual style features, Indicates semantic features.

[0087] The joint latent features, prosodic features, and sentiment features are input into the Transformer encoder. The encoder then sequentially passes through a self-attention module, a prosodic cross-attention module, and a sentiment cross-attention module to fuse the prosodic and sentiment features. The fused features are then denoised to obtain a clean latent motion sample at step t. Noise is removed progressively over T time steps to obtain the predicted latent motion features. The mathematical expression for this process is:

[0088] ;

[0089] .

[0090] in, This represents the potential motion sample that passes through the prosody cross-attention module at step t; This represents a potential motion sample that passes through the self-attention module at step t; This represents the potential motion sample that passes through the emotion cross-attention module at step t; DP ( ) denotes the Dropout operator; LS() denotes layer normalization; RhythmCA ( )and EmotionCA ( ) represent the prosody cross-attention module and the emotion cross-attention module, respectively.

[0091] Furthermore, the mathematical expressions for the prosodic cross-attention module and the affective cross-attention module are as follows:

[0092] ,

[0093] .

[0094] in, ; ; ; ; ; . W Q , W K and W V All of these are learnable parameters.

[0095] S44. Construct a loss function between the predicted potential motion features and the original potential motion features, and continuously train and optimize the model through gradient descent to obtain a speech-driven three-dimensional human motion overall model.

[0096] The loss function for the speech-driven stylized 3D human motion model is:

[0097] .

[0098] in, Z 0 represents the original latent motion features; S represents the local perceptual style features. Indicates rhythmic features; Indicates emotional characteristics; Z t This represents the clean potential motion sample obtained from the prediction at step t; .

[0099] S5. Measure the performance of the speech-driven stylized 3D human motion model based on the test set, and obtain the best-performing speech-driven stylized 3D human motion model.

[0100] Optionally, the quantitative metrics used are gesture distribution distance (FGD), beat alignment score (BC), diversity score, and style accuracy (SRA). FGD fits the features obtained from the real and generated sequences using the same feature extractor to Gaussian distributions and calculates their Fréchet distance; BC is based on the time difference between each movement beat and the nearest musical beat, normalized and averaged across the sequence; Diversity divides the action into equal-length, non-overlapping segments, calculates the differences between segments, and averages them; SRA uses a pre-trained style classifier to discriminate the generated sequence, measuring consistency by the classification accuracy or average confidence score of the target style. Finally, the parameters and structure of the entire model with the best performance under each metric are saved.

[0101] The above describes the specific construction method of the speech-driven stylized 3D human motion overall model. In practice, a reference style motion sequence and speech samples from the actual environment are input into the best-performing speech-driven stylized 3D human motion overall model to output a 3D human motion network with the reference style. Figure 3 As shown, the specific process is as follows:

[0102] Locally perceptual style features S are extracted from the reference style motion sequence, while semantic features are extracted from the speech samples. Rhythmic features Emotional characteristics And the latent motion features Z. Given these guiding conditions, the process is initialized with random noise, and then denoised iteratively from time step T to 1. After T steps of denoising, the predicted latent motion features are processed by a motion decoder. Decoding is performed to obtain a three-dimensional human motion network with a reference style. .

[0103] In the above process, an incremental classifier-free guidance strategy is used for denoising. Specifically, the mathematical expression of the incremental classifier-free guidance strategy is:

[0104] .

[0105] in, w c and w s These represent the guiding weights of semantic features and style features, respectively. w re This represents the shared weights of rhythmic and emotional features. To simplify notation, a unified representation is defined for rhythmic and emotional features. During training, conditional inputs (i.e., set to a probability of p = 0.1) are randomly discarded. φ They learn both conditional and unconditional distributions together. During inference, the stylization of the generated motion can be flexibly controlled by adjusting the guidance weights.

[0106] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention, and these will not affect the effectiveness of the implementation of the present invention or the practicality of the patent.

Claims

1. A method for speech-driven 3D human motion based on local style encoder, characterized in that, The voice-driven stylized three-dimensional human motion overall model construction step and the three-dimensional human motion network output step with a reference style are included. The voice-driven stylized three-dimensional human motion overall model construction step includes: S1, obtaining three-dimensional human mesh sequence samples and original speech, and dividing them into a training set and a test set; S2, preprocessing the original speech, and extracting semantic features, prosodic features and emotional features from the preprocessed original speech; S3, obtaining a reference style motion sequence from the three-dimensional human mesh sequence samples in the training set, inputting the reference style motion sequence into a local style encoder for pre-training to obtain local perceptual style features; S4, injecting the local perceptual style features, semantic features, prosodic features and emotional features into a partial perceptual diffusion model for training to obtain a voice-driven three-dimensional human motion overall model; In step S4, the partial perceptual diffusion model is composed of a partial perceptual latent denoiser, a motion encoder and a motion decoder; The specific method for training the local perceptual style features, semantic features, prosodic features and emotional features through the partial perceptual diffusion model to obtain the voice-driven three-dimensional human motion overall model is: S41, inputting the three-dimensional human mesh sequence samples in the training set into the motion encoder to obtain original latent motion features; S42, sampling random noise, performing T-step noise processing on the original latent motion features to obtain noisy latent motion features in the diffusion process; S43, inputting the semantic features, prosodic features, emotional features, local perceptual style features and noisy latent motion features into the partial perceptual latent denoiser, and obtaining predicted latent motion features after T-step denoising; S44, constructing a loss function between the predicted latent motion features and the original latent motion features, continuously training and optimizing the diffusion model by gradient descent method to obtain the voice-driven three-dimensional human motion overall model; S5, measuring the performance of the voice-driven stylized three-dimensional human motion overall model according to the test set, and obtaining the voice-driven stylized three-dimensional human motion overall model with the best performance; The three-dimensional human motion network output step with a reference style is used to input the reference style motion sequence and the voice sample in the actual environment into the voice-driven stylized three-dimensional human motion overall model with the best performance to output a three-dimensional human motion network with a reference style.

2. The speech-driven 3D human motion method based on local style encoder according to claim 1, wherein, In step S3, the local style encoder includes a plurality of local style encoders with the same structure.

3. The speech-driven 3D human motion method based on local style encoder according to claim 2, characterized in that, The local style encoder is constructed in the following manner: S31, dividing the reference style motion sequence of the three-dimensional human mesh sequence samples in the training set into a plurality of local motion sequences according to body parts; S32, constructing a positive-negative sample pair of the local motion sequence; S33, constructing a loss function between the positive-negative sample pair, and continuously training and optimizing the encoder by gradient descent method to obtain a local style encoder capable of representing local perceptual style features.

4. The speech-driven 3D human motion method based on local style encoder of claim 3, wherein, The loss function constructed between the positive-negative sample pair is: ； In the formula, is a loss function constructed between positive and negative sample pairs; l |k≠i| is an indicator function; τ is a temperature parameter; N s is the number of three-dimensional human body mesh sequence samples; s i is a sample i; s j is a sample j.

5. The speech-driven 3D human motion method based on local style encoder of claim 1, wherein, In step S2, the original speech is preprocessed according to the multi-dimension information contained in the speech, and then input into a speech coding module to extract the semantic feature, the prosody feature and the emotion feature.

6. The speech-driven 3D human motion method based on local style encoder of claim 1, wherein, In step S42, a Gaussian noise is added to the original latent motion feature, and the expression of the Gaussian noise is: ； where q Z t Z t-1 is the conditional transition distribution of the forward diffusion process; Z t denotes the noise latent vector at time t e {1, 2,..., T}; Z t-1 denotes the noise latent vector at time t - 1 e {1, 2,..., T - 1}; a t is the diffusion coefficient for each step; denotes the standard Gaussian distribution function.​​ 7. The speech-driven 3D human motion method based on local style encoder of claim 1, wherein, In step S43, specifically: The semantic feature, the local perceptual style feature and the noisy latent motion feature are spliced to construct a joint latent feature; the joint latent feature, the prosody feature and the emotion feature are input into a partial perceptual latent denoiser composed of multiple Transformer encoders, so that the joint latent feature sequentially passes through a self-attention module, a prosody cross-attention module and an emotion cross-attention module of the Transformer encoder, is fused with the prosody feature and the emotion feature, and the fused feature is subjected to step-by-step denoising processing at T time steps to obtain a predicted latent motion feature.

8. The speech-driven 3D human motion method based on local style encoder of claim 1, wherein, In step S44, the expression of a loss function constructed between the predicted latent motion feature and the original latent motion feature is: ； wherein L net is the reconstruction loss for the network; L 1 smooth is the smoothing L 1loss operator; Z 0is the original latent motion feature; is the predicted latent motion feature.

Images