A method for generating stylized human motion based on spatio-temporal attention

Abstract

This invention discloses a stylized human motion generation method based on spatiotemporal attention, comprising the following steps: using a publicly available human motion style dataset, preprocessing it, and then dividing it into training and testing sets; constructing a stylized motion generation network model based on spatiotemporal attention; training the stylized motion generation network model based on spatiotemporal attention using the training set and updating its parameters; using the trained human motion generation model, taking the original motion content sequence and target style information as input, and outputting a stylized human motion sequence that retains the original action semantics and possesses the target style characteristics. This invention overcomes the problems of style mean-averaging and loss of fine-grained dynamics caused by temporal averaging aggregation in existing technologies, as well as the lack of adaptive attention to key spatiotemporal positions. It achieves fine modeling of multi-scale, non-uniformly distributed style cues, thereby generating human motion sequences with distinct styles, rich details, and well-preserved semantic structure.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of computer vision and artificial intelligence, and specifically relates to a stylized human motion generation method based on spatiotemporal attention. Background Technology

[0002] With the development of deep learning technology, human motion generation has been widely used in fields such as virtual reality, digital human driving, animation production, and human-computer interaction. In recent years, neural network-based motion modeling methods have been able to learn the spatiotemporal representation of human motion from large-scale data, enabling tasks such as motion reconstruction, prediction, and generation.

[0003] Building on this, stylized motion generation has gradually become a research hotspot. Its goal is to controllably adjust the expressive style of actions (such as emotions, age, behavioral characteristics, etc.) while preserving the original semantic structure of the actions, thereby generating human motion sequences with specific stylistic expressions.

[0004] Existing technologies typically employ an encoder-decoder architecture. First, convolutional networks or graph neural networks extract spatiotemporal features from the input motion sequence. Then, the extracted features are statistically aggregated over time (e.g., average pooling) to obtain a global style vector, which is then injected into the decoding stage for style modulation. However, this approach suffers from the following key technical drawbacks: First, due to the use of simple temporal averaging, style contributions from different time steps are treated equally, leading to "meaning" of style information. This makes it difficult to retain style attributes with fine-grained dynamic features such as rhythmic variations and amplitude differences, easily resulting in distorted or ambiguous style representation. Second, the style feature extraction process lacks an adaptive attention mechanism for key spatiotemporal locations, failing to dynamically identify and strengthen style-sensitive joints or time frames based on the action content. This limits style representation capabilities and makes it difficult to capture complex and diverse style patterns. Furthermore, most methods rely solely on the deepest abstract features of the encoder to construct the style vector, neglecting the synergistic effect of multi-scale spatiotemporal features, further weakening the expressiveness and generalization of style modeling. Meanwhile, some solutions require retraining the model or iterative optimization for each new style, which is inefficient and fails to meet the needs of flexible and efficient style control in practical applications.

[0005] In summary, existing technologies have significant shortcomings in the extraction and utilization of style features. The core problem is that style vectors cannot effectively aggregate multi-scale spatiotemporal style information from global and local, key and non-key frames, resulting in weak style expressiveness, loss of detail, and poor controllability of generated actions. Summary of the Invention

[0006] The purpose of this invention is to provide a stylized human motion generation method based on spatiotemporal attention, which aims to overcome the problems in the prior art where the style information is averaged and fine-grained dynamics are lost due to temporal averaging aggregation, and the style features cannot adaptively focus on key spatiotemporal positions, resulting in insufficient fine-grained expressive ability. This invention achieves fine modeling of multi-scale, non-uniformly distributed style cues, thereby generating human motion sequences with distinctive style, rich details, and well-preserved semantic structure.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A stylized human motion generation method based on spatiotemporal attention includes the following steps: Step 1: Using a publicly available human motion style dataset, preprocess it and then divide it into training and testing sets; Step 2: Construct a stylized motion generation network model based on spatiotemporal attention; For a given source motion sequence ,in Represents the number of frames. Represents the number of joints in the human body. Representing the feature dimensions of each joint, the network model supports two input styles: (1) Reference mode: Input a reference motion with the target style ; (2) Sampling mode: Input a target style domain label ,in The total number of style domains; and a potential random code that follows a normal distribution. ; The network model takes the original motion content and target style information as input and outputs a stylized motion sequence that preserves semantic content and has target style characteristics. It includes four sub-modules: MotionStylizer Generation (G), Style Feature Extractor (E), Mapping Network (F), and Style Discriminator (D). The motion stylization generator is based on the Transformer's attention mechanism. It extracts content representations from the source motion sequence using multiple spatiotemporal attention layers and reconstructs a stylized motion sequence by combining the target style features. The motion stylization generator includes a Content Feature Extractor and a Style-Guided Synthesis Decoder (SGSD). The Content Feature Extractor adopts a multi-layer residual Transformer based on pre-norm logic and introduces an InstanceNorm layer at the end of the extractor to constrain the feature distribution. The Style-Guided Synthesis Decoder (SGSD) is responsible for combining the extracted content features with the target style features. Perform deep fusion and reconstruct the final motion sequence; The style feature extractor extracts style sequence features from the reference pattern and receives a reference motion sequence with the target style. As input, its implicit style attributes are captured through multiple spatiotemporal attention layers, and the final output is a feature vector that can express the style of the reference motion, namely the style token. The style mapping network receives two pieces of information upon input: one is a low-dimensional latent vector sampled from a standard normal distribution. Secondly, it represents discrete labels indicating the target style category. Random latent codes that follow a normal distribution With target style domain tags Perform nonlinear fusion mapping to generate the corresponding style feature vector; The style discriminator D includes a shared feature extraction layer and multiple independent output branches corresponding to different style domains. It adopts a multi-task adversarial learning architecture to guide the optimization direction of the motion stylization generator. During training, it is responsible for judging whether the generated motion sequence conforms to the style distribution of the target style domain. Step 3: Train the stylized motion generation network model based on spatiotemporal attention using the training set and update the parameters; Step 3.1: Reconstruct the original joint space data into a spatiotemporal joint representation; Step 3.2: Obtain the target style vector: In the reference mode, the reference motion sequence containing the target style is input into the style feature extractor E. The implicit style attributes are extracted through the temporal attention layer to obtain the style feature vector. In the sampling mode, random latent vectors are sampled from the standard normal distribution. Then, the style category labels of the reference motion sequence. The encoding is a one-hot embedding, and both are input into the style mapping network F to obtain the target style vector. Step 3.3: Input the original content motion sequence into the motion stylization generator G, model the spatiotemporal relationship between joints through the multi-layer spatiotemporal attention layer of the content feature extractor, and then extract the pure content semantic features without style through instance normalization (InstanceNorm); Step 3.4: The two style vectors obtained in Step 3.2 are used as style guidance signals, and together with the content semantic features obtained in Step 3.3, they are input into the style guidance synthesis network of the motion stylization generator G, and the generated stylized motion sequence is output. Step 3.5: Combine the stylized motion sequence generated in Step 3.4 with the target style tag. The generated samples are input into the style discriminator D to calculate the discrimination score in the target style domain, while also referencing the motion sequence and the target style label. Input the data and calculate the discrimination score for the real sample. Step 3.6: Calculate the total loss based on the input motion sequence, the generated motion sequence, and the discrimination score obtained in Step 3.5. Update the network parameters of each submodule; Step 4: Using the trained human motion generation model, take the original motion content sequence and target style information as input, and output a stylized human motion sequence that retains the original action semantics and has the target style characteristics.

[0008] Furthermore, the preprocessing in step 1 involves: redirecting all motion data in the human motion style dataset to a unified CMU skeletal topology, which contains N=21 joints, and downsampling the original sequence to 60 fps to reduce time redundancy; subsequently, data augmentation of the motion sequence is performed by mirror flipping to expand the training samples.

[0009] Furthermore, the content feature extractor described in step 2, during the feature transformation process, provides... Each spatiotemporal feature vector is encoded with temporal position and spatial position respectively to explicitly represent the temporal sequence and spatial structure of motion. A spatiotemporal separation attention mechanism is used to transform the features layer by layer: first, multi-head self-attention is performed in the spatial dimension to calculate the spatial relationship between P functional units within each frame, and then multi-head self-attention is performed in the temporal dimension to capture the temporal dependency relationship across frames, thereby obtaining the phase change and rhythm features of the action.

[0010] Furthermore, the style-guided synthesis network described in step 2 achieves precise style injection during the synthesis process through multiple synthesis layers containing Adaptive Instance Normalization (AdaIN); in each Transformer block, the style features are extracted from the reference style feature extractor. or style mapping network Target style characteristics As a conditional signal, the feature distribution of the spatiotemporal feature vector is dynamically adjusted.

[0011] Furthermore, the style feature extractor described in step 2 introduces a global aggregation-based approach: a learnable style token (LST) is concatenated at the beginning of the input spatiotemporal feature sequence to form an enhanced spatiotemporal feature sequence, which then enters a multi-layer spatiotemporal attention layer. The LST, as a query vector, actively extracts and aggregates motion features at different time points and in different spaces of the entire sequence using a self-attention mechanism. After multiple attention updates, the LST gradually evolves from the initial learnable vector into a high-dimensional representation that aggregates global style information.

[0012] Furthermore, the style mapping network described in step 2 employs a dual-branch parallel feature extraction method, with latent vectors... The one-hot embedding and the feature vector are input into two separate fully connected layers. Each branch consists of a linear mapping layer and a LeakyReLU activation function. The features from the two branches are then concatenated along the channel dimension to form a joint feature representation, which is then input into a shared mapping module. This shared mapping module consists of a multi-layer fully connected network that performs depth mapping on the joint features through layer-by-layer linear transformations and activation functions, with the hidden layer dimensions gradually expanding to enhance the model's ability to represent complex style distributions. Finally, the network maps the features to a preset style dimension through a linear output layer. ; The style mapping network finally uses the Tanh activation function to constrain the range of style feature values; the style mapping network ultimately outputs a style vector. As a target style feature, it can be directly fed into the style-guided synthesis network.

[0013] Furthermore, the style discriminator described in step 2 adopts a multi-task adversarial learning architecture. The style discriminator includes a shared feature extraction layer and multiple independent output branches corresponding to different style domains. For the input sequence, a unified spatiotemporal feature representation is first extracted. Then, this global discriminative feature is fed into the multi-task classification network, outputting discriminative score vectors for all style domains, given a target style label. The model selects only the discrimination score of the corresponding style domain as the final output, thereby achieving conditional discrimination: During adversarial training, the optimization objective of the style discriminator is divided into the following two parts: (1) Discrimination of real samples: The style discriminator D receives real motion data from the dataset. The goal is to maximize the probability that the output is "true" under the corresponding style tag, that is... In order to learn the spatiotemporal characteristics and distribution patterns of the target style domain; (2) Detection of fake samples: The style discriminator D receives the motion generated by the motion stylization generator G. The goal is to maximize the probability that its output is "false" under the same style conditions, that is... By identifying the distributional differences between generated motion and real data, potential style deficiencies or unnatural phenomena in generated motion can be discovered.

[0014] Furthermore, the spatiotemporal joint representation process described in step 3.1 includes: Human motion sequences are usually represented as Each joint node's features include three-dimensional local location features. Four-dimensional unit quaternion rotation characteristics And the global motion trajectory, which includes the horizontal linear velocity. angular velocity of rotation about the vertical axis and the vertical height of the root node relative to the ground Together, they constitute the d-dimensional original feature vector; Based on human kinematics, N joint nodes are divided into P disjoint semantic sets. Through feature mapping and aggregation, the variable-length local joint data is transformed into a standardized input format that the network can process. For each time t, the features of the joints contained in the p-th functional unit are concatenated in sequence to obtain the original state vector of that functional unit. Since the number of joints in each functional unit varies, a set of learnable linear projection layers is introduced to project the joints of different dimensions. Mapped to a unified embedding dimension And apply LayerNorm to normalize it along the feature dimension: After projection mapping, the original motion data can be represented as a sequence of lengths... spatiotemporal joint sequence, For time frames: .

[0015] Furthermore, the total loss described in step 3.6 A training strategy employing joint optimization of multiple losses is adopted, and the overall training process is based on an adversarial generative framework for the motion stylization generator. Style discriminator Style Feature Extractor and style mapping network Perform end-to-end training; A conditional discrimination strategy is adopted, given a style tag Under the condition that the style discriminator Determining the authenticity of the input motion sequence will help mitigate the adversarial loss. Defined as: in, For real motion samples from the dataset, These are the style domain labels corresponding to the real samples. For the target style domain tag, Corresponding style discriminator Corresponding style domain The output branch represents the probability that the input is judged as a real sample. Represents motion stylization generator According to the source movement and style characteristics Stylized movements through synthesis; To enhance training stability, the style discriminator... Additional R1 regularization loss : in Style discriminator The gradient with respect to the real sample input; To ensure that the semantic structure of the original motion is not destroyed during the stylization process, a content reconstruction loss is employed. With cycle consistency loss Constrain the generated results; When the motion stylization generator Encoded in the original style When used as style input, its output should be able to reconstruct the original motion, and the corresponding reconstruction loss is defined as: in , representing the style features extracted from the original motion by the motion stylization generator; Cyclic consistency constraints require that an action can still obtain the original motion after undergoing a mapping from "source style → target style → source style": To ensure that the generated motion accurately reflects the target style characteristics, a style reconstruction loss is employed. The requirement is that the style feature representation extracted after the motion passes through the style feature extractor must be consistent with the input style features. in , L1 norm is used to measure the differences between style features; Meanwhile, to mitigate the problem of generative models degenerating into a single output in the style space, an improved diversity-sensitive loss is used. This encourages a rich variety of outputs by comparing the differences between the generated results of randomly sampled style vectors. in and Style feature vectors generated for different random samples This represents a cutoff function used to limit the difference to a threshold. This is to prevent numerical instability during training; To improve the continuity and plausibility of generated motion in the temporal dimension, a regularization constraint based on kinematic priors is introduced; velocity regularization is used to limit joint position changes between adjacent frames, and is defined as: Acceleration regularization further constrains the second-order rate of change of motion, preventing instantaneous joint abrupt changes and jitter phenomena: Among the various losses The subscript is used to limit the variables involved in the calculation of this loss term. This means that the corresponding loss values ​​of all samples in the current batch are arithmetically averaged, and the sample mean is used as an unbiased estimate of the overall distribution characteristics. Finally, the following weighted objective function is used to jointly optimize the model, with the total loss being... Defined as: in These are the hyperparameter weights corresponding to the loss term.

[0016] Compared with the prior art, the present invention has the following beneficial effects: This invention proposes a stylized human motion generation method based on spatiotemporal attention. Addressing the differences in spatial structure and temporal dynamics of human motion, it introduces a Transformer architecture to overcome the limited receptive field of traditional convolutional models. A unified generation framework is constructed, consisting of a motion stylization generator, a style feature extractor, a style mapping network, and a style discriminator. This framework supports both reference mode and sampling mode for stylization generation, reducing dependence on specific reference sequences. For feature extraction, functional unit partitioning and a spatiotemporal attention mechanism are used to obtain spatial dependencies and cross-frame temporal dynamic relationships. In the generation stage, a "de-stylize, then add style" strategy, combined with diversity-sensitive loss and kinematic constraints, is used to jointly constrain stylized motion generation, ensuring that the generated motion expresses the target style while maintaining the semantic consistency of the original action and the rationality of the physical motion.

[0017] In terms of style feature extraction, this invention differs from existing methods that use a fixed statistical approach of temporal averaging and aggregation of features at the final layer of the encoder. Instead, it introduces a learnable style token at the beginning of the input spatiotemporal feature sequence. This style token, as an independent parameter, continuously interacts with features at all time steps and spatial joint positions within a multi-layer spatiotemporal attention mechanism, and adaptively aggregates key information most relevant to style expression during layer-by-layer propagation. Through this mechanism, the style token can dynamically focus on style-sensitive spatiotemporal regions (such as specific joints or keyframes), achieving cross-scale, globally perceptive style modeling, thereby forming a more discriminative, complete, and semantically consistent style representation. Based on this style representation, the decoding stage can inject target style information more accurately and efficiently, significantly improving the performance of generated actions in terms of style expression intensity, temporal consistency, and controllability. In qualitative experiments, compared to existing methods based on temporal averaging and aggregation, this invention exhibits more distinct and stable style differences in dimensions such as emotional intensity, movement amplitude changes, and rhythmic features, especially maintaining good style coherence in long sequence generation, fully validating the effectiveness and superiority of the proposed method in stylized human motion generation.

[0018] On the Fibonacci Depth of Motion (FMD) metric, which reflects the realism of motion, the method of this invention achieved a state-of-the-art result of 7.13, significantly outperforming the method of Aberman et al. based on a one-dimensional convolutional structure (with an FMD of 16.82) and also outperforming the method of Park et al. which introduced spatial structure modeling (with an FMD of 7.88). This result demonstrates that by introducing a global self-attention mechanism to model the long-range dependencies of the entire motion sequence, the limitations of traditional one-dimensional convolution, confined to local receptive fields, are effectively overcome. This significantly enhances the model's ability to model long-term temporal relationships and overall motion distribution characteristics, thereby improving the consistency between generated and real motion at the distribution level and enhancing the stability of the macroscopic temporal structure.

[0019] In terms of Content Retention Ability (CRA), the method of this invention achieves 65.21%, significantly higher than one-dimensional convolutional models that rely solely on local feature extraction (whose CRA is below 40%). This indicates that the spatiotemporal attention mechanism employed can more effectively decouple motion content from style features, thereby significantly enhancing content retention. In contrast, although MoST is close to this method in terms of CRA, its Style Recognition Ability (SRA) is only 33.42%, reflecting its shortcomings in content-style separation. The method of this invention achieves 44.29% SRA, superior to Motion Puzzle's 43.94%, further demonstrating that this invention can not only stably retain the original motion content but also more accurately model the target style features, achieving a more balanced and effective content-style decoupling effect. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of a stylized generative network architecture based on spatiotemporal attention. Figure 2 is a schematic diagram of the motion stylization generator architecture, where Figure 2(a) is the architecture diagram of the content feature extractor and Figure 2(b) is a schematic diagram of the style-guided synthesis network architecture. Figure 3 A schematic diagram of a style mapping network; Figure 4 The result is a motion stylization based on a reference sequence; Figure 5 This is the motion stylization result based on latent variable sampling. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] The stylized human motion generation method based on spatiotemporal attention described in this embodiment includes the following steps: Step 1: Using a publicly available human motion style dataset, preprocess it and then divide it into training and test sets. The dataset used is the human motion style dataset compiled by Xia et al. This dataset contains various style-labeled human motion sequences, covering a wide range of action content and style categories. In the experiments, all motion data were redirected to a unified CMU skeletal topology, containing N=21 joints. To reduce temporal redundancy, the original motion sequences were downsampled to 60 fps. Furthermore, data augmentation was performed by mirroring the motion sequences to increase the number of training samples. The dataset was then divided into training and testing sets. The training set, containing typical motion sequences for each style category, was used to learn the model parameters. The testing set, while maintaining the same style category as the training set, did not overlap with the specific motion sequences and was used to evaluate the model's style transfer and generalization capabilities under unseen motion conditions. During training, motion sequences were divided into short segments of length T=64 frames, with a T / 2 frame overlap between adjacent segments. To improve training stability, all joint data were standardized before inputting into the network by subtracting the mean pose of the entire dataset and dividing by the standard deviation. During testing, the model directly applied the complete motion sequences.

[0023] Step 2: Construct a stylized motion generation network model based on spatiotemporal attention. The stylized motion generation network model based on spatiotemporal attention proposed in this invention aims to handle two stylization modes: reference sequence-based and category label-based. The model takes the original motion content and target style information as input, and outputs a stylized motion sequence that preserves semantic content and possesses the target style characteristics. The network structure is as follows: Figure 1 As shown.

[0024] For a given source motion sequence ,in Represents the number of frames. Represents the number of joints in the human body. This represents the feature dimension of each joint. The network model proposed in this invention supports two input styles: (1) Reference mode: Input a reference motion with the target style .

[0025] (2) Sampling mode: Input a target style domain label (in (the total number of style domains) and a potential random code that follows a normal distribution. .

[0026] The stylized motion generation network model based on spatiotemporal attention consists of four sub-modules that work together to achieve feature extraction and style fusion: (a) Motion Stylizer Generation (G): As the core generation unit, this module is based on the Transformer's attention mechanism. It extracts content representations from the source motion sequence using multiple spatiotemporal attention layers and reconstructs stylized motion sequences by combining them with target style features s. It includes a Content Feature Extractor and a Style-Guided Synthesis Decoder (SGSD). Through a specific normalization and spatiotemporal separation attention mechanism, it achieves the extraction of motion features and the generation of stylized motion. Its internal design is described in detail below.

[0027] The goal of the content feature extractor is to extract stable and clean action content features from the input spatiotemporal joint sequence S. To this end, the module adopts a multi-layer residual Transformer based on pre-norm logic in its structure, and introduces an InstanceNorm layer at the end of the extractor to constrain the feature distribution.

[0028] The structure of the content feature extractor is shown in Figure 2(a). During the feature transformation process, this module is used for... Each spatiotemporal feature vector is encoded with temporal position and spatial position respectively to explicitly represent the temporal sequence and spatial structure of motion. A spatiotemporal separation attention mechanism is used to transform the features layer by layer: first, multi-head self-attention is performed in the spatial dimension to calculate the spatial relationship between P functional units within each frame, and then multi-head self-attention is performed in the temporal dimension to capture the temporal dependency relationship across frames, thereby obtaining the phase change and rhythm features of the action.

[0029] A decoupling mechanism based on InstanceNorm is introduced after the last layer of the content feature extractor. From the perspective of feature distribution, this process effectively weakens the style-correlated statistical information in the original motion, such as average pose height and overall motion amplitude, by normalizing the mean and variance. This allows the encoded features to retain more of the temporal structure and motion logic of the action itself, reducing the dependence on specific styles.

[0030] Style-Guided Synthesis Network (SGSD) is responsible for matching the extracted content features with the target style features. The module performs deep fusion and reconstructs the final motion sequence. During synthesis, it achieves precise style injection through multiple synthesis layers containing Adaptive Instance Normalization (AdaIN), as shown in Figure 2(b). In each Transformer block, the target style features... (From the reference style feature extractor) or style mapping network As a conditional signal, the feature distribution of the spatiotemporal feature vector is dynamically adjusted. Corresponding to the normalization process in the feature extraction stage, the synthesis network learns specific affine transformations to reintroduce the statistical characteristics of the target style into the normalized content features, achieving effective style reshaping. This "remove style first, then add style" approach ensures that the stylization process goes beyond simple feature layering; it reconstructs actions from the feature distribution level. Simultaneously, SGSD utilizes skip connections to preserve content features, maintaining semantic consistency of the original actions while ensuring significant style changes, ultimately generating a stylized motion sequence.

[0031] (b) Style feature extractor (E): This module is specifically designed to extract style sequence features from a reference pattern, receiving a reference motion sequence with the target style. As input, its implicit style attributes are captured through multiple spatiotemporal attention layers, and the final output is a feature vector that can express the style of the reference motion, namely styletoken.

[0032] To address the issue of variable time length in human motion sequences, the style feature extractor introduces a global aggregation-based approach. Specifically, a learnable style token (LST) is introduced at the beginning of the input spatiotemporal feature sequence. This token has the same embedding dimension as the other spatiotemporal feature vectors and exists as an independent learnable parameter during training, used to aggregate style information from the entire sequence. During feature extraction, the style token interacts with all spatiotemporal features in the sequence through an attention mechanism. In multi-layer attention computation, the style token adaptively gathers style-related information from different time frames and body parts, thereby achieving overall style feature extraction.

[0033] In terms of network architecture design, the style feature extractor still uses the spatiotemporal attention architecture employed by the content feature extractor to ensure consistency in the overall network design. Furthermore, compared to global average pooling, this attention-based style feature aggregation strategy can adaptively allocate weights for different time segments and spatial locations based on the content differences of the input sequence. This more accurately highlights frames that contribute more to style expression, which is beneficial for extracting style features from complex motions. After transformations through multiple layers of spatiotemporal attention, style information from the reference motion sequence is gradually aggregated into style tokens. Finally, the style token output from the last layer is selected as the style feature encoding for the reference motion. This style encoding will then be fed into the Style Guided Synthesis Network (SGSD) to guide the stylistic reconstruction of subsequent motion sequences.

[0034] (c) Style Mapping Network (F): This module is the core of the sampling pattern implementation and is responsible for mapping the random latent codes that follow a normal distribution. With target style domain tags Performing nonlinear fusion mapping to generate corresponding style feature vectors is a key module for achieving diversified style generation during the inference process by sampling in the latent space.

[0035] like Figure 3 As shown, upon input, the style mapping network receives two pieces of information: one is a low-dimensional latent vector sampled from a standard normal distribution. First, it introduces randomness to enhance the diversity of generated results; second, it represents discrete labels indicating the target style category. To facilitate network processing, discrete labels are first converted into one-hot encoded form, resulting in a dimension of... Style category vectors.

[0036] In terms of network architecture design, the style mapping network employs a dual-branch parallel feature extraction method. Specifically, the latent vector... The one-hot embedding and the feature vector are input into two separate fully connected layers. Each branch consists of a linear mapping layer and a LeakyReLU activation function, used to map the two different input sources to an intermediate feature space of the same dimension, facilitating the initial alignment of random and categorical information in the early stages. Subsequently, the features from the two branches are concatenated along the channel dimension to form a joint feature representation, which is then input into the shared mapping module. The shared mapping module consists of a multi-layer fully connected network that performs depth mapping on the joint features through layer-by-layer linear transformations and activation functions, with the hidden layer dimension gradually expanding to enhance the model's ability to represent complex style distributions. Finally, the network maps the features to a predefined style dimension through a linear output layer. .

[0037] To ensure the numerical stability of the output style features, the style mapping network uses the Tanh activation function to constrain the range of style feature values. The style mapping network ultimately outputs a style vector. As the target style feature, it can be directly fed into a style-guided synthesis network for adaptive modulation of content features. Through this style mapping network, the model can adjust random latent vectors under a fixed style label. Generate distinctive style features, thereby enabling diverse stylized human motion generation without the need for reference motion sequences.

[0038] (d) Style Discriminator (D): This module adopts a multi-task adversarial learning architecture to guide the optimization direction of the motion stylization generator. During the training process, it is responsible for judging whether the generated motion sequence conforms to the style distribution of the target style domain. Through continuous game with the motion stylization generator G during the training phase, it forces the generated motion to have accurate and significant target style domain features while maintaining the naturalness of the action.

[0039] The style discriminator D comprises a shared feature extraction layer and multiple independent output branches corresponding to different style domains. For the input sequence, it first extracts a unified spatiotemporal feature representation. This feature extraction process uses the same spatiotemporal separation attention architecture as the style feature extractor, and also introduces a learnable global style token (style token) in the time dimension to aggregate global style information for the entire sequence. However, the discriminator only requires one layer of feature transformation; the hidden state corresponding to this style token is considered the global discriminative feature of the input sequence. Subsequently, this global discriminative feature is fed into a multi-task classification network, outputting a discriminative score vector for all style domains, given a target style label. The model selects only the discrimination score of the corresponding style domain as the final output, thereby achieving conditional discrimination: During adversarial training, the optimization objective of the style discriminator is divided into the following two parts: (1) Discrimination of real samples: Style discriminator D receives real motion data from the dataset. The goal is to maximize the probability that the output is "true" under the corresponding style tag, that is... This allows us to learn the spatiotemporal characteristics and distribution patterns of the target style domain.

[0040] (2) Detection of fake samples: The style discriminator D receives the motion generated by the motion stylization generator G. The goal is to maximize the probability that its output is "false" under the same style conditions, that is... By identifying the distributional differences between generated motion and real data, potential style deficiencies or unnatural phenomena in generated motion can be discovered.

[0041] Through the aforementioned adversarial game process, the style discriminator continuously strengthens its ability to distinguish style and motion realism, thereby forcing the motion stylization generator G to continuously optimize its output, striving to meet the constraints of style accuracy and motion rationality, so that the final generated motion closely approximates real style motion data in both overall distribution and detailed representation.

[0042] Step 3: Train the spatiotemporal attention-based stylized motion generation network model using the training set and update the parameters. Step 3.1: Reconstruct the original joint space data into a spatiotemporal joint representation. Human motion sequences are usually represented as In this embodiment, each frame is composed of It consists of several joints, and the features of each joint node include three-dimensional local position features. Four-dimensional unit quaternion rotation characteristics And the global motion trajectory, which includes the horizontal linear velocity. angular velocity of rotation about the vertical axis and the vertical height of the root node relative to the ground Together constitute The original feature vector.

[0043] To establish fine-grained feature representations, this embodiment divides N joint nodes into P disjoint semantic sets based on human kinematics. This partitioning method deconstructs the globally complex motion into multiple functional units. Through feature mapping and aggregation, the variable-length local joint data is transformed into a standardized input format that the network can process. For each time t, the features of the joints contained in the p-th functional unit are concatenated sequentially to obtain the original state vector of that functional unit. Since the number of joints in each functional unit varies, a set of learnable linear projection layers is introduced to project the joints of different dimensions. Mapped to a unified embedding dimension And apply LayerNorm to normalize it along the feature dimension: After projection mapping, the original motion data can be represented as a sequence of lengths... spatiotemporal joint sequence, For time frames: Step 3.2: Obtain the target style vector: In the reference mode, the reference motion sequence containing the target style is input into the style feature extractor E. The implicit style attributes are extracted through the temporal attention layer to obtain the style feature vector. In the sampling mode, random latent vectors are sampled from the standard normal distribution. Then, the style category labels of the reference motion sequence. The encoding is a one-hot embedding, and both are input into the style mapping network F to obtain the target style vector. Step 3.3: Input the original content motion sequence into the motion stylization generator G. The spatiotemporal relationships between joints are modeled through a multi-layer spatiotemporal attention layer of the content feature extractor. Then, pure content semantic features without style are extracted through instance normalization. Step 3.4: Using the two style vectors obtained in Step 3.2 as style guidance signals, and combining them with the content semantic features obtained in Step 3.3, input them into the style guidance synthesis network of the motion stylization generator G, and output the generated stylized motion sequence. Step 3.5: Combine the stylized motion sequence generated in Step 3.4 with the target style tag. The generated samples are input into the style discriminator D to calculate the discrimination score in the target style domain, while also referencing the motion sequence and the target style label. Input, calculate the discrimination score of the real sample. Step 3.6: Calculate the total loss based on the input motion sequence, the generated motion sequence, and the discrimination score obtained in Step 3.5. Update the network parameters of each submodule Loss function design: A training strategy employing joint optimization of multiple loss functions is used. The overall training process is based on an adversarial generative framework, applied to the motion style generator. Style discriminator Style Feature Extractor and style mapping network We will conduct end-to-end training, and the specific losses will be described below.

[0044] Combat losses This is used to constrain the generated actions to maintain consistency with the true target style domain at the overall distribution level. A conditional discrimination strategy is employed, given a style label... Under these conditions, style discriminator Determining the authenticity of the input motion sequence will help mitigate the adversarial loss. Defined as: in, For real motion samples from the dataset, These are the style domain labels corresponding to the real samples. For the target style domain tag, Corresponding style discriminator Corresponding style domain The output branch represents the probability that the input is judged as a real sample. Represents motion stylization generator According to the source movement and style characteristics A stylized movement of synthesis.

[0045] To enhance training stability, the style discriminator... Additional R1 regularization loss : in Style discriminator By penalizing the gradient norm at real sample inputs, the discriminator is prevented from overfitting, thus mitigating the gradient instability problem commonly encountered in adversarial generative networks training.

[0046] To ensure that the semantic structure of the original motion is not destroyed during the stylization process, a content reconstruction loss is employed. With cycle consistency loss Constrain the generated results.

[0047] When motion stylization generator Encoded in the original style When used as style input, its output should be able to reconstruct the original motion, and the corresponding reconstruction loss is defined as: in , representing the style features extracted from the original motion by the style feature extractor.

[0048] Cyclic consistency constraints require that an action can still obtain the original motion after undergoing a mapping from "source style → target style → source style": To ensure that the generated motion accurately reflects the target style characteristics, a style reconstruction loss is employed. The requirement is that the style feature representation extracted from the generated motion after passing through the style feature extractor must be consistent with the input style features. in , This represents the L1 norm, used to measure the differences between style features.

[0049] Meanwhile, to mitigate the problem of generative models degenerating into a single output in the style space, an improved diversity-sensitive loss is used. This encourages a rich variety of outputs by comparing the differences between the generated results of randomly sampled style vectors. in and Style feature vectors generated for different random samples This represents a cutoff function used to limit the difference to a threshold. This is to prevent numerical instability during training.

[0050] To improve the continuity and plausibility of generated motion over time, a regularization constraint based on kinematic priors is introduced. Velocity regularization, used to limit joint position changes between adjacent frames, is defined as: Acceleration regularization further constrains the second-order rate of change of motion, preventing instantaneous joint abrupt changes and jitter phenomena: Among the various losses The subscript is used to limit the variables involved in the calculation of this loss term. This means that the corresponding loss values ​​of all samples in the current batch are arithmetically averaged, and the sample mean is used as an unbiased estimate of the overall distribution characteristics.

[0051] Finally, the following weighted objective function is used to jointly optimize the model, where The hyperparameter weights for the corresponding loss terms: Step 4: Using the trained human motion generation model, take the original motion content sequence and target style information as input, and output a stylized human motion sequence that retains the original action semantics and possesses the target style characteristics. Experimental section: This invention is implemented based on the PyTorch deep learning framework, and all experiments were conducted on a single NVIDIA GeForce RTX 3060 GPU for both training and testing. The main hardware and software configurations used in the experiments are shown in Table 1.

[0052] Table 1 Experimental Environment To support multi-task learning and cross-style mapping, the model employs a joint training strategy with multiple optimizers. Independent optimizers are set for the motion stylization generator, style discriminator, style feature extractor, and style mapping network, with parameters updated separately within the same training iteration. A weight decay term is introduced during optimization to suppress overfitting. Considering the different tasks of different modules in adversarial training, this embodiment sets different initial learning rates for them. Specifically, the motion stylization generator… and style discriminator The learning rate is relatively large, and all are set to a style feature extractor. Style mapping network The learning rates are respectively set as and Furthermore, an exponentially decaying learning rate scheduling strategy was introduced during the optimization of all modules. During training, the learning rate gradually decays at a fixed rate, and its update rule can be expressed as: in, This represents the initial learning rate. The attenuation coefficient is set to 0.95. This represents the number of scheduling steps. This strategy can gradually reduce the learning rate as training progresses, which is beneficial for the model to achieve more stable convergence under the constraints of complex loss functions.

[0053] Qualitative analysis of experimental results The stylized motion generation effect of this invention will be qualitatively analyzed by displaying stylized effects based on reference sequences and generating diversity through random sampling under fixed style conditions.

[0054] First, a qualitative analysis of the stylized motion generation capability of the method of this invention in reference mode is performed. In this mode, the model does not rely on style category labels, but only inputs two motion sequences. The model extracts style features from the untrained reference motion sequence and injects them into the source content sequence to generate stylized motion. Experimental results are shown below. Figure 4 As shown, to fully verify the model's reliability in scenarios where content and style are misaligned, the experiment selected multiple combinations with different content semantics and styles as test samples. The style and content information of the input sequence were labeled below the motion sequence. The generated results show that the model can generate stylized motion sequences with accurate style representation and well-preserved content even with misaligned combinations. Figure 4 The second line uses "Old Kicking" as the source content sequence and "Proud Walking" as the reference sequence. The generated stylized sequence fully retains the kicking action and does not contain content information such as arm swinging and walking in the reference sequence. At the same time, the "proud" style feature with an upright torso in the reference content sequence clearly replaces the "old" style feature with a bowed head and bent back in the content sequence. Figure 4 The third line uses "Depressed Running" as the source content sequence and "Proud Jumping" as the reference sequence. Although there are significant differences in the upper and lower limb movement patterns of the two movements, the generated stylized sequence still retains the gait characteristics of running movements while effectively removing the "depressed" style features such as head down and arms hanging down in the source sequence, and successfully injects the feature of arms outstretched in the reference sequence.

[0055] In summary, the qualitative experimental results under reference mode demonstrate that the method of this invention can effectively extract and inject unknown reference motion styles without relying on style tags. Even when the body parts used by the reference action and the source action differ significantly, the model can still ensure semantic invariance and accurately complete style injection, exhibiting good content-style decoupling ability and cross-content generalization performance.

[0056] In addition, to intuitively verify the generation diversity of this method under the condition of fixed style label, a fixed source content motion sequence is selected as input. While keeping the content unchanged, a specified target style category label is input, and multiple sets of stylized motion sequences are generated by independently sampling the latent variables of the input style mapping network multiple times.

[0057] Stylized motion sequence generation results are as follows Figure 5 As shown, jump and run, which have significantly different motion structure complexities, were selected as the content sequence inputs. The target style domains were designated as old and proud, respectively. For each experiment, four sets of latent variables were obtained by independent random sampling from a standard Gaussian distribution. via style mapping network It is mapped to a style feature vector under the target style domain, and finally passed through a motion stylization generator. Synthetic stylized motion. Figure 5 The first line is the source content sequence, the second line is the specified target style tag, and the remaining four lines are the stylized motions generated for four different latent variables.

[0058] It can be observed that Figure 5 In the first column of the "jump+old" stylized results, all generated movements consistently exhibited characteristics of an aged style from takeoff to landing, showing obvious back hunching and forward leaning. However, differences remained in details, such as variations in the torso's forward lean angle and arm amplitude, demonstrating that the method of this invention can effectively generate diverse styles within the same style domain. In the second column of the "run+proud" experimental results, compared to the source motion sequence, the generated "proud" style running motion generally displayed a more upright and chest-out upper limb posture, with the arms swinging from small amplitude to large amplitude. Simultaneously, different random samples also showed significant differences in arm swing height and amplitude.

[0059] In summary, the diversity generation experiment effectively demonstrates that, under the condition of fixed content and style tags, the method of this invention does not degenerate into a single repetitive generation of the same style. The style mapping network successfully learns the continuous latent distribution within the style domain, allowing the model to control fine-grained changes in style performance through random latent variable z while ensuring the consistency of the style specified by the style tag.

[0060] Quantitative comparison and analysis To objectively evaluate the performance of the method of the present invention in terms of stylized motion generation quality, content semantic preservation ability, and target style expression accuracy, this embodiment uses FMD (Fréchet Motion Distance), Content Recognition Accuracy (CRA), and Style Recognition Accuracy (SRA) as quantitative evaluation indicators.

[0061] Fréchet Distance (FMD) is used to measure the overall difference between the generated motion distribution and the real motion distribution. Since motion data lacks a universal feature extraction network, this embodiment refers to the method proposed by Yan et al., and pre-trains a motion classifier based on a spatiotemporal graph convolutional network. High-level features are extracted from the global pooling layer of the classifier to represent the statistical distribution characteristics of the motion sequence. Then, the distribution deviation (FMD value) between the real samples and the generated results in this feature space is quantitatively measured. FMD can comprehensively measure the quality of generated motion from aspects such as temporal variation and spatial coordination. Its value is negatively correlated with the generation quality; the lower the value, the closer it is to the real motion distribution, and the better the generation quality.

[0062] Content recognition accuracy is used to quantify the ability to preserve the semantics of the original content during stylization. In this embodiment, a pre-trained action classifier is still used to predict the content category of the generated stylized motion, and then the proportion that matches the real content label of the source content sequence is counted. A high content recognition accuracy means that the model can successfully preserve the core kinematic features of the original action and will not cause the original semantics to become blurred due to over-stylization.

[0063] Style recognition accuracy is used to evaluate the strength of the target style features expressed in the generated result. In this embodiment, an independent style classifier is first trained based on the style-annotated data in the training set. Then, the generated stylized motion is input into the classifier to predict the style, and the proportion of those identified as the target style category is counted. A high style recognition accuracy indicates that the model can accurately capture the key motion features of the target style domain, making the generated result significant in terms of the target style.

[0064] To verify the effectiveness of the method of the present invention, the experiment selected several comparative models, including Holden et al. and Aberman et al., which are based on 1D CNN convolution in the time axis dimension, Park et al., which introduces spatiotemporal modeling, Motion Puzzle, which supports style control of multiple parts, and Most model based on Transformer architecture.

[0065] The experimental results are shown in Table 2. On the FMD metric, which reflects the realism of motion, the method of this invention scored 7.13, the best among all the compared models, and significantly better than Aberman et al.'s 16.82 based on one-dimensional convolution. This verifies that traditional one-dimensional convolution is limited by the receptive field, resulting in poor macroscopic consistency in the generated motion. The global self-attention mechanism of this invention can capture the complete dependencies of the entire sequence, making the generated motion closer to the real distribution. Even compared with Park et al. (7.88) which introduces spatial structure, this method still has an advantage in FMD. Regarding content preservation capability, this invention achieved the highest score of 65.21%, while the one-dimensional convolution model CRA, which can only extract local features, was below 40%. This confirms that the spatiotemporal attention mechanism of this invention can more accurately separate motion content features. Although MoST achieved 63.32% on CRA, its style recognition rate was only 33.42%, indicating that the method still has an imbalance in content and style decoupling. The method of this invention achieves an SRA of 44.29%, surpassing the highest among the comparative models, Motion Puzzle (43.94%). This is attributed to the latent space mapping mechanism during the training phase, which forces the distribution of the style space to be more generalizable through latent space sampling. This allows the style feature extractor to map the reference sequence to a more discriminative feature space during the testing phase. Furthermore, the global modeling capability of the Transformer allows the model to not only focus on local motion amplitudes but also extract style features from the overall distribution. Compared to Motion Puzzle's one-sided focus on body parts, style features containing global information are more discriminative during classification, thus achieving a higher style recognition accuracy.

[0066] Table 2 Comparison of experimental results

Claims

1. A stylized human motion generation method based on spatiotemporal attention, characterized in that, Includes the following steps: Step 1: Using a publicly available human motion style dataset, preprocess it and then divide it into training and testing sets; Step 2: Construct a stylized motion generation network model based on spatiotemporal attention; For a given source motion sequence ,in Represents the number of frames. Represents the number of joints in the human body. Representing the feature dimensions of each joint, the network model supports two input styles: (1) Reference mode: Input a reference motion with the target style ; (2) Sampling mode: Input a target style domain label ,in The total number of style domains; and a potential random code that follows a normal distribution. ; The network model takes the original motion content and target style information as input and outputs a stylized motion sequence that preserves semantic content and has target style characteristics. It includes four sub-modules: MotionStylizer Generation (G), Style Feature Extractor (E), Mapping Network (F), and Style Discriminator (D). The motion stylization generator is based on the Transformer's attention mechanism. It extracts content representations from the source motion sequence using multiple spatiotemporal attention layers and reconstructs a stylized motion sequence by combining the target style features. The motion stylization generator includes a Content Feature Extractor and a Style-Guided Synthesis Decoder (SGSD). The Content Feature Extractor adopts a multi-layer residual Transformer based on pre-norm logic and introduces an InstanceNorm layer at the end of the extractor to constrain the feature distribution. The Style-Guided Synthesis Decoder (SGSD) is responsible for combining the extracted content features with the target style features. Perform deep fusion and reconstruct the final motion sequence; The style feature extractor extracts style sequence features from the reference pattern and receives a reference motion sequence with the target style. As input, its implicit style attributes are captured through multiple spatiotemporal attention layers, and the final output is a feature vector that can express the style of the reference motion, namely the style token. The style mapping network receives two pieces of information upon input: one is a low-dimensional latent vector sampled from a standard normal distribution. Secondly, it represents discrete labels indicating the target style category. Random latent codes that follow a normal distribution With target style domain tags Perform nonlinear fusion mapping to generate the corresponding style feature vector; The style discriminator D includes a shared feature extraction layer and multiple independent output branches corresponding to different style domains. It adopts a multi-task adversarial learning architecture to guide the optimization direction of the motion stylization generator. During training, it is responsible for judging whether the generated motion sequence conforms to the style distribution of the target style domain. Step 3: Train the stylized motion generation network model based on spatiotemporal attention using the training set and update the parameters; Step 3.1: Reconstruct the original joint space data into a spatiotemporal joint representation; Step 3.2: Obtain the target style vector: In the reference mode, the reference motion sequence containing the target style is input into the style feature extractor E. The implicit style attributes are extracted through the temporal attention layer to obtain the style feature vector. In the sampling mode, random latent vectors are sampled from the standard normal distribution. Then, the style category labels of the reference motion sequence. The encoding is a one-hot embedding, and both are input into the style mapping network F to obtain the target style vector. Step 3.3: Input the original content motion sequence into the motion stylization generator G, model the spatiotemporal relationship between joints through the multi-layer spatiotemporal attention layer of the content feature extractor, and then extract the pure content semantic features without style through instance normalization (InstanceNorm); Step 3.4: The two style vectors obtained in Step 3.2 are used as style guidance signals, and together with the content semantic features obtained in Step 3.3, they are input into the style guidance synthesis network of the motion stylization generator G, and the generated stylized motion sequence is output. Step 3.5: Combine the stylized motion sequence generated in Step 3.4 with the target style tag. The generated samples are input into the style discriminator D to calculate the discrimination score in the target style domain, while also referencing the motion sequence and the target style label. Input the data and calculate the discrimination score for the real sample. Step 3.6: Calculate the total loss based on the input motion sequence, the generated motion sequence, and the discrimination score obtained in Step 3.

5. Update the network parameters of each submodule; Step 4: Using the trained human motion generation model, take the original motion content sequence and target style information as input, and output a stylized human motion sequence that retains the original action semantics and has the target style characteristics.

2. The stylized human motion generation method based on spatiotemporal attention according to claim 1, characterized in that, The preprocessing described in step 1 is as follows: all motion data in the human motion style dataset are redirected to a unified CMU skeletal topology, which contains N=21 joints, and the original sequence is downsampled to 60 fps to reduce time redundancy. Subsequently, the action sequence was augmented by mirroring to expand the training samples.

3. The stylized human motion generation method based on spatiotemporal attention according to claim 2, characterized in that, The content feature extractor described in step 2, during the feature transformation process, provides... Each spatiotemporal feature vector is encoded with temporal position and spatial position respectively to explicitly represent the temporal sequence and spatial structure of motion. A spatiotemporal separation attention mechanism is used to transform the features layer by layer: first, multi-head self-attention is performed in the spatial dimension to calculate the spatial relationship between P functional units within each frame, and then multi-head self-attention is performed in the temporal dimension to capture the temporal dependency relationship across frames, thereby obtaining the phase change and rhythm features of the action.

4. The stylized human motion generation method based on spatiotemporal attention according to claim 3, characterized in that, Step 2 describes a style-guided synthesis network that achieves precise style injection during synthesis through multiple synthesis layers containing Adaptive Instance Normalization (AdaIN); in each Transformer block, style features are extracted from the reference style feature extractor. or style mapping network Target style characteristics As a conditional signal, the feature distribution of the spatiotemporal feature vector is dynamically adjusted.

5. A stylized human motion generation method based on spatiotemporal attention according to claim 4, characterized in that, Step 2 introduces a global aggregation-based approach to the style feature extractor: a learnable style token (LST) is concatenated at the beginning of the input spatiotemporal feature sequence to form an enhanced spatiotemporal feature sequence. This sequence then enters a multi-layer spatiotemporal attention layer. The LST, acting as a query vector, actively extracts and aggregates motion features from different time points and spaces within the entire sequence using a self-attention mechanism. After multiple attention updates, the LST gradually evolves from an initial learnable vector into a high-dimensional representation that aggregates global style information.

6. The stylized human motion generation method based on spatiotemporal attention according to claim 5, characterized in that, Step 2 describes a style mapping network that employs a dual-branch parallel feature extraction method, with latent vectors... The one-hot embedding and the feature vector are input into two separate fully connected layers. Each branch consists of a linear mapping layer and a LeakyReLU activation function. The features from the two branches are then concatenated along the channel dimension to form a joint feature representation, which is then input into a shared mapping module. This shared mapping module consists of a multi-layer fully connected network that performs depth mapping on the joint features through layer-by-layer linear transformations and activation functions, with the hidden layer dimensions gradually expanding to enhance the model's ability to represent complex style distributions. Finally, the network maps the features to a preset style dimension through a linear output layer. ; The style mapping network finally uses the Tanh activation function to constrain the range of style feature values; the style mapping network ultimately outputs a style vector. As a target style feature, it can be directly fed into the style-guided synthesis network.

7. The stylized human motion generation method based on spatiotemporal attention according to claim 6, characterized in that, Step 2 describes a style discriminator that employs a multi-task adversarial learning architecture. This discriminator includes a shared feature extraction layer and multiple independent output branches corresponding to different style domains. For the input sequence, it first extracts a unified spatiotemporal feature representation, then feeds this global discriminative feature into the multi-task classification network, outputting a discriminative score vector for all style domains, given a target style label. The model selects only the discrimination score of the corresponding style domain as the final output, thereby achieving conditional discrimination: During adversarial training, the optimization objective of the style discriminator is divided into the following two parts: (1) Discrimination of real samples: The style discriminator D receives real motion data from the dataset. The goal is to maximize the probability that the output is "true" under the corresponding style tag, that is... In order to learn the spatiotemporal characteristics and distribution patterns of the target style domain; (2) Detection of fake samples: The style discriminator D receives the motion generated by the motion stylization generator G. The goal is to maximize the probability that its output is "false" under the same style conditions, that is... By identifying the distributional differences between generated motion and real data, potential style deficiencies or unnatural phenomena in generated motion can be discovered.

8. The stylized human motion generation method based on spatiotemporal attention according to claim 7, characterized in that, Step 3.1 describes the spatiotemporal joint representation process, which includes: Human motion sequences are usually represented as Each joint node's features include three-dimensional local location features. Four-dimensional unit quaternion rotation characteristics And the global motion trajectory, which includes the horizontal linear velocity. angular velocity of rotation about the vertical axis and the vertical height of the root node relative to the ground Together, they constitute the d-dimensional original feature vector; Based on human kinematics, N joint nodes are divided into P disjoint semantic sets. Through feature mapping and aggregation, the variable-length local joint data is transformed into a standardized input format that the network can process. For each time t, the features of the joints contained in the p-th functional unit are concatenated in sequence to obtain the original state vector of that functional unit. Since the number of joints in each functional unit varies, a set of learnable linear projection layers is introduced to project the joints of different dimensions. Mapped to a unified embedding dimension And apply LayerNorm to normalize it along the feature dimension: After projection mapping, the original motion data can be represented as a sequence of lengths... spatiotemporal joint sequence, For time frames: 。 9. The stylized human motion generation method based on spatiotemporal attention according to claim 8, characterized in that, The total loss described in step 3.6 A training strategy employing joint optimization of multiple losses is adopted, and the overall training process is based on an adversarial generative framework for the motion stylization generator. Style discriminator Style Feature Extractor and style mapping network Perform end-to-end training; A conditional discrimination strategy is adopted, given a style tag Under the condition that the style discriminator Determining the authenticity of the input motion sequence will help mitigate the adversarial loss. Defined as: in, For real motion samples from the dataset, These are the style domain labels corresponding to the real samples. For the target style domain tag, Corresponding style discriminator Corresponding style domain The output branch represents the probability that the input is judged as a real sample. Represents motion stylization generator According to the source movement and style characteristics Stylized movements through synthesis; To enhance training stability, the style discriminator... Additional R1 regularization loss : in Style discriminator The gradient with respect to the real sample input; To ensure that the semantic structure of the original motion is not destroyed during the stylization process, a content reconstruction loss is employed. With cycle consistency loss Constrain the generated results; When the motion stylization generator Encoded in the original style When used as style input, its output should be able to reconstruct the original motion, and the corresponding reconstruction loss is defined as: in , representing the style features extracted from the original motion by the motion stylization generator; Cyclic consistency constraints require that an action can still obtain the original motion after undergoing a mapping from "source style → target style → source style": To ensure that the generated motion accurately reflects the target style characteristics, a style reconstruction loss is employed. The requirement is that the style feature representation extracted after the motion passes through the style feature extractor must be consistent with the input style features. in , L1 norm is used to measure the differences between style features; Meanwhile, to mitigate the problem of generative models degenerating into a single output in the style space, an improved diversity-sensitive loss is used. This encourages a rich variety of outputs by comparing the differences between the generated results of randomly sampled style vectors. in and Style feature vectors generated for different random samples This represents a cutoff function used to limit the difference to a threshold. This is to prevent numerical instability during training; To improve the continuity and plausibility of generated motion in the temporal dimension, a regularization constraint based on kinematic priors is introduced; velocity regularization is used to limit joint position changes between adjacent frames, and is defined as: Acceleration regularization further constrains the second-order rate of change of motion, preventing instantaneous joint abrupt changes and jitter phenomena: Among the various losses The subscript is used to limit the variables involved in the calculation of this loss term. This means that the corresponding loss values ​​of all samples in the current batch are arithmetically averaged, and the sample mean is used as an unbiased estimate of the overall distribution characteristics. Finally, the following weighted objective function is used to jointly optimize the model, with the total loss being... Defined as: in These are the hyperparameter weights corresponding to the loss term.

Images