Attitude generation method based on style space attention perception conditional adversarial network

Abstract

The invention discloses a gesture generation method based on a style space attention perception condition adversarial network, which comprises the following steps of: constructing a style space attention perception condition generative adversarial network model, performing attention dynamic graph perception operation on the model, extracting spatial position characteristics of human body three-dimensional skeleton data, and calculating the spatial position characteristics of the human body three-dimensional skeleton data; fusing the spatial direction vector information and the style vector information, and performing feature reconstruction and aggregation; the model constructs a condition generator and a condition recognizer, and the condition recognizer guides the condition generator to generate a three-dimensional attitude with a reasonable structure and a clear style in combination with authenticity discrimination and style classification tasks; the model is combined with skeleton side length consistency and joint included angle stability, an attitude geometric regularization loss function is constructed, a condition generator and condition identifier loss function is constructed, and model parameters are optimized; and constructing a human body three-dimensional skeleton data set, carrying out iterative training on the model by utilizing the three-dimensional skeleton data set until the model is converged, and generating three-dimensional human body posture samples with diversified styles by utilizing the model.

Description

Technical Field

[0001] This invention relates to the field of elevator control technology, and more specifically, to a pose generation method based on a style space attention-aware conditional adversarial network. Background Technology

[0002] 3D human pose generation, as an important research direction in computer vision and artificial intelligence, is widely used in various scenarios such as virtual human driving, motion capture, game animation, behavior recognition, and style transfer. With the development of technologies such as virtual reality, augmented reality, and digital humans, the demand for high-quality and diverse human pose generation is increasing. This requires not only that the generated results have a reasonable spatial geometric structure, but also that they possess stylistic and controllable expressive capabilities to adapt to the needs of tasks such as personalized modeling and specific semantic transfer.

[0003] Therefore, achieving joint modeling and high-quality generation of 3D pose data in terms of style and structure has become a research hotspot of common interest in both academia and industry. Obtaining 3D skeleton models typically relies on reconstructing the positions of 3D keypoints from 2D images or video sequences. This process often involves multi-view projection, depth estimation, or optimization algorithms based on inverse rendering, making the computation complex and demanding on computational resources. Furthermore, in the absence of labeled or multi-view data, the generalization ability and stability of such methods are difficult to guarantee.

[0004] In contrast, some studies attempting to directly generate skeletons using traditional machine learning or deep learning methods (such as those based on pose dictionaries, pose embeddings, or simple neural networks) have certain advantages in computational efficiency. However, due to limited model capacity and structural expressiveness, they struggle to effectively capture complex human pose distributions and style variations, often exhibiting underfitting and generating poses lacking realism and structural integrity. Therefore, many unresolved issues remain regarding simplifying the modeling path, improving model expressiveness, and enhancing style control and structural consistency.

[0005] Three-dimensional human skeleton data consists of the coordinates of a set of numbered key points in three-dimensional space, such as... Figure 1 As shown, it contains rich spatial topological structure and semantic relationships of limbs. These key points not only exist in fixed anatomical levels, but are also influenced by both the physiological laws of human movement and stylistic expression. Compared with the features of image pixels or video frames, 3D skeleton data is sparse, and some advanced and complex neural network convolution operations cannot extract its spatial location features.

[0006] Efficiently modeling and capturing the spatial location information, orientation relationships, and motion constraints of 3D skeleton data samples has become a key challenge in 3D pose generation. Furthermore, traditional deep learning methods often employ fully connected or sequential modeling approaches to model skeleton points, making it difficult to capture the dependencies and stylistic differences between local spatial structures in sparse data. This results in generated poses that are messy or lack clear motion topology and individual characteristics. Figure 2 and Figure 3 As shown.

[0007] In recent years, Generative Adversarial Networks (GANs) have made groundbreaking progress in the fields of image and sequence generation. Their strong generative capabilities and learnable data distributions have made them a promising candidate for 3D pose generation tasks. In particular, within the framework of Conditional Generative Adversarial Networks (CGANs), researchers have begun to introduce semantic information or labels as conditional vectors to guide the generator in producing samples with more target features.

[0008] However, most existing conditional generative adversarial networks (GANs) focus on processing and generating samples in two-dimensional images, and they still face many challenges in three-dimensional pose generation tasks. For example, how to effectively integrate style information to achieve diversity and clear style expression in the generated results; how to model the complex spatial topology between pose joints to improve the structural fidelity of the generated pose; and how to design a reasonable regularization mechanism to ensure the geometric rationality and motion feasibility of the generated skeleton. These issues are all technical difficulties in generating three-dimensional pose data. Summary of the Invention

[0009] The purpose of this invention is to overcome the defects and shortcomings of the prior art and provide a pose generation method based on style space attention-aware conditional adversarial network. The method constructs a style space attention-aware conditional generative network (SSA-CGAN) model and introduces a dynamic fusion mechanism of style vectors and joint space orientation vectors into the generator to achieve controllable generation of multi-style 3D poses.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A pose generation method based on style space attention-aware conditional adversarial networks includes the following steps:

[0012] A style space attention-aware conditional generative adversarial network model is constructed. The model takes style labels as conditional input and generates corresponding multi-style 3D human poses based on the input style labels.

[0013] The model employs attention-based dynamic graph perception computation to extract spatial location features from the human 3D skeleton data, and integrates spatial orientation vectors and style vectors for feature reconstruction and aggregation.

[0014] The model constructs a condition generator and a condition recognizer. The condition recognizer combines the tasks of authenticity discrimination and style classification to guide the condition generator to generate 3D poses with reasonable structure and clear style.

[0015] The model combines skeleton side length consistency and joint angle stability to construct a pose geometry regularization loss function, construct condition generator and condition recognizer loss functions, and optimize model parameters.

[0016] Construct a 3D human skeleton dataset, use the 3D skeleton dataset to iteratively train the model until the model converges, and use the model to generate 3D human pose samples with diverse styles.

[0017] Compared to existing technologies, this invention employs attention-based dynamic graph perception computation, enabling it to dynamically adjust edge weight distribution based on keypoints for spatial orientation and style conditions. This enhances spatial topological perception capabilities and significantly improves the spatial representation quality of pose modeling and the quality of generated 3D pose samples. The model constructs a condition generator and a condition recognizer. The condition recognizer, combined with true / false discrimination and style classification tasks, guides the condition generator to generate 3D poses with reasonable structure and clear style. The model combines skeleton side length consistency and joint spatial angle rationality to construct a pose geometry regularization loss function, explicitly constraining the physical consistency and structural stability of generated poses. This allows SSA-CGAN to better capture the spatial positional features of each limb's movement patterns during training, thereby generating 3D pose samples that conform to human movement patterns. Attached Figure Description

[0018] Figure 1 This is a normal pose sample of a 3D skeleton.

[0019] Figure 2 The image is a 3D skeleton with a messy, inhuman-like pose.

[0020] Figure 3 The sample is a 3D skeleton that does not conform to the logic of human movement.

[0021] Figure 4 Dynamic attention perception computation of style space attention perception conditional generative adversarial network.

[0022] Figure 5 A condition generator for style space attention-aware conditional generative adversarial networks.

[0023] Figure 6 A conditional recognizer for a style space attention-aware conditional generative adversarial network.

[0024] Figure 7 An experimental example for obtaining 3D pose.

[0025] Figure 8 Partial data samples generated for MLP-cGAN.

[0026] Figure 9 Partial data samples generated for Proj-cGAN.

[0027] Figure 10 Partial data samples generated for Transformer-cGAN.

[0028] Figure 11 Partial data samples generated for ResMLP-cGAN.

[0029] Figure 12 Partial data samples generated for GNN-cGAN.

[0030] Figure 13 Partial data samples generated for CNN-cGAN.

[0031] Figure 14 A partial data sample generated for SSA-CGAN.

[0032] Figure 15 Remove some data samples generated by the attention-based dynamic graph perception mechanism from SSA-CGAN.

[0033] Figure 16 This refers to a portion of the data samples generated by removing the attitude geometry regularization constraints from SSA-CGAN.

[0034] Figure 17 This is a partial data sample generated for the full version of SSA-CGAN. Detailed Implementation

[0035] The pose generation method based on style space attention-aware conditional adversarial network of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0036] Please see Figure 1 This invention discloses a pose generation method based on style space attention-aware conditional adversarial networks, comprising the following steps:

[0037] A style space attention-aware conditional generative adversarial network model is constructed. The model takes style labels as conditional input and generates corresponding multi-style 3D human poses based on the input style labels.

[0038] The model employs attention-based dynamic graph perception computation to extract spatial location features from the human 3D skeleton data, and integrates spatial orientation vectors and style vectors for feature reconstruction and aggregation.

[0039] The model constructs a condition generator and a condition recognizer. The condition recognizer combines the tasks of authenticity discrimination and style classification to guide the condition generator to generate 3D poses with reasonable structure and clear style.

[0040] The model combines skeleton side length consistency and joint angle stability to construct a pose geometry regularization loss function, construct condition generator and condition recognizer loss functions, and optimize model parameters.

[0041] Construct a 3D human skeleton dataset, use the 3D skeleton dataset to iteratively train the model until the model converges, and use the model to generate 3D human pose samples with diverse styles.

[0042] 3D human pose generation is a key task in computer vision, with wide applications in scenarios such as virtual human driving, behavior modeling, and style transfer. Addressing the weaknesses of traditional methods, such as weak style representation and unreasonable spatial geometry, this invention proposes a Style-SpatialAttention Conditional GAN ​​(SSA-CGAN) to achieve high-quality generation of multi-style 3D poses.

[0043] The core of this method lies in employing Dynamic Graph Perception with Attention (DGPA), which fuses spatial orientation vectors and style vectors, dynamically adjusting the weights of adjacent edges to achieve feature aggregation based on local topology and style. Simultaneously, the recognizer combines the dual tasks of style classification and true / false detection, guiding the generator to produce pose sequences with clear styles and reasonable spatial structures.

[0044] To further improve the consistency and physical plausibility of posture structure, this invention introduces a posture geometry regularization loss, which enhances the geometric plausibility of generated samples at two levels: skeleton side length and joint angle. Test experiments on style-labeled 3D skeleton data samples constructed based on the Human3.6M dataset show that the samples generated by the proposed method significantly outperform existing methods in terms of style expressiveness and spatial structure fidelity.

[0045] This invention proposes a Style-Spatial Attention Conditional GAN ​​(SSA-CGAN) model. This model introduces a conditional adversarial generative mechanism, using style labels as input and employing Dynamic Graph Perception with Attention (DGPA) to extract spatial location features from 3D skeleton data samples. A pose geometry regularization constraint model is then established to guide the generator, generating 3D pose samples with differentiated styles while preserving the rationality of the motion.

[0046] Attention-based dynamic graph perception computation integrates spatial orientation vectors and style vectors. By adaptively constructing graph structures and edge weight distributions, it reconstructs and aggregates motion morphological features of keypoints under different style conditions. This graph structure not only supports dynamic learning of local spatial topological relationships but also possesses style control capabilities, thereby improving geometric consistency while achieving controllability of the spatial structure and style consistency of generated poses.

[0047] The SSA-CGAN proposed in this invention can improve the style control capability and spatial structure consistency of 3D pose generation. The core of this method lies in the constructed attention-based dynamic graph perception operation, which utilizes spatial direction vectors and style vectors to jointly adjust the adjacency weights between joints, achieving adaptive spatial location feature aggregation and local topology awareness. The attention-based dynamic graph perception constructs dynamic graph edge weights based on the relative spatial position and style conditions between nodes, explicitly modeling the correlation between node pairs. Simultaneously, an attention mechanism is introduced to highlight the contribution of key nodes, enhancing the model's ability to capture style-driven features.

[0048] In terms of adversarial structure design, this invention introduces a dual-branch discriminator, which not only judges the realism of generated samples but also classifies and learns their style categories. This multi-task learning approach guides the generator to improve style controllability and expression consistency. Regarding the loss function, to further constrain the structural rationality of generated poses, this invention proposes a Posture Geometric Regularization Loss (PGRL) function to train the model. This function comprehensively considers the consistency of skeleton side lengths and the stability of joint angles, ensuring that the generated samples conform to human motion laws and topological constraints in spatial structure. This loss design effectively suppresses unreasonable skeleton stretching and pose distortion, improving the spatial geometric quality and realism of the generated samples.

[0049] This invention constructs a style-labeled 3D skeleton dataset on the Human3.6M public dataset and comprehensively evaluates it through visual visualization comparison and style classification accuracy-related indicators. Experimental results show that the proposed SSA-CGAN significantly outperforms existing methods in recent years in terms of multi-style generation, spatial structure fidelity, and style discrimination consistency. Test experiments verify its effectiveness and versatility in high-quality stylized 3D pose generation.

[0050] The SSA-CGAN framework innovatively integrates conditional generation and dynamic graph attention mechanisms, enabling fine-grained style control and structural modeling of 3D poses, such as... Figure 4 , Figure 5 and Figure 6 As shown, by deeply fusing style vectors and pose noise at the node level, the generator can accurately capture style features and guide the pose to evolve towards the target style; the recognizer not only judges the authenticity of samples, but also supervises the style consistency of the generated results through the style classification head, forming a closed-loop style reinforcement training.

[0051] Meanwhile, the attention dynamic graph perception operation is based on node pair features and geometric direction vectors to dynamically construct the attention graph structure. Its multi-head mechanism effectively models the topological relationship of the skeleton, significantly improving the model's comprehensive modeling ability for pose space structure, style expression and geometric rationality, and ultimately achieving a 3D pose generation effect that is natural in structure, distinctive in style and flexible in control.

[0052] Traditional Graph Convolutional Networks (GCNs) perform well when processing graphs with fixed structures, but their performance suffers when the connections between nodes (skeleton keypoints) change with the input (e.g., skeleton pose changes over time or style), different nodes have different impacts on the target task (i.e., neighbors have unequal importance), and fixed weights cannot adapt to style, temporal, or contextual changes. When processing 3D pose sample sequences, where the spatial structure and constraints of keypoints are complex, the limitations of GCNs become particularly pronounced.

[0053] Therefore, an attention mechanism and a dynamic graph construction mechanism are introduced to allow the existence and weight of edges to be dynamically adjusted with the input, thereby assigning learnable importance to different neighboring nodes. Based on this, an attention-based dynamic graph perception operation is established. Let the original 3D skeleton data sample be... ,in For the training batch size, N×3 represents the 3D coordinates of N (e.g., 17) skeleton keypoints. The style vectors are set (d styles are defined). After concatenation, the input features are obtained:

[0054] (1)

[0055] In the formula, The input feature values ​​are concatenated. To concatenate operation functions, For the real number field;

[0056] Subsequently, edge features are constructed for each node:

[0057] (2)

[0058] In the formula, Let (i, j) be the edge features of the node pair (i, j). , These are the tensors after expanding the style vector, which are... Subtensors;

[0059] Next, the score of the multi-head attention mechanism is calculated:

[0060] (3)

[0061] In the formula, Let H be the score of the h-th attention head; H is the number of attention heads. , These are weight tensors and offset tensors;

[0062] Normalize the attention scores to obtain the attention weights:

[0063] (4)

[0064] In the formula, for through Normalized weight values, This is the normalization function;

[0065] Neighbor feature aggregation is performed using attention-weighted summation:

[0066] (5)

[0067] In the formula, The neighbor aggregation feature value of the h-th attention head;

[0068] Next, the three-way fusion is performed on the self-feature, the aggregated neighbor features, and the global directional features:

[0069] , (6)

[0070] In the formula, It represents the vector difference from the center key point of the 3D skeleton model to each key point, and is used to describe the geometric relationship between key points; The neighbor aggregation feature value of the h-th attention head; For the three-way feature fusion tensor; For the i-th skeleton keypoint of a single 3D skeleton data sample, This refers to the 0th skeleton keypoint of a single 3D skeleton data sample.

[0071] Feature updating based on fused features:

[0072] (7)

[0073] In the formula, Feature update tensor values, For weight tensors, For offset tensors, For the three-way feature fusion tensor;

[0074] Finally, the outputs of each individual are averaged and merged:

[0075] (8)

[0076] In the formula, The average tensor for multi-head computation; spatial features are encoded as Dimension refers to the feature dimension of the attention dynamic graph perception output to the generator and recognizer for subsequent calculations.

[0077] From equations (1) to (7), it can be seen that in the attention-based dynamic graph perception operation, the dynamic graph construction mechanism constructs edge features for each pair of nodes, integrating positional differences and node features to achieve adaptive graph structure construction, enabling the network to perceive geometric differences brought about by pose or style changes. The attention mechanism then assigns weights to each edge, allowing the network to automatically focus on neighboring features that are more important to the current node. The two work together to give the network structure flexibility and information selectivity, thus enabling more accurate modeling of subtle differences between 3D pose and style.

[0078] The condition generator is now established. Let the attitude noise tensor be... Its elements follow a normal distribution; the style vector is , representing the style control conditions to which each sample belongs, with a total of d style categories.

[0079] First, the style vector is copied and expanded to include the style features attached to each node:

[0080] (9)

[0081] In the formula, For tensor expansion functions, This is the tensor that expands s to each node;

[0082] Then concatenate it with attitude noise:

[0083] (10)

[0084] In the formula, This is the concatenated node feature tensor;

[0085] The concatenated node features are input into the attention-based dynamic graph perceptual computation:

[0086] (11)

[0087] in, The feature tensor output by the attention dynamic graph perception operation; This is attention-guided dynamic graph perception. Attention-guided dynamic graph perception constructs edge weights between nodes and aggregates neighborhood information to output a feature representation for each node, with a dimension of [missing information]. ;

[0088] Reconstruct and flatten the graph features into the input form of a fully connected network:

[0089] (12)

[0090] In the formula, For the reconstructed tensor, For tensor reconstruction functions;

[0091] The pose output vector is generated through a two-layer fully connected network:

[0092] (13)

[0093] In the formula, Output vector for attitude; , For weight tensors, and It is the offset tensor; The hyperbolic tangent function is used; ReLU is the activation function.

[0094] The attitude output vector Reconstructed into a 3D coordinate tensor, the final output tensor will be linear:

[0095] (14)

[0096] In the formula, The generated pose is represented by the three-dimensional coordinates of N joints;

[0097] With model hyperparameters The condition generator is represented as:

[0098] (15)

[0099] In the formula, This is a generator function.

[0100] To identify the authenticity of generated samples and whether their style conforms to the settings, a recognizer is constructed below. The input is a 3D pose sample. With style vectors Perform joint encoding: First, expand the style vector and then concatenate it with the pose:

[0101] (16)

[0102] (17)

[0103] In the formula, Three-dimensional pose samples With extended style features The tensor obtained by splicing;

[0104] Feature extraction using attention-based dynamic graph perceptual computation:

[0105] (18)

[0106] In the formula, The feature tensor output by the attention dynamic graph perception operation;

[0107] Flatten out the reconstructed node features:

[0108] (19)

[0109] In the formula, For characteristic tensors The tensor obtained after reconstruction;

[0110] Map it to a shared semantic space through a shared coding layer:

[0111] (20)

[0112] In the formula, For tensors mapped to a shared semantic space;

[0113] Calculate the true / false judgment output:

[0114] ;(twenty one)

[0115] In the formula, The output is the true / false label value. These are the weight tensor and offset tensor of the linear layer, respectively; Use the Sigmoid activation function;

[0116] Calculate style classification output:

[0117] ;(twenty two)

[0118] In the formula, Score the style category of the output. These are the weight tensor and the offset tensor, respectively.

[0119] In summary, from equation (16) to equation (22), the condition recognizer can be obtained as follows:

[0120] ;(twenty three)

[0121] ;(twenty four)

[0122] Equations (23) and (24) are the identifiers for data authenticity determination and style classification, respectively. These are the model hyperparameters of the conditional recognizer; , These are the recognizer functions for authenticity detection and style classification, respectively.

[0123] During the training process of generating 3D poses, since the data is randomly generated, without constraint guidance, a large number of invalid or even non-human motion patterns will be produced. Therefore, by constructing regularization constraints such as spatial, angular, symmetric, torsional consistency, and shoulder or hip line horizontality, the spatial structure rationality and naturalness of the generated poses are enhanced, thereby improving the quality of stylized 3D pose generation. The constraint model is constructed below.

[0124] Spatial bone segment length loss The calculation is as follows:

[0125] (25)

[0126] in, This represents the set of connections between key points (limb segments). , , where are the coordinates of the i-th keypoint in the real pose and the generated pose, respectively, and B is the training batch size. This represents the number of elements in set C, i.e., the number of limb segments.

[0127] Bone angle loss The calculation is as follows:

[0128] (26)

[0129] (27)

[0130] Where T is a set of triplets of angles of the left and right arms, left and right thighs and calves, spine, and head and neck joints (e.g., Figure 1 Points in (representing the spine), |T| represents the number of joint angles to be calculated. , These represent the joint angles of the generated pose or the actual pose, respectively. >0 is the included angle tolerance threshold, used to prevent small deviations from interfering with training. B and The definition is the same as in equation (25).

[0131] Symmetrical loss of left and right limbs The calculation is as follows:

[0132] (28)

[0133] in, It is a set of indices for symmetrical bone segments (such as the left and right thighs). The number of elements in the set. The symmetry objective requires paired symmetrical limbs to maintain the same length; B and... The definition is the same as that of formula (25).

[0134] Reversing Consistency Loss Its goal is to maintain the consistency of the shoulder, pelvis, and spine, preventing excessive twisting or unreasonable crossing, in accordance with the normal torsional limits of the human body. The calculation is as follows:

[0135] (29)

[0136] in, For the torsional triad set of the shoulder and hip (e.g.) Figure 1 Points in (Indicates right shoulder → chest → waist) | | represents the number of elements in the set. B and The definition is the same as that of formula (25).

[0137] horizontal loss of shoulder or hip line The heights of the shoulders or hips of the three-dimensional pose sample should be close to avoid unreasonable tilting. The calculation is as follows:

[0138] (30)

[0139] in, Indicates the shoulder line; Indicates the hip line, that is Figure 1 The vectors connecting points 11 to 14 and points 1 to 4 in the figure; Equation (30) requires the shoulder line and hip line to be consistent in the height direction in order to minimize the loss. This indicates the operation of calculating the average. They are respectively Figure 1 The coordinates of key points 1, 4, 11 or 14.

[0140] Combining the loss functions of the above five constraint models, the overall pose geometry regularization loss is obtained. The calculation is as follows:

[0141] (31)

[0142] In equation (31), , , , and These are the weight coefficients for each loss function, which need to be set according to the specific training situation.

[0143] Loss function of condition generator The structure is as follows:

[0144] (32)

[0145] Among them, the generated sample style classification loss Defined as:

[0146] (33)

[0147] Generate sample diversity loss Defined as:

[0148] (34)

[0149] In equation (32), Noise' is another set of random noise with the same dimension and distribution as Noise. This represents the style label vector corresponding to the real sample, and its dimension is the same as s; , and These are the weighting coefficients for each type of loss.

[0150] In equation (33), This indicates that after the conditional recognizer outputs, the i-th sample is classified into style. logical value, >0 represents the allowed difference threshold.

[0151] Equation (32) aims to enable the generated sample to "deceive" the conditional recognizer, that is, to be recognized as a real sample; The regularization loss is calculated according to Equation (31); Equations (33) and (34) are used to ensure the style accuracy of the generated samples and the diversity of the generated results, respectively.

[0152] The loss function of the conditional recognizer is constructed as follows:

[0153] (35)

[0154] (36)

[0155] In the formula, The loss function is used to train the conditional recognizer. This represents the loss of style recognition capability in the conditional recognizer. for Weighting coefficients; Encode the style category for the i-th sample; For the first The sample at the th Unnormalized scores in style category; for The result of exponentiation.

[0156] Equation (35) requires the recognizer not only to determine the authenticity of the sample, but also to accurately identify the style label s from the real sample, thereby improving the ability to identify the style and providing style supervision signals for the condition generator.

[0157] During final training, the hyperparameters of both the condition generator and the recognizer are jointly optimized by minimizing the sum of their losses. and ,Right now:

[0158] (37)

[0159] In the formula, This represents the operation of taking the value of the independent variable that minimizes the function; the optimization process involves iteratively updating the parameters through adversarial training between the generator and the recognizer. and Until it converges.

[0160] Three-dimensional pose is represented by a skeleton model, which can be obtained through three-dimensional reconstruction algorithms of two-dimensional images or videos. The resulting reconstruction results are used as training, validation, and test samples, such as... Figure 7As shown in Table 1. Alternatively, the dataset can be constructed based on a public dataset (such as Human3.6m): key points are selected from the 3D pose data acquired by sensors and converted to the format required by the model, such as 17×3, i.e., the 3D coordinates of 17 key points. Specifically, video sequences are selected from the Human 3.6m dataset, and the corresponding 3D pose samples are obtained after 3D reconstruction, thus constructing the dataset. The constructed dataset contains samples of 10 different styles, and their specific information is shown in Table 1. The environment configuration for model training and testing is shown in Table 2.

[0161] Table 1. Dataset pose sample acquisition information

[0162]

[0163] Table 2 Training and Testing Environment Parameters

[0164]

[0165] Table 3. SSA-CGAN training pseudocode

[0166]

[0167]

[0168] Under the environmental parameters shown in Table 2, the training pseudocode for the conditional adversarial model SSA-CGAN is shown in Table 3. When training SSA-CGAN, given the lack of objective and unified numerical evaluation standards for 3D pose generation tasks, a comprehensive evaluation strategy combining manual visual assessment and quantitative indicators was adopted. The generation results were visualized every few epochs, and manual evaluation was performed from dimensions such as pose naturalness, style expression clarity, and structural consistency. Quantitative indicators such as the style classification accuracy and generation diversity of the recognizer were used as references to determine whether the training had converged or whether to terminate it early.

[0169] During training, setting appropriate style labels and effectively avoiding overfitting are crucial for ensuring model performance. Labels should be input into both the generator and recognizer using a balanced one-hot encoding format. The recognizer needs to perform both true / false style discrimination and style classification tasks simultaneously to enhance the model's style expressiveness. To prevent overfitting, strategies such as Dropout, various regularization losses (e.g., pose geometric consistency loss, symmetry loss), style perturbation, and diversity constraints can be comprehensively employed. Simultaneously, dividing the training and validation sets and combining visualization results for supervision and evaluation ensures that the generated poses are both natural and reasonable, while maintaining style controllability. Furthermore, using appropriate optimizer parameters and an early stopping mechanism can further improve the model's generalization ability and stability.

[0170] To facilitate visual comparisons with recent conditional generative adversarial network (GAN) models, the selected models needed to be able to handle sparse 3D skeleton data. Therefore, the experiments compared various models, including MLP-cGAN (Multi-Layer Perceptron conditional GAN), Proj-cGAN (Projection conditional GAN), Transformer-cGAN (Transformer-based conditional GAN), ResMLP-cGAN (Residual Multi-Layer Perceptron conditional GAN), GNN-cGAN (Graph Neural Network conditional GAN), and CNN-cGAN (Convolutional Neural Network conditional GAN).

[0171] These conditional adversarial models are built upon deep learning frameworks, employing core computations from current mainstream deep learning models, such as one-dimensional convolution and graph neural networks. Experiments used the same dataset shown in Table 1 for training, validation, and testing. Subsequently, 10 different styles of 3D skeleton data samples were randomly generated and compared with the most similar real data. The differences between the generated and real data were observed through visualization. The results are as follows: Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 , Figure 13 and Figure 14 As shown, the first row contains real samples, and the second row contains samples generated by each model.

[0172] Depend on Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 , Figure 13 and Figure 14 As can be seen, compared with other conditional adversarial network models, the 3D skeleton generated by the SSA-CGAN model proposed in this invention is closest to the real data, while also showing some distinction, manifesting as different samples under the same style, and exhibiting good diversity. The samples generated by SSA-CGAN appear natural and realistic in terms of limb proportions, height, and posture.

[0173] Other models exhibit numerous problems: some, such as MLP-cGAN, ResMLP-cGAN, and SSA-CGAN, generate poses with significant variations in style, exhibiting distortions in limbs that violate the laws of human movement, and even displaying disproportionate features. More seriously, models like Transformer-cGAN, GNN-cGAN, and CNN-cGAN fail to generate even effective human skeleton models. This indicates that, compared to SSA-CGAN, these models have failed to fully learn the motion patterns, spatial positions, and proportional characteristics of the 3D skeleton, and their core computations cannot effectively extract key features from the data samples.

[0174] To further validate the effectiveness of the core design of the SSA-CGAN model, ablation experiments were conducted. Attention-based dynamic graph perception operations and pose geometry regularization constraints were removed, and 3D pose samples were regenerated. The ablation experiments were evaluated through visual comparison and downstream task testing. For the downstream task, a graph convolutional network-based style classifier, GCN-Classifier, was trained using the dataset in Table 1. This classifier performed well on this dataset, achieving training, validation, and test recognition rates of 99.49%, 98.81%, and 97.97%, respectively.

[0175] Samples generated from different ablation models are input into the graph convolutional network classifier for classification. If the generated samples are close to reality and effective, their input style should have high consistency with the classifier's predicted style. Therefore, traditional classification metrics such as precision, recall, accuracy, and F1 score can be used for evaluation. By comparing these metrics, the effectiveness of the designed core and regularization constraints can be determined. Visualization of the ablation experiments is provided, for example... Figure 15 , Figure 16 and Figure 17 As shown in Table 4, the relevant recognition rate indicators are as follows.

[0176] Table 4 shows the test results of the recognition rate of the convolutional network classifier.

[0177]

[0178] Depend on Figure 15 , Figure 16 , Figure 17As shown in Table 4, the complete SSA-CGAN significantly outperforms models that remove attention-based dynamic graph perception or lack pose geometric regularization constraints in terms of generated visual effects, recognition accuracy, and F1 score. In tests with graph convolutional network classifiers, the complete SSA-CGAN achieved a recognition accuracy of only 66.00% for generated samples. This is because the generated 3D poses are randomly variable, and the training samples for each style originate from video reconstruction sequences, resulting in some poses being quite similar and lacking clear style boundaries.

[0179] Therefore, many generated samples are in a transitional state between two styles, lacking distinct style features. Even if the graph convolutional network classifier performs perfectly on the original dataset, its generalization ability to newly generated poses remains limited, as the new poses themselves do not belong to the training set. This precisely illustrates the diversity of SSA-CGAN generated samples: they originate from training samples, yet differ from them. If the recognition accuracy of generated samples is too high (e.g., exceeding 95%), it may mean that the generated samples are too similar to the samples seen during classifier training, failing to fully reflect the diversity within the same style.

[0180] Conversely, if the accuracy is too low (e.g., below 50%), it likely indicates that a large number of invalid poses have been generated. Figure 15 , Figure 16 and Figure 17 It is evident that while the incomplete SSA-CGAN model exhibits a certain degree of diversity, it also produces many invalid poses (such as disproportionate limbs, chaotic joints, and excessive differences from realistic styles).

[0181] Therefore, the method of combining visualization observations with downstream tasks for evaluation is objective and effective. It demonstrates that the full version of SSA-CGAN outperforms its incomplete version and shows that the full model can generate effective and diverse 3D pose samples. The accuracy of downstream task recognition can well reflect the effectiveness of the generated samples, and its ideal range is directly related to the quality of the training, validation, and test datasets.

[0182] In the SSA-CGAN model, attention-based dynamic graph perception computation can adaptively aggregate features based on the dynamic topological relationships between joints, capturing the correlation between local and global poses under different styles. Simultaneously, the pose geometry regularization constraints introduced by the model (including spatial distance consistency and joint rotation angle consistency) ensure that the generated 3D poses conform to physical laws in terms of structural rationality and motion smoothness. The combination of these two aspects allows the model to effectively maintain joint constraints that conform to human kinematics while learning stylized motion features, thereby significantly improving the quality of motion feature learning and the realism of generated 3D pose samples.

[0183] Some evaluation methods for 3D skeleton models, such as the average error of joint positions or the cosine similarity between vectors, are difficult to effectively assess the rationality of the generated data. This is because these metrics cannot fully identify motion features and 3D spatial structure information, which may lead to invalid data appearing superior in numerical computation. Therefore, this invention uses a downstream task to test the effectiveness of the data generated by each model, namely, evaluating it using the recognition accuracy of a graph convolutional network classifier.

[0184] As shown in Table 5, this invention tested 500 3D pose samples generated by each model. The results show that SSA-CGAN outperforms the other six models, with its recognition accuracy and related index values ​​being significantly higher than the comparison models, ranging from 60% to 70%. Combined with visual observation results, this demonstrates that the 3D pose samples generated by SSA-CGAN are not only more effective than the other six models, but also possess corresponding stylistic diversity.

[0185] Table 5. Test results of GCN-Classifier recognition rate for each model

[0186]

[0187] In this comparative experiment, SSA-CGAN significantly outperformed MLP-cGAN, Proj-cGAN, Transformer-cGAN, ResMLP-cGAN, GNN-cGAN, and CNN-cGAN in key metrics such as effective pose proportion. Its performance advantage stems primarily from two aspects: First, it introduces attention-based dynamic graph perception computation into both the generator and recognizer, enabling adaptive modeling of dynamic dependencies between joints based on input pose and style vectors, thereby enhancing the ability to capture details in complex pose spaces. Second, it incorporates pose geometric regularization constraints (including bone length consistency, joint angle restrictions, and torsional alignment) during training, effectively suppressing non-physical distortions and ensuring the rationality of limb movements in the generated samples.

[0188] Furthermore, style embedding continues to play a role in node-by-node feature aggregation, and the dual-task supervision of conditional discriminant discrimination and style classification further enhances the style consistency and realism of the generated results. These designs enable SSA-CGAN to significantly improve sample diversity and style controllability while maintaining high sample generation efficiency.

[0189] This invention addresses the problem of 3D pose generation by proposing the SSA-CGAN model. This model can effectively capture the spatial position information of limbs and their motion characteristics from 3D skeleton data samples, learn the distribution of samples of various styles, and generate effective new samples that conform to the target style and are different from the training set.

[0190] Attention-based dynamic graph perception significantly enhances the model's ability to model inter-joint dependencies. SSA-CGAN's attention-based dynamic graph perception not only utilizes the inherent topological structure of the human skeleton but also dynamically adjusts the weights of edges in the graph and the allocation of multi-head attention based on input pose and style information. This mechanism allows the model to simultaneously capture local details (such as hand micro-movements) and global coordination (such as the linkage between upper and lower limbs), significantly improving the performance of generated poses in terms of spatial structural accuracy and style representation compared to static graph convolution or simple multilayer linear perception.

[0191] Posture geometry regularization constraints effectively ensure that the generated postures conform to the laws of human movement. During model training, regularization terms such as spatial bone length constraints and joint angle constraints are introduced to ensure that the generated postures maintain stable bone lengths and reasonable joint rotations in three-dimensional space, thereby significantly reducing phenomena that violate biomechanical principles, such as limb overlap and abnormal twisting. These physical constraints are particularly crucial in style-controlled generation, enabling the maintenance of the spatial structural realism and stability of the posture model while achieving diverse outputs.

[0192] The synergistic effect of style consistency and conditional adversarial training improves the generation quality and controllability. SSA-CGAN combines the style classification loss of generated samples with the conditional adversarial loss, enabling the generator to maintain consistency between the output and style labels when learning different style features, while driving the recognizer to strengthen its style recognition ability in true / false discrimination. Experiments show that this strategy significantly outperforms other models (such as MLP-cGAN, Proj-cGAN, Transformer-cGAN, etc.) in multi-style scenarios in terms of F1 score, accuracy, etc., demonstrating its comprehensive advantages in style controllability, generation quality, and generalization ability.

[0193] Furthermore, if the training data can be more rigorously and precisely arranged and selected according to style, the recognition accuracy of the generated samples in downstream style classification tasks is expected to improve further. At the same time, this also helps to better maintain style diversity while enhancing the effectiveness of the generated samples.

[0194] The above description is a detailed description of the preferred embodiments of the present invention. However, the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modifications made under the technical spirit disclosed in the present invention should fall within the patent scope covered by the present invention.

Claims

1. A pose generation method based on style-space attention-aware conditional adversarial networks, characterized in that, Includes the following steps: A style space attention-aware conditional generative adversarial network model is constructed. The model takes style labels as conditional input and generates corresponding multi-style 3D human poses based on the input style labels. The model employs attention-based dynamic graph perception computation to extract spatial location features from the human 3D skeleton data, and integrates spatial orientation vectors and style vectors for feature reconstruction and aggregation. The model constructs a condition generator and a condition recognizer. The condition recognizer combines the tasks of authenticity discrimination and style classification to guide the condition generator to generate 3D poses with reasonable structure and clear style. The model combines skeleton side length consistency and joint angle stability to construct a pose geometry regularization loss function, construct condition generator and condition recognizer loss functions, and optimize model parameters. Construct a 3D human skeleton dataset, use the 3D skeleton dataset to iteratively train the model until the model converges, and use the model to generate 3D human pose samples with diverse styles.

2. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 1, characterized in that, Attention-based dynamic graph perception computation is employed, including: Let the original 3D skeleton data sample be ,in For the training batch size, N×3 represents the 3D coordinates of N skeleton keypoints. The style vector is defined by d styles; the concatenation of these styles yields the input features: ； In the formula, The input feature values ​​are concatenated. To concatenate operation functions, For the real number field; Subsequently, edge features are constructed for each node: ； In the formula, Let (i, j) be the edge features of the node pair (i, j). , These are the tensors after expanding the style vector, which are... Subtensors; Next, the score of the multi-head attention mechanism is calculated: ； In the formula, Let H be the score of the h-th attention head; H is the number of attention heads. , These are weight tensors and offset tensors; Normalize the attention scores to obtain the attention weights: ； In the formula, for through Normalized weight values, This is the normalization function.

3. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 2, characterized in that, Attention-based dynamic graph perception computation is employed, including: Neighbor feature aggregation is performed using attention-weighted summation: ； In the formula, The neighbor aggregation feature value of the h-th attention head; Next, the three-way fusion is performed on the self-feature, the aggregated neighbor features, and the global directional features: ， ； In the formula, This represents the vector difference from the center key point of the 3D skeleton model to each key point; The neighbor aggregation feature value of the h-th attention head; For the three-way feature fusion tensor; For the i-th skeleton keypoint of a single 3D skeleton data sample, This refers to the 0th skeleton keypoint of a single 3D skeleton data sample. Feature updating based on fused features: ； In the formula, Feature update tensor values, For weight tensors, For offset tensors, For the three-way feature fusion tensor; Finally, the outputs of each head are averaged and fused: ； In the formula, The average tensor for multi-head computation; spatial features are encoded as Dimension refers to the feature dimension of the attention dynamic graph perception output to the generator and recognizer for subsequent calculations.

4. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 3, characterized in that, Construct a condition generator, specifically as follows: Let the attitude noise tensor be... Its elements follow a normal distribution; style vector This indicates the style control conditions for each sample; First, the style vector is copied and expanded to include the style features attached to each node: ； In the formula, For tensor expansion functions, This is the tensor that expands s to each node; Then concatenate it with attitude noise: ； In the formula, This is the concatenated node feature tensor; The concatenated node features are input into the attention-based dynamic graph perceptual computation: ； in, The feature tensor output by the attention-driven dynamic graph perception operation; DGPA is the attention-driven dynamic graph perception operation; the DGPA operation constructs the edge weights between nodes through attention guidance and performs neighborhood information aggregation, outputting the feature representation of each node, which has a dimension of F; Reconstruct and flatten the graph features into the input form of a fully connected network: ； In the formula, For the reconstructed tensor, For tensor reconstruction functions; The pose output vector is generated through a two-layer fully connected network: ； In the formula, Output vector for attitude; , For weight tensors, and It is the offset tensor; The hyperbolic tangent function is used; ReLU is the activation function. The attitude output vector Reconstructed into a 3D coordinate tensor, the final output tensor will be linear: ； In the formula, The generated pose is represented by the three-dimensional coordinates of N joints; With model hyperparameters The condition generator is represented as: ； In the formula, This is a generator function.

5. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 4, characterized in that, Construct a condition recognizer, specifically as follows: Input 3D skeleton data sample With style vectors Perform joint encoding: First, expand the style vector and then concatenate it with the pose: ； ； In the formula, 3D skeleton data sample With extended style features The tensor obtained by splicing; Feature extraction using attention-based dynamic graph perceptual computation: ； In the formula, The feature tensor output by the attention dynamic graph perception operation; Flatten out the reconstructed node features: ； In the formula, For characteristic tensors The tensor obtained after reconstruction; Map it to a shared semantic space through a shared coding layer: ； In the formula, For tensors mapped to a shared semantic space; Calculate the true / false judgment output: ； In the formula, The output is the true / false label value. These are the weight tensor and the offset tensor, respectively. Use the Sigmoid activation function; Calculate style classification output: ； In the formula, Score the style category of the output. These are the weight tensor and the offset tensor, respectively. The resulting condition recognizer is as follows: ； ； In the formula, These are the model hyperparameters of the conditional recognizer; , These are the recognizer functions for authenticity detection and style classification, respectively.

6. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 5, characterized in that, Constructing the pose geometry regularization loss function includes: Spatial bone segment length loss The calculation is as follows: ； in, Represents the set of key connections. , These are the coordinates of the i-th keypoint in the real pose and the generated pose, respectively. This indicates the number of elements in set C; Represents the square of the L2 norm; Bone angle loss The calculation is as follows: ； ； Where T is the set of triplets of the angles of the left and right arms, left and right thighs and calves, spine and head and neck joints, and |T| is the number of joint angles to be calculated; , These represent the joint angles of the generated pose or the actual pose, respectively; >0 represents the included angle tolerance threshold; It is the inverse cosine function; This is the function for finding the maximum value.

7. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 6, characterized in that, Constructing the pose geometry regularization loss function includes: Symmetrical loss of left and right limbs The calculation is as follows: ； in, A set of indices for symmetrical bone segments. The number of elements in the set; Reversing Consistency Loss The calculation is as follows: ； in, For the torsional triplet set of the shoulder and hip, | | represents the number of elements in the set; The L2 norm of a vector; horizontal loss of shoulder or hip line The calculation is as follows: ； in, Represents the shoulder line vector; Represents the hip line vector; To calculate the average value; This indicates the operation of taking a slice of a vector or tensor, that is, taking the elements from the first column to the last column of the vector.

8. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 7, characterized in that, Combining the various loss functions, we obtain the pose geometry regularization loss function. The calculation is as follows: ； In the formula, , , , and These are the weights of their respective loss functions.

9. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 8, characterized in that, The loss function for constructing the condition generator is as follows: Loss function of condition generator The structure is as follows: ； In the formula, Represents the mathematical expectation; The loss is used to generate sample style classification. To generate sample diversity loss; for The weights; for The weights; for The weights; Generated Sample Style Classification Loss Defined as: ； In the formula, This represents the function for calculating cross-entropy; This represents the style label vector corresponding to the real sample; This indicates that after the conditional recognizer outputs, the i-th sample is classified into style. Logical value; Generate sample diversity loss Defined as: ； In the formula, , The data sample sequences generated for the first and second condition generators; Represents the probability distribution function; The sample sequences generated for the first and second iterations both come from the same condition generator and both obey the same condition. The probability distribution function of ; Noise' is another set of random noise with the same dimension and distribution as Noise; >0 represents the allowed difference threshold.

10. The pose generation method based on style-space attention-aware conditional adversarial network according to claim 9, characterized in that, The loss function for constructing the conditional recognizer includes: The loss function of the conditional recognizer is constructed as follows: ； ； In the formula, The loss function is used to train the conditional recognizer. This represents the loss of style recognition capability in the conditional recognizer. for Weighting coefficients; Encode the style category for the i-th sample; For the first The sample at the th Unnormalized scores in terms of style; for The result of exponentiation; During final training, the loss of the condition generator and the recognizer is minimized by alternately optimizing them. Jointly optimize the hyperparameters of both models. and ,Right now: ； In the formula, This represents the operation of taking the value of the independent variable that minimizes the function; the optimization process involves iteratively updating the parameters through adversarial training between the generator and the recognizer. and Until it converges.

Images