Virtual image generation method and system

Abstract

The invention relates to the technical field of intelligent interaction, and provides a virtual image generation method and system, which can quickly generate a stylized VPA image with a target head-body ratio and realize automatic modeling by extracting body feature parameters of a body input by a user and combining a diffusion model and a conditional constraint network. The intelligent skeleton binding and self-learning optimization technology is combined, self-learning adapts to the input body, the hierarchical skeleton is automatically generated based on the body structure, a preset template is not needed, the flexibility is high, the expressive force is good, and it is ensured that role actions are natural and smooth. Meanwhile, a personalized adjustment mechanism driven by user interaction is introduced, the intelligence and interactivity of VPA image generation are improved through a closed loop of shape understanding, stylization generation, intelligent binding and continuous personalized optimization, the image is continuously optimized based on user feedback, and then the intelligent virtual image demonstration requirement in the education and entertainment field is met.

Description

Technical Field

[0001] This invention relates to the field of intelligent interactive technology, and in particular to a method and system for generating virtual avatars. Background Technology

[0002] With the widespread application of AI big data models in intelligent interaction and personalized services, VPA (Virtual Personal Assistant) has become an important player in human-computer interaction, and is widely used in fields such as smart cockpits, smart homes, social platforms, and gaming. Currently, VPA avatar generation mainly relies on preset templates or manual design, making it difficult to adapt to the personalized needs of different users and lacking the ability to adaptively adjust to different geometric shapes.

[0003] Current technologies for generating intelligent virtual avatars typically focus on realistic styles or cartoonish figures with fixed proportions. However, adapting and creating cute and adorable designs for non-standard geometric shapes (such as ellipses, triangles, and cones) remains a challenge. In VPA modeling, skeletal rigging is a crucial step, connecting a 3D model to a series of virtual bones that can be manipulated and animated to achieve dynamic effects for characters or objects. Current VPA skeletal rigging methods often require manual adjustments, leading to high development costs and a lack of flexibility. Furthermore, custom bone hierarchies are needed for non-standard body types, potentially disrupting universal rigging templates and impacting the model's animation expressiveness and naturalness. Summary of the Invention

[0004] This invention provides a virtual avatar generation method and system, which solves the technical problems of existing intelligent virtual avatar generation technologies that rely on preset templates or manual design, cannot adapt to non-standard modeling designs, have low flexibility and poor expressiveness when performing skeletal binding, and thus cannot meet the personalized needs of different users.

[0005] To solve the above technical problems, the present invention provides a method for generating virtual images, comprising: The input shape is acquired, and the shape feature parameters of the input shape are identified. Based on the diffusion model, an initial VPA image is constructed using the aforementioned body feature parameters as conditional parameters, and the head-to-body ratio is forcibly constrained according to the conditional constraint network to achieve style transfer and generate a stylized VPA image. Bind the character geometry of the VPA image to the skeleton; The VPA avatar is personalized based on user interaction data.

[0006] This basic solution extracts the shape feature parameters of the user-input form and combines a diffusion model with a conditional constraint network to quickly generate a stylized VPA avatar with a target head-to-body ratio, achieving automatic modeling. By combining intelligent skeletal binding and self-learning optimization technology, it learns to adapt to the input form and automatically generates hierarchical skeletons based on the form's structure, eliminating the need for preset templates. This results in high flexibility, excellent expressiveness, and ensures natural and smooth character movements. Simultaneously, a user-interactive driven personalized adjustment mechanism is introduced. Through a closed loop of form understanding → stylized generation → intelligent binding → continuous personalized optimization, the intelligence and interactivity of VPA avatar generation are improved. The avatar is continuously optimized based on user feedback, thereby meeting the intelligent virtual avatar demonstration needs in the education and entertainment fields.

[0007] In a further implementation, the input shape is acquired, and the shape feature parameters of the input shape are identified, including: Input and Feature Extraction: Obtain point cloud data of the input shape and perform feature extraction based on the backbone network; Task-specific feature generation: Obtain the spatial structural feature representation and global semantic feature vector output by the backbone network; Key point localization based on deformable convolution: The spatial structure feature representation is input into a deformable convolutional network, which outputs a heatmap of each preset key point. Coordinate decoding is applied to the heatmap of each preset key point to map and decode the probability distribution into the three-dimensional coordinates of the corresponding preset key point. Shape feature parameter regression and normalization: The global semantic feature vector and the three-dimensional coordinates of the preset key points are fused, and the initial shape feature parameters are regressed based on the parametric regression network and normalization is performed.

[0008] This solution extracts multi-dimensional features from point cloud data based on the backbone network, ensuring the integrity of the original input shape information. It uses deformable convolutional keypoint localization technology to capture the details of the 3D spatial structure with high precision, laying a reliable foundation for subsequent modeling. Through feature fusion and regression normalization, it outputs standardized and interpretable shape feature parameters. This hierarchical and precise shape feature extraction significantly improves the realism and controllability of virtual character generation, providing high-quality and highly consistent foundational data for downstream tasks such as parameter-based style transfer and skeletal binding.

[0009] In a further implementation, based on a diffusion model, an initial VPA image is constructed using the aforementioned body feature parameters as conditional parameters, and the head-to-body ratio is forcibly constrained according to a conditional constraint network to achieve style transfer and generate a stylized VPA image, including: Conditional diffusion model generation: Construct a point cloud-based conditional diffusion model as a virtual avatar generator G, and generate an initial VPA avatar using the normalized shape feature parameters as conditional parameters. Based on the aforementioned body shape parameters, a precise adjustment of the head-to-body ratio based on parameter constraints is further performed to obtain the intermediate VPA image; Based on generative adversarial networks or diffusion models, intermediate VPA images are mapped to the corresponding design space for style transfer, resulting in stylized VPA images. A deformation algorithm is used to optimize and adjust the geometric proportions of the stylized VPA image.

[0010] This solution ensures that the output VPA image strictly conforms to expectations (e.g., the standard two-head-tall proportion) by combining forced constraints on head-to-body ratio and style transfer. Based on generative adversarial networks or diffusion models, the VPA image is mapped to a pre-defined design space (e.g., a chibi design space), separating form parameters (head-to-body ratio, limb length) from style parameters (color, texture), supporting seamless style switching under a fixed form structure. It can meet the precision requirements of professional fields while also serving the personalized creative needs of general users.

[0011] In a further implementation, based on the aforementioned body shape parameters, precise head-to-body ratio adjustment based on parameter constraints is further performed, including: Construct a conditional constraint network, which includes a lightweight multilayer perceptron or a convolutional network; The shape feature parameters are input into the conditional constraint network to predict the scaling factor and displacement vector field for a predetermined part. Based on the scaling factor and displacement vector field, the corresponding mesh vertices of the initial VPA image are adjusted, and the head-to-body ratio is adjusted to obtain an intermediate VPA image as the input for the next step of style transfer.

[0012] This solution utilizes a conditional constraint network constructed with a lightweight multilayer perceptron or convolutional network to achieve precise and automated control of the head-to-body ratio of the VPA (Virtual Personal Avatar) character. It transforms shape feature parameters into scaling factors and displacement vector fields for specific parts, directly applying them to the vertices of the 3D mesh. The control method is intuitive, allowing for millimeter-level adjustments based on input parameters to ensure the human proportions conform to aesthetic or specific style standards. It boasts high efficiency due to the lightweight network and low computational overhead, enabling rapid generation of intermediate characters that meet requirements. Furthermore, it exhibits low coupling by decoupling proportion control from subsequent style transfer, allowing for independent optimization of both steps and enhancing the overall workflow's flexibility and robustness. This provides a reliable technical foundation for generating high-quality VPA characters with harmonious proportions and diverse styles.

[0013] In a further implementation, the character geometry of the VPA image is bound to the skeleton, including: The motion propagation matrix of each vertex in the above VPA image is calculated based on a graph neural network, and the skeletal structure is automatically generated using an automatic skeleton matching algorithm. The distribution of bone nodes in the skeletal structure is predicted by a hybrid density network, and the position of the bone points is adaptively adjusted according to the input geometric features and shape data. The VPA image is skinned and bound, and the influence weight of each bone node on the surface vertex is calculated to ensure natural deformation of the non-rigid binding.

[0014] This solution calculates the motion propagation matrix of each vertex in the aforementioned VPA avatar based on a graph neural network, and automatically generates the skeletal structure using an automatic skeleton matching algorithm. The motion of each skeletal node propagates according to the relationship between adjacent nodes, optimizing the skeletal binding effect and achieving natural dynamic propagation of the skeleton. Based on the training of the motion propagation matrix, the dynamic behavior of the skeleton can be adjusted according to the structural and motion relationships between nodes, achieving accurate motion transmission and harmonious body proportions, ensuring the natural and smooth movement of the virtual avatar. Traditional skinning and binding methods combined with dynamic weight allocation techniques are employed, calculating the influence weight of each skeletal node on surface vertices to ensure natural and smooth vertex deformation during the virtual avatar animation process.

[0015] In a further implementation, the prediction formula for the skeletal node distribution is as follows:

[0016] In the formula, The input data includes geometric features and shape data; Indicates the given input Under the given conditions, the predicted probability distribution of each skeletal node in the VPA image; Indicates the number of mixture Gaussian distributions; This represents the Gaussian distribution weights at each position of each skeletal node in the VPA image, and the weights of all Gaussian distributions. The sum is 1; The probability density function representing the distribution of each skeletal node in the VPA image; It is the mean of the current Gaussian distribution; It is the covariance matrix, which represents the strength of the relationship or interdependence between nodes.

[0017] This solution uses Gaussian Mixture Model (GMM) to model the skeletal nodes, which can simultaneously predict multiple possible positions of skeletal nodes (such as predicting the number of Gaussian distributions of each skeletal node position in the VPA avatar), rather than forcing a single fixed value to be output. By modeling the skeletal node distribution using MDN, the virtual avatar system can more realistically reflect the inherent uncertainty of biological movement.

[0018] The present invention also provides a virtual image generation system, applied to implement the virtual image generation method described above, comprising: a geometric feature analysis module, used to acquire an input shape and identify the shape feature parameters of the input shape; The image generation module connected to the geometric feature parsing module is used to construct an initial VPA image based on the diffusion model and the shape feature parameters as conditional parameters, and to force the head-to-body ratio according to the conditional constraint network to achieve style transfer and generate a stylized VPA image. The skeletal dynamics binding module, connected to the image generation module, is used to bind the character geometry of the VPA image to the skeleton; The personalization adjustment module, connected to the skeletal dynamics binding module, is used to personalize the VPA image based on user interaction data.

[0019] In a further embodiment, the geometric feature parsing module includes a feature extraction unit, a feature generation unit, a key point recognition unit, and a parameter output unit connected in sequence. The feature extraction unit is used to acquire point cloud data of the input shape and perform feature extraction based on the backbone network; The feature generation unit is used to obtain the spatial structure feature representation and global semantic feature vector output by the backbone network; The key point recognition unit is used to input the spatial structure feature representation into a deformable convolutional network, output a heat map of each preset key point, and apply a coordinate decoding operation to the heat map of each preset key point to map and decode the probability distribution into the three-dimensional coordinates of the corresponding preset key point. The parameter output unit is used to fuse the global semantic feature vector with the three-dimensional coordinates of the preset key points, and regress the initial shape feature parameters based on the parameter regression network and perform normalization processing.

[0020] In a further implementation, the image generation module includes a generator construction unit, a head-to-body ratio constraint unit, and a model output unit connected in sequence. The generator construction unit is used to construct a point cloud-based conditional diffusion model as a virtual image generator G, and generate an initial VPA image using the normalized shape feature parameters as conditional parameters. The head-to-body ratio constraint unit is used to further perform precise adjustment of the head-to-body ratio based on the parameter constraints according to the body feature parameters, so as to obtain the intermediate VPA image; The model output unit is used to map the intermediate VPA image to the corresponding design space for style transfer based on generative adversarial networks or diffusion models, so as to obtain a stylized VPA image; it is also used to optimize and adjust the geometric proportions of the stylized VPA image using deformation algorithms.

[0021] In a further embodiment, the skeletal dynamics binding module includes a GNN unit, a skeletal node distribution unit, and a weight allocation unit connected in sequence. The GNN unit is used to calculate the motion propagation matrix of each vertex in the above VPA image based on the graph neural network, and to automatically generate the skeletal structure using an automatic skeleton matching algorithm. The skeletal node distribution unit is used to predict the distribution of skeletal nodes in the skeletal structure through a hybrid density network, and to adaptively adjust the position of the skeletal points based on the input geometric features and shape data. The weight allocation unit is used to skin and bind the VPA image and calculate the influence weight of each bone node on the surface vertex to ensure natural deformation of non-rigid binding.

[0022] The GNN unit in this scheme utilizes a graph neural network (GNN) to learn the motion propagation matrix between skeletal nodes and generate a skeletal structure that conforms to natural motion based on this matrix. The connection relationships and motion patterns of each skeletal node are automatically calculated to ensure that the generated skeletal structure can adapt to different body features. The skeletal node distribution unit adaptively adjusts the position of skeletal points based on the input geometric features and body data, ensuring the naturalness and stability of the skeletal structure under different postures. The weight allocation unit uses skinning to allow each skeletal node to naturally influence the surrounding mesh vertices during movement, achieving natural flow of the bones during deformation and avoiding unreasonable distortion or stiffness, enabling the VPA image to be animated smoothly. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the process of a virtual avatar generation method provided in an embodiment of the present invention; Figure 2 This is a system framework diagram of a virtual avatar generation system provided in an embodiment of the present invention; The module includes: a geometric feature parsing module 1, a feature extraction unit 11, a feature generation unit 12, a key point recognition unit 13, and a parameter output unit 14; an image generation module 2, a generator construction unit 21, a head-to-body ratio constraint unit 22, and a model output unit 23; a skeletal dynamics binding module 3, a GNN unit 31, a skeletal node distribution unit 32, and a weight allocation unit 33; and a personalized adjustment module 4. Detailed Implementation

[0024] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. The embodiments are given for illustrative purposes only and should not be construed as limiting the present invention. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of patent protection of the present invention, because many changes can be made to the present invention without departing from the spirit and scope of the present invention.

[0025] Example 1 This invention provides a method for generating virtual avatars, such as... Figure 1 As shown, in this embodiment, steps S1 to S4 are included: S1. Obtain the input shape and identify the shape feature parameters of the input shape, including: S11. Input and Feature Extraction: Obtain the point cloud data of the input shape and perform feature extraction based on the backbone network; The backbone network includes a 3D point cloud feature extraction backbone network (e.g., PointNet++). In this embodiment, the input shape is a three-dimensional polygonal mesh or point cloud data; when the input is a polygonal mesh, a preset number of points are sampled from its surface to form point cloud data.

[0026] In this embodiment, the backbone network adopts the feature extraction part of PointNet++, including multiple SetAbstraction (SA) layers, which are used to progressively extract multi-scale features from the point cloud data. The feature map output by the last SA layer of the backbone network serves as a spatial structure feature representation, with dimensions [N, D], where N is the number of point clouds and D is the feature dimension.

[0027] S12. Task-specific feature generation: Obtain the spatial structural feature representation and global semantic feature vector output by the backbone network; Spatial structural feature representation and global semantic feature vector are two different forms of feature representation: The output of the deep network in the backbone network is obtained to obtain the spatial structure feature representation, which preserves the spatial feature map of the spatial correspondence of the input point cloud.

[0028] A global aggregation operation (such as global max pooling) is performed on the output of the deep network of the backbone network (i.e., the spatial structure feature representation) to obtain a global geometric feature vector as a global semantic feature vector, which is used to encode the overall semantic information of the input shape.

[0029] S13. Key point localization based on deformable convolution: Input the spatial feature map in the spatial structure feature representation into the deformable convolutional network, output the heat map of each preset key point, apply coordinate decoding operation (such as soft-argmax) to the heat map of each preset key point, and map and decode the probability distribution into the three-dimensional coordinates of the corresponding preset key point.

[0030] Each heatmap represents the probability distribution of the preset key point in the input shape space; the preset key points include, but are not limited to, the endpoints, peak points, symmetry points, and centroid projection points of the shape.

[0031] By directly mapping the 3D coordinates of preset key points to the geometric structure of the input shape, structured positional information is provided for subsequent feature parameter regression.

[0032] In this embodiment, the deformable convolutional network has three layers, each followed by a ReLU activation function and batch normalization. Based on the deformable convolutional kernel, it adaptively samples the spatial context of the input feature map, progressively mapping high-dimensional features to a probability distribution space corresponding to keypoint locations. Finally, it outputs a heatmap with K channels, each channel corresponding to a preset probability distribution of a keypoint. The deformable convolutional network is trained through supervised learning, and the training data includes the input point cloud and the coordinates of its labeled keypoints.

[0033] S14. Regression and normalization of shape feature parameters: The global semantic feature vector (global geometric feature vector) is fused with the three-dimensional coordinates of the preset key points, and the initial shape feature parameters are regressed based on the parameter regression network and normalization is performed. The normalized shape feature parameters include: scale parameters, shape description parameters, and key position parameters.

[0034] Specifically, the global geometric feature vector and the coordinate information of the three-dimensional coordinates of the preset key points are input into a lightweight parametric regression network to regress the initial shape feature parameters, and the initial shape feature parameters are normalized.

[0035] In this embodiment, the parameter regression network includes a three-layer multilayer perceptron (MLP), with each layer containing 512, 256, and 128 neurons, respectively, and uses the ReLU activation function. Specifically: The key point vector is constructed based on the three-dimensional coordinates of all preset key points, and then fused and concatenated with the global semantic feature vector. The fused vector is then input into the multilayer perceptron (MLP). The multilayer perceptron (MLP) outputs an intermediate feature vector (corresponding to the intermediate feature vector of the shape feature parameters) based on the input fusion vector, and maps and normalizes the intermediate feature vector to the [0,1] interval through the Sigmoid function to obtain the final shape feature parameters.

[0036] In this embodiment, the normalization process refers to ensuring that the shape feature parameters fall within a preset standard value range (e.g., the [0,1] interval or a standardized range based on statistical priors). Normalization aims to eliminate the influence of the input shape scale and ensure that the generated image conforms to the target aesthetic specifications (e.g., a "2-head-tall" proportion).

[0037] In this embodiment, the proportion parameters include, but are not limited to, head-to-body ratio, torso-to-limb length ratio, and shoulder-to-height ratio; the shape description parameters include, but are not limited to, overall height-to-width ratio, contour compactness, and symmetry intensity; and the key position parameters include a normalized set of preset key point coordinates.

[0038] The above-mentioned shape feature parameters—"proportion parameters, shape description parameters, and key position parameters"—will be applied in the subsequent image generation module 2, playing a particularly important role in adjusting the "two-head-tall" proportion and the cute style. The normalized parameters ensure that the generated image meets specific geometric and aesthetic requirements.

[0039] This embodiment extracts multi-dimensional features from point cloud data based on the backbone network, ensuring the integrity of the original input shape information. Deformable convolutional keypoint localization technology is used to capture the details of the 3D spatial structure with high precision, laying a reliable foundation for subsequent modeling. Through feature fusion and regression normalization, standardized and interpretable shape feature parameters are output. This hierarchical and precise shape feature extraction significantly improves the realism and controllability of virtual character generation, providing high-quality and highly consistent foundational data for downstream tasks such as parameter-based style transfer and skeletal binding.

[0040] S2. Based on the diffusion model, an initial VPA image is constructed using the aforementioned body feature parameters as conditional parameters, and the head-to-body ratio is forcibly constrained according to the conditional constraint network to achieve style transfer and generate a stylized VPA image, including: S21. Conditional Diffusion Model Generation: Construct a point cloud-based conditional diffusion model (DiffusionModel) as a virtual avatar generator G, and generate an initial VPA avatar using the normalized shape feature parameters as conditional parameters.

[0041] In this embodiment, the conditional diffusion model includes a denoising network based on a U-Net structure. The normalized shape feature parameters are used as conditional parameters and injected into the intermediate layer of the U-Net structure through a cross-attention mechanism. Combined with a surface reconstruction algorithm, the generation of the virtual image is controlled to obtain the initial VPA image.

[0042] Specifically, the conditional diffusion model learns to generate 3D point clouds that meet conditional parameters from random noise during training. Then, it converts the generated point clouds into triangular mesh representations using surface reconstruction algorithms such as Poisson reconstruction. The diffusion model generates high-fidelity images through progressive denoising, which can delicately depict the texture, lighting, and geometric details of virtual characters.

[0043] S22. Based on the aforementioned body shape parameters, further perform precise head-to-body ratio adjustment based on parameter constraints to obtain the intermediate VPA image: Construct a conditional constraint network, which includes a lightweight multilayer perceptron (MLP) or a convolutional network; The shape feature parameters are input into the conditional constraint network to predict the scaling factor and displacement vector field for a predetermined part. Based on the scaling factor and displacement vector field, the corresponding mesh vertices of the initial VPA image are adjusted, and the head-to-body ratio is adjusted to obtain an intermediate VPA image as the input for the next step of style transfer.

[0044] Specifically, a conditional constraint network is constructed, which is a lightweight multilayer perceptron (MLP) or convolutional network. Its input is the "proportional parameters, shape description parameters, and key position parameters" in step S14 as head-to-body ratio parameter values. The "proportional parameters, shape description parameters, and key position parameters" predict a scaling factor and displacement vector field for a predetermined part (such as head, neck, and torso) based on the head-to-body ratio parameter values, and apply this field to the corresponding mesh vertices of the initial VPA image, thereby achieving precise adjustment of the head-to-body ratio.

[0045] This embodiment achieves precise and automated control of the head-to-body ratio of the VPA (Virtual Personal Image) avatar through a conditional constraint network constructed using a lightweight multilayer perceptron or convolutional network. It transforms shape feature parameters into scaling factors and displacement vector fields for specific parts, directly acting on the vertices of the 3D mesh. The control method is intuitive, allowing for millimeter-level adjustments based on input parameters to ensure the human proportions conform to aesthetic or specific style standards. It boasts high efficiency due to the lightweight network and low computational overhead, enabling rapid generation of intermediate avatars that meet requirements. Furthermore, it exhibits low coupling by decoupling proportion control from subsequent style transfer, allowing for independent optimization of the two steps and enhancing the overall workflow's flexibility and robustness. This provides a reliable technical foundation for generating high-quality VPA avatars with harmonious proportions and diverse styles.

[0046] S23. Based on generative adversarial networks (GANs) or diffusion models, the intermediate VPA image is mapped to the corresponding design space for style transfer, resulting in a stylized VPA image.

[0047] Specifically, the intermediate VPA image after adjusting the head-to-body ratio is mapped to a pre-defined design space (such as the latent space of StyleGAN) that encodes features of a specific art style (such as Q version style) to generate a target VPA image that meets the preset target parameters (such as large eye coefficient > 0.7 and curvature smoothness < 0.3). This step is primarily used to generate the facial and body details of the VPA avatar.

[0048] S24. Use deformation algorithms (such as Morphing) to optimize and adjust the geometric proportions of the VPA image.

[0049] During the image generation process, the deformation algorithm adjusts the geometric proportions according to different body features (such as the shape of the head, torso, and limbs), making the transition of the VPA image smoother and more natural, thus giving the stylized VPA image a natural anthropomorphic appearance.

[0050] This embodiment ensures that the output VPA image strictly conforms to expectations (e.g., the standard two-head-body ratio) by combining forced constraints on head-to-body ratio and style transfer. Based on generative adversarial networks or diffusion models, the VPA image is mapped to a pre-defined design space (e.g., a chibi design space), separating body parameters (head-to-body ratio, limb length) and style parameters (color, texture), supporting seamless style switching under a fixed body structure. It can meet the precision requirements of professional fields while also serving the personalized creative needs of general users.

[0051] S3. Binding the character geometry of the VPA image to the skeleton, including: S31. Calculate the motion propagation matrix of each vertex in the above VPA image based on graph neural network (GNN, used to learn the dependencies between vertices), and automatically generate the skeletal structure using an automatic skeleton matching algorithm; The motion propagation matrix of the vertex is expressed as: , where N is the number of nodes in the skeletal network. The element Wg[i][j] in the matrix represents the connection strength between node i and node j.

[0052] Using a graph neural network (GNN), the system is able to learn the motion propagation matrix between skeletal nodes. And based on this matrix, a skeletal structure that conforms to natural movement is generated.

[0053] For example, in the skeletal binding process of a VPA (Virtual Angle Image), suppose node i represents the elbow, node j represents the shoulder, and node k represents the wrist. The GNN will learn how to reasonably adjust the movements of the shoulder node j and the wrist node k when the movement of the elbow node i changes, thereby achieving natural arm movements.

[0054] S32. Predict the distribution of bone nodes in the skeletal structure using a hybrid density network, and adaptively adjust the position of the bone points based on the input geometric features and shape data; In this embodiment, the prediction formula for the distribution of skeletal nodes is as follows:

[0055] In the formula, Input data typically represents certain features or parameters of a shape; in this embodiment, it includes geometric features and shape data. Indicates the given input Under the given conditions, the predicted probability distribution of each skeletal node in the VPA image; This represents the number of Gaussian mixture distributions, i.e., the number of Gaussian distributions predicting the location of each skeletal node in the VPA image. In this case, It can determine the complexity of the distribution of each skeletal node.

[0056] This represents the weight of the Gaussian distribution at each position of each skeletal node in the VPA image, indicating the importance of that Gaussian distribution in the overall prediction. The weights of all Gaussian distributions are listed below. The sum is 1; It is a Gaussian distribution, representing the probability density function of the distribution of each skeletal node in the VPA image; It is the mean of the current Gaussian distribution (i.e., the position of the skeletal node); It is the covariance matrix, which represents the strength of the relationship or interdependence between nodes.

[0057] These parameters can be used to predict the probability of skeletal nodes in different locations.

[0058] For example, if the upper body skeletal model of a VPA avatar includes three key nodes: head, shoulders, and arms, the system needs to predict the optimal position of the shoulder node during the skeletal binding process. This prediction can be performed using MDN, as follows: A) Input data Input data x includes information such as the relative position of the shoulders and head, and the angle between the shoulders and arms.

[0059] B) Determine the Gaussian distribution of the target location based on the input data x. The MDN model learns multiple Gaussian distribution models to predict the possible location of a target body part (e.g., the shoulder).

[0060] For example, suppose the system discovers through training that the optimal position of a target part (such as the shoulder) can vary within a certain range, then the probability of different shoulder positions is calculated based on multiple Gaussian distribution models.

[0061] C) Determine the weights of the Gaussian distribution. .

[0062] The weights of different Gaussian distributions determine the reliability and importance of each model. For example, assuming that the shoulder distribution may differ under different postures—in some cases the shoulder may be closer to the neck, and in others closer to the chest—MDN will learn the weights of each distribution.

[0063] This solution uses Gaussian Mixture Model (GMM) to model the skeletal nodes, which can simultaneously predict multiple possible positions of skeletal nodes (such as predicting the number of Gaussian distributions of each skeletal node position in the VPA avatar), rather than forcing a single fixed value to be output. By modeling the skeletal node distribution using MDN, the virtual avatar system can more realistically reflect the inherent uncertainty of biological movement.

[0064] S33. Skin the VPA image and calculate the influence weight of each bone node on the surface vertex to ensure natural deformation of the non-rigid binding.

[0065] Weight allocation is typically calculated based on factors such as the distance, angle, and velocity between bones and vertices, and the weights are dynamically adjusted according to the movement of bone nodes. To avoid unnatural deformations, the algorithm also includes optimization steps such as smoothing and weight normalization. This embodiment uses common skinning binding methods such as linear blending skinning, and calculates weights based on the bone structure and vertex-bone relationships generated by this scheme. Specific implementation can employ techniques known in the art.

[0066] This solution calculates the motion propagation matrix of each vertex in the aforementioned VPA avatar based on a graph neural network, and automatically generates the skeletal structure using an automatic skeleton matching algorithm. The motion of each skeletal node propagates according to the relationship between adjacent nodes, optimizing the skeletal binding effect and achieving natural dynamic propagation of the skeleton. Based on the training of the motion propagation matrix, the dynamic behavior of the skeleton can be adjusted according to the structural and motion relationships between nodes, achieving accurate motion transmission and harmonious body proportions, ensuring the natural and smooth movement of the virtual avatar. Traditional skinning and binding methods combined with dynamic weight allocation techniques are employed, calculating the influence weight of each skeletal node on surface vertices to ensure natural and smooth vertex deformation during the virtual avatar animation process.

[0067] S4. Personalize the VPA image based on user interaction data, for example: A) Acquire user interaction data and use reinforcement learning algorithms to adjust and optimize the appearance of the VPA avatar. The adjustments and optimizations include, but are not limited to, the VPA avatar's geometric features, movement, and animation effects.

[0068] Alternatively, B) obtain user interaction data, determine user behavior patterns and preferences, and adjust and optimize the distribution and movement patterns of skeletal nodes based on these behavior patterns and preferences.

[0069] This invention extracts the shape feature parameters of the user-input shape and combines them with a diffusion model and a conditional constraint network to quickly generate a stylized Virtual Personal Image (VPA) with a target head-to-body ratio, achieving automatic modeling. By combining intelligent skeletal binding and self-learning optimization technology, the VPA adapts to the input shape and automatically generates hierarchical skeletons based on the shape structure, eliminating the need for preset templates. This results in high flexibility, excellent expressiveness, and ensures natural and smooth character movements. Simultaneously, a user-interactive driven personalized adjustment mechanism is introduced. Through a closed loop of shape understanding → stylized generation → intelligent binding → continuous personalized optimization, the intelligence and interactivity of VPA generation are improved. The VPA is continuously optimized based on user feedback, thereby meeting the needs of intelligent virtual image demonstrations in education and entertainment fields.

[0070] Example 2 This invention also provides a virtual avatar generation system, applied to implement the virtual avatar generation method described above. See [link to relevant documentation]. Figure 2 ,include: Geometric feature analysis module 1 is used to acquire the input shape and identify the shape feature parameters of the input shape; The image generation module 2, which is connected to the geometric feature analysis module 1, is used to construct an initial VPA image based on the diffusion model and the shape feature parameters as conditional parameters, and to force the head-to-body ratio according to the conditional constraint network to achieve style transfer and generate a stylized VPA image. The skeletal dynamics binding module 3, which is connected to the image generation module 2, is used to bind the character geometry of the VPA image to the skeleton; The personalization adjustment module 4, which is connected to the skeletal dynamics binding module 3, is used to personalize the VPA image based on user interaction data.

[0071] In a further embodiment, the geometric feature parsing module 1 includes a feature extraction unit 11, a feature generation unit 12, a key point recognition unit 13, and a parameter output unit 14 connected in sequence. The feature extraction unit 11 is used to acquire point cloud data of the input shape and perform feature extraction based on the backbone network. The feature generation unit 12 is used to obtain the spatial structural feature representation and global semantic feature vector output by the backbone network; The key point recognition unit 13 is used to input the spatial structure feature representation into a deformable convolutional network, output a heat map of each preset key point, and apply a coordinate decoding operation to the heat map of each preset key point to map and decode the probability distribution into the three-dimensional coordinates of the corresponding preset key point. The parameter output unit 14 is used to fuse the global semantic feature vector with the three-dimensional coordinates of the preset key points, and regress the initial shape feature parameters based on the parameter regression network and perform normalization processing.

[0072] In a further embodiment, the image generation module 2 includes a generator construction unit 21, a head-to-body ratio constraint unit 22, and a model output unit 23 connected in sequence. The generator construction unit 21 is used to construct a point cloud-based conditional diffusion model as a virtual image generator G, and generate an initial VPA image using the normalized shape feature parameters as conditional parameters. The head-to-body ratio constraint unit 22 is used to further perform precise head-to-body ratio control based on parameter constraints according to the body feature parameters to obtain the intermediate VPA image; Model output unit 23 is used to map the intermediate VPA image to the corresponding design space for style transfer based on generative adversarial network or diffusion model to obtain a stylized VPA image; it is also used to optimize and adjust the geometric proportions of the stylized VPA image using deformation algorithm.

[0073] In this embodiment, the skeletal dynamics binding module 3 includes a GNN unit 31, a skeletal node distribution unit 32, and a weight allocation unit 33 connected in sequence. The GNN unit 31 is used to calculate the motion propagation matrix of each vertex in the above VPA image based on the graph neural network, and to automatically generate the skeletal structure using an automatic skeleton matching algorithm. The skeletal node distribution unit 32 is used to predict the distribution of skeletal nodes in the skeletal structure through a hybrid density network, and to adaptively adjust the position of the skeletal points according to the input geometric features and shape data. The weight allocation unit 33 is used to skin and bind the VPA image and calculate the influence weight of each bone node on the surface vertex to ensure natural deformation of non-rigid binding.

[0074] In this embodiment, the GNN unit 31 uses a graph neural network (GNN) to learn the motion propagation matrix between skeletal nodes and generate a skeletal structure that conforms to natural motion based on this matrix. The connection relationships and motion patterns of each skeletal node are automatically calculated to ensure that the generated skeletal structure can adapt to different body features. The skeletal node distribution unit 32 adaptively adjusts the position of skeletal points based on the input geometric features and body data to ensure the naturalness and stability of the skeletal structure under different postures. The weight allocation unit 33 uses skinning to allow each skeletal node to naturally influence the surrounding mesh vertices during movement, achieving natural flow of the bones during deformation, avoiding unreasonable distortion or stiffness, and enabling the VPA image to be animated smoothly.

[0075] This invention creates a virtual avatar generation system. By combining the functions of various modules, it achieves steps S1 to S4 in embodiment 1 above. By recognizing the input shape and adaptively generating a cute VPA smart companion, combined with intelligent skeletal binding and self-learning optimization technology, it can automatically convert into a standard "two-head-tall" cute avatar without manual modeling, thereby improving the intelligence and interactivity of VPA avatar generation. Based on user feedback, it continuously optimizes the avatar to enhance the sense of immersion.

[0076] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for generating virtual avatars, characterized in that, include: The input shape is acquired, and the shape feature parameters of the input shape are identified. Based on the diffusion model, an initial VPA image is constructed using the aforementioned body feature parameters as conditional parameters, and the head-to-body ratio is forcibly constrained according to the conditional constraint network to achieve style transfer and generate a stylized VPA image. Bind the character geometry of the VPA image to the skeleton; The VPA avatar is personalized based on user interaction data.

2. The virtual avatar generation method as described in claim 1, characterized in that, The input shape is acquired, and the shape feature parameters of the input shape are identified, including: Input and Feature Extraction: Obtain point cloud data of the input shape and perform feature extraction based on the backbone network; Task-specific feature generation: Obtain the spatial structural feature representation and global semantic feature vector output by the backbone network; Key point localization based on deformable convolution: The spatial structure feature representation is input into a deformable convolutional network, which outputs a heatmap of each preset key point. Coordinate decoding is applied to the heatmap of each preset key point to map and decode the probability distribution into the three-dimensional coordinates of the corresponding preset key point. Shape feature parameter regression and normalization: The global semantic feature vector and the three-dimensional coordinates of the preset key points are fused, and the initial shape feature parameters are regressed based on the parametric regression network and normalization is performed.

3. The virtual avatar generation method as described in claim 1, characterized in that, Based on the diffusion model, an initial VPA image is constructed using the aforementioned shape feature parameters as conditional parameters. Then, a head-to-body ratio is forcibly constrained according to a conditional constraint network to achieve style transfer and generate a stylized VPA image, including: Conditional diffusion model generation: Construct a point cloud-based conditional diffusion model as a virtual avatar generator G, and generate an initial VPA avatar using the normalized shape feature parameters as conditional parameters. Based on the aforementioned body shape parameters, a precise adjustment of the head-to-body ratio based on parameter constraints is further performed to obtain the intermediate VPA image; Based on generative adversarial networks or diffusion models, intermediate VPA images are mapped to the corresponding design space for style transfer, resulting in stylized VPA images. A deformation algorithm is used to optimize and adjust the geometric proportions of the stylized VPA image.

4. The virtual avatar generation method as described in claim 3, characterized in that, Based on the aforementioned body shape parameters, further precise adjustment of the head-to-body ratio based on parameter constraints is performed, including: Construct a conditional constraint network, which includes a lightweight multilayer perceptron or a convolutional network; The shape feature parameters are input into the conditional constraint network to predict the scaling factor and displacement vector field for a predetermined part. Based on the scaling factor and displacement vector field, the corresponding mesh vertices of the initial VPA image are adjusted, and the head-to-body ratio is adjusted to obtain an intermediate VPA image as the input for the next step of style transfer.

5. The virtual avatar generation method as described in claim 1, characterized in that, Binding the VPA character's geometry to the skeleton includes: The motion propagation matrix of each vertex in the above VPA image is calculated based on a graph neural network, and the skeletal structure is automatically generated using an automatic skeleton matching algorithm. The distribution of bone nodes in the skeletal structure is predicted by a hybrid density network, and the position of the bone points is adaptively adjusted according to the input geometric features and shape data. The VPA image is skinned and bound, and the influence weight of each bone node on the surface vertex is calculated to ensure natural deformation of the non-rigid binding.

6. The virtual avatar generation method as described in claim 5, characterized in that, The prediction formula for the distribution of skeletal nodes is as follows: In the formula, The input data includes geometric features and shape data; Indicates the given input Under the given conditions, the predicted probability distribution of each skeletal node in the VPA image; Indicates the number of mixture Gaussian distributions; This represents the Gaussian distribution weights at each position of each skeletal node in the VPA image, and the weights of all Gaussian distributions. The sum is 1; The probability density function representing the distribution of each skeletal node in the VPA image; It is the mean of the current Gaussian distribution; It is the covariance matrix, which represents the strength of the relationship or interdependence between nodes.

7. A virtual avatar generation system, applied to implement the virtual avatar generation method as described in any one of claims 1 to 6, characterized in that, include: The geometric feature analysis module is used to acquire the input shape and identify the shape feature parameters of the input shape. The image generation module connected to the geometric feature parsing module is used to construct an initial VPA image based on the diffusion model and the shape feature parameters as conditional parameters, and to force the head-to-body ratio according to the conditional constraint network to achieve style transfer and generate a stylized VPA image. The skeletal dynamics binding module, connected to the image generation module, is used to bind the character geometry of the VPA image to the skeleton; The personalization adjustment module, connected to the skeletal dynamics binding module, is used to personalize the VPA image based on user interaction data.

8. The virtual avatar generation system as described in claim 7, characterized in that: The geometric feature parsing module includes a feature extraction unit, a feature generation unit, a key point recognition unit, and a parameter output unit connected in sequence. The feature extraction unit is used to acquire point cloud data of the input shape and perform feature extraction based on the backbone network; The feature generation unit is used to obtain the spatial structure feature representation and global semantic feature vector output by the backbone network; The key point recognition unit is used to input the spatial structure feature representation into a deformable convolutional network, output a heat map of each preset key point, and apply a coordinate decoding operation to the heat map of each preset key point to map and decode the probability distribution into the three-dimensional coordinates of the corresponding preset key point. The parameter output unit is used to fuse the global semantic feature vector with the three-dimensional coordinates of the preset key points, and regress the initial shape feature parameters based on the parameter regression network and perform normalization processing.

9. A virtual avatar generation system as described in claim 8, characterized in that, The image generation module includes a generator construction unit, a head-to-body ratio constraint unit, and a model output unit connected in sequence. The generator construction unit is used to construct a point cloud-based conditional diffusion model as a virtual image generator G, and generate an initial VPA image using the normalized shape feature parameters as conditional parameters. The head-to-body ratio constraint unit is used to further perform precise adjustment of the head-to-body ratio based on the parameter constraints according to the body feature parameters, so as to obtain the intermediate VPA image; The model output unit is used to map the intermediate VPA image to the corresponding design space for style transfer based on generative adversarial networks or diffusion models, so as to obtain a stylized VPA image; it is also used to optimize and adjust the geometric proportions of the stylized VPA image using deformation algorithms.

10. A virtual avatar generation system as described in claim 9, characterized in that: The skeletal dynamics binding module includes a GNN unit, a skeletal node distribution unit, and a weight allocation unit connected in sequence. The GNN unit is used to calculate the motion propagation matrix of each vertex in the above VPA image based on the graph neural network, and to automatically generate the skeletal structure using an automatic skeleton matching algorithm. The skeletal node distribution unit is used to predict the distribution of skeletal nodes in the skeletal structure through a hybrid density network, and to adaptively adjust the position of the skeletal points based on the input geometric features and shape data. The weight allocation unit is used to skin and bind the VPA image and calculate the influence weight of each bone node on the surface vertex to ensure natural deformation of non-rigid binding.

Images