Unified autoregressive three-dimensional model automatic skeleton binding method and device

Abstract

The application relates to the technical field of intelligent generation, in particular to a unified autoregressive three-dimensional model automatic skeleton binding method and device, wherein the method comprises the following steps: converting an input three-dimensional model into point cloud data; performing sampling and normalization processing on the point cloud data to extract local and global geometric features of the three-dimensional model, and determining feature embedding of the point cloud data according to the local and global geometric features; based on the feature embedding and a preset autoregressive model, generating a skeleton tree token sequence and a discrete skin token sequence, to generate a skeleton tree and skin weights, and fusing the skeleton tree and the skin weights to obtain an automatic three-dimensional grid binding result suitable for an animation driving condition. Therefore, the problem that animation effect presentation is affected due to the fact that related technologies cannot uniformly model the skeleton and the skin weights and cannot process sparse skin weights in a more stable manner is solved.

Description

Technical Field

[0001] This application relates to the field of intelligent content generation technology, and in particular to a method and apparatus for automatic skeleton binding of a unified autoregressive 3D model. Background Technology

[0002] With the rapid development of 3D content generation and 3D asset creation, massive 3D models need to complete skeleton binding in the production process, that is, to generate skeletal structures and assign skinning weights to mesh vertices.

[0003] In related technologies, traditional manual rigging relies on the experience of professional animators, which is time-consuming and difficult to scale. Existing automatic rigging solutions usually separate bone generation and skin weight prediction into two stages, or treat skin weight as a high-dimensional continuous matrix regression problem.

[0004] However, separating bone generation and skin weight prediction into two stages can lead to the following drawbacks: (1) The skin weight matrix is ​​large and extremely sparse. Direct regression training is easily dominated by zero values, which leads to instability and noise in the non-zero region. (2) Decoupling of skeleton generation and skin prediction means that the skeleton structure cannot be constrained by the "deformation requirement" in reverse, and the error accumulates in the pipeline. (3) When faced with “outdoor models” such as non-closed meshes, broken parts, and complex attachments, unstable geometric descriptions can lead to missing skeletons or leakage of weights, affecting the animation effect.

[0005] Therefore, there is an urgent need for an automatic binding method that can model bones and skin in a unified manner and handle sparse skin weights with a more stable representation. Summary of the Invention

[0006] This application provides a unified autoregressive 3D model automatic skeleton binding method and apparatus to solve the problems caused by the inability of related technologies to model bones and skin weights in a unified manner and to process sparse skin weights with a more stable representation, which in turn affects the presentation of animation effects.

[0007] The first aspect of this application provides a unified autoregressive 3D model automatic skeleton binding method, comprising the following steps: converting the input 3D model into point cloud data; sampling and normalizing the point cloud data to extract local and global geometric features of the 3D model, and determining the feature embedding of the point cloud data based on the local and global geometric features; generating a skeleton tree token sequence and a discrete skinning token sequence based on the feature embedding and a preset autoregressive model to generate a skeleton tree and skinning weights, and fusing the skeleton tree and skinning weights to obtain an automatic 3D mesh binding result suitable for animation-driven conditions.

[0008] Through the above-mentioned technical means, the embodiments of this application can convert the input 3D model into a point cloud, generate feature embedding, and then generate a skeleton tree token sequence and a discrete skin token sequence based on the embedding through a preset autoregressive model. In this way, the skeleton tree and skin weights are generated and fused, thereby realizing end-to-end joint modeling of skeleton and skin. This not only gets rid of the limitations of traditional binding on topological dependence and format adaptation, but also accurately captures the key geometric information of the model through feature embedding and improves the generation efficiency and accuracy through tokenization representation. Finally, it can quickly output high-quality automatic 3D mesh binding results adapted to animation-driven models.

[0009] Optionally, in one embodiment of this application, generating a skeleton tree token sequence and a discrete skinning token sequence based on the feature embedding and a preset autoregressive model to generate a skeleton tree and skinning weights includes: discretizing continuous joints using the skeleton tree token sequence and introducing type identifiers to generate topological information; and generating the skeleton tree with a hierarchical structure based on the topological information.

[0010] Through the above-mentioned technical means, the embodiments of this application can achieve a compact and unified representation of the bone structure by serializing the bone hierarchy structure, uniformly quantizing the joint coordinates and converting them into a discrete integer token sequence. This not only preserves the hierarchical relationship and spatial position information of the bones, but also adapts to the sequence modeling logic of the autoregressive model, laying the foundation for subsequent coherent sequence fusion with the surface skin. At the same time, the discretization process reduces the model's sensitivity to bone geometric noise and improves the stability and generalization of bone structure generation.

[0011] Optionally, in one embodiment of this application, generating a skeleton tree token sequence and a discrete skin token sequence based on the feature embedding and a preset autoregressive model to generate a skeleton tree and skin weights includes: compressing high-dimensional sparse skin weights into discrete skin tokens based on a pre-built finite scalar quantization conditional variational autoencoder; and reconstructing a high-quality skin weight matrix based on the discrete skin tokens and a preset decoder to determine the skin weights.

[0012] Through the above-mentioned technical means, the embodiments of this application can compress high-dimensional sparse skin weights into discrete skin tokens based on a pre-constructed finite scalar quantization conditional variational autoencoder, and then reconstruct a high-quality skin weight matrix through a preset decoder to determine the skin weights. This transforms the originally difficult-to-optimize high-dimensional sparse regression problem into an efficient discrete token processing task, which can achieve compact compression and accurate reconstruction of skin weights, and avoid the noise and redundancy problems caused by direct regression. At the same time, the discretized representation adapts to the sequence modeling logic, providing technical support for joint modeling with skeletal structures.

[0013] Optionally, in one embodiment of this application, it further includes: during the model reinforcement learning stage, generating learning parameters based on the automatic 3D mesh binding results, according to the geometry and deformation rewards; The preset autoregressive model is optimized based on the learning parameters.

[0014] Optionally, in one embodiment of this application, the formula for calculating the deformation reward is: , in Rewards for voxel joint coverage. The weights corresponding to the voxel joint coverage reward. The skeleton-mesh contains rewards. The skeleton-mesh contains the weights corresponding to the rewards. For skin coverage and sparsity rewards The weights corresponding to skin coverage and sparsity rewards. The deformation smoothness reward (based on the distortion metric of LBS deformation under random pose). The weights corresponding to the deformation smoothness reward.

[0015] Through the aforementioned reward constraints, the unified autoregressive model can correct common failure modes (such as missing bones, exposed bones, unbound vertices and excessively dense weights), and output more stable and usable automatic bone binding results.

[0016] Optionally, in one embodiment of this application, the generation conditions of the preset autoregressive model are: , in, For the generated binding token sequence, This is the conditional embedding obtained.

[0017] A second aspect of this application provides a unified autoregressive 3D model automatic skeleton binding device, comprising: a conversion module for converting an input 3D model into point cloud data; an extraction module for sampling and normalizing the point cloud data to extract local and global geometric features of the 3D model, and determining feature embeddings of the point cloud data based on the local and global geometric features; and a binding module for generating a skeleton tree token sequence and a discrete skinning token sequence based on the feature embeddings and a preset autoregressive model to generate a skeleton tree and skinning weights, and fusing the skeleton tree and skinning weights to obtain an automatic 3D mesh binding result suitable for animation-driven conditions.

[0018] Through the above-mentioned technical means, the embodiments of this application can convert the input 3D model into a point cloud, generate feature embedding, and then generate a skeleton tree token sequence and a discrete skin token sequence based on the embedding through a preset autoregressive model. In this way, the skeleton tree and skin weights are generated and fused, thereby realizing end-to-end joint modeling of skeleton and skin. This not only gets rid of the limitations of traditional binding on topological dependence and format adaptation, but also accurately captures the key geometric information of the model through feature embedding and improves the generation efficiency and accuracy through tokenization representation. Finally, it can quickly output high-quality automatic 3D mesh binding results adapted to animation-driven models.

[0019] Optionally, in one embodiment of this application, the binding module includes: a first generation unit, configured to discretize continuous joints using the skeleton tree token sequence and introduce type identifiers to generate topology information; and a second generation unit, configured to generate the skeleton tree with a hierarchical structure based on the topology information.

[0020] Through the above-mentioned technical means, the embodiments of this application can achieve a compact and unified representation of the bone structure by serializing the bone hierarchy structure, uniformly quantizing the joint coordinates and converting them into a discrete integer token sequence. This not only preserves the hierarchical relationship and spatial position information of the bones, but also adapts to the sequence modeling logic of the autoregressive model, laying the foundation for subsequent coherent sequence fusion with the surface skin. At the same time, the discretization process reduces the model's sensitivity to bone geometric noise and improves the stability and generalization of bone structure generation.

[0021] Optionally, in one embodiment of this application, the binding module includes: a compression unit, configured to compress high-dimensional sparse skin weights into discrete skin tokens based on a pre-built finite scalar quantization conditional variational autoencoder; and a determination unit, configured to reconstruct a high-quality skin weight matrix based on the discrete skin tokens and a preset decoder, so as to determine the skin weights.

[0022] Through the above-mentioned technical means, the embodiments of this application can compress high-dimensional sparse skin weights into discrete skin tokens based on a pre-constructed finite scalar quantization conditional variational autoencoder, and then reconstruct a high-quality skin weight matrix through a preset decoder to determine the skin weights. This transforms the originally difficult-to-optimize high-dimensional sparse regression problem into an efficient discrete token processing task, which can achieve compact compression and accurate reconstruction of skin weights, and avoid the noise and redundancy problems caused by direct regression. At the same time, the discretized representation adapts to the sequence modeling logic, providing technical support for joint modeling with skeletal structures.

[0023] Optionally, in one embodiment of this application, the unified autoregressive 3D model automatic skeleton binding device is further used to: generate learning parameters based on the automatic 3D mesh binding results according to the geometry and deformation rewards during the model reinforcement learning stage; and optimize the preset autoregressive model according to the learning parameters.

[0024] Optionally, in one embodiment of this application, the formula for calculating the deformation reward is: , in Rewards for voxel joint coverage. The weights corresponding to the voxel joint coverage reward. The skeleton-mesh contains rewards. The skeleton-mesh contains the weights corresponding to the rewards. For skin coverage and sparsity rewards The weights corresponding to skin coverage and sparsity rewards. The deformation smoothness reward (based on the distortion metric of LBS deformation under random pose). The weights corresponding to the deformation smoothness reward.

[0025] Through the aforementioned reward constraints, the unified autoregressive model can correct common failure modes (such as missing bones, exposed bones, unbound vertices and excessively dense weights), and output more stable and usable automatic bone binding results.

[0026] Optionally, in one embodiment of this application, the generation conditions of the preset autoregressive model are: , in, For the generated binding token sequence, This is the conditional embedding obtained.

[0027] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the unified autoregressive 3D model automatic skeleton binding method as described in the above embodiments.

[0028] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described unified autoregressive 3D model automatic skeleton binding method.

[0029] A fifth aspect of this application provides a computer program product that stores a computer program that, when executed by a processor, implements the above-described unified autoregressive 3D model automatic skeleton binding method.

[0030] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a unified autoregressive 3D model automatic skeleton binding method provided according to an embodiment of this application; Figure 2 This is a schematic diagram of a unified autoregressive 3D model automatic skeleton binding framework according to a specific embodiment of this application; Figure 3 This is a schematic diagram of the structure of the unified autoregressive 3D model automatic skeleton binding device according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0032] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0033] The following describes a unified autoregressive 3D model automatic skeleton binding method and apparatus according to embodiments of this application, with reference to the accompanying drawings. Addressing the issues mentioned in the background art, such as the inability to uniformly model bones and skin weights and the inability to handle sparse skin weights with more stable representations, which negatively impact animation effects, this application provides a unified autoregressive 3D model automatic skeleton binding method. In this method, the input 3D model is first converted into a point cloud representation and sampled and normalized. A geometric encoder is used to extract global and local geometric features as conditional inputs. Subsequently, a unified autoregressive generation model generates a single token sequence containing bone structure parameters and SkinTokens. The bone portion employs a bone tree tokenization scheme to bind continuous joints. This method discretizes and introduces type identifiers to generate a hierarchical skeleton tree. For the skinning part, a finite scalar quantization conditional variational autoencoder (FSQ-CVAE) compresses high-dimensional sparse skin weights into discrete SkinTokens, which are then reconstructed by the decoder to obtain a high-quality skin weight matrix. This transforms skin prediction from a continuous regression problem into a more stable discrete token prediction problem. Furthermore, a reinforcement learning fine-tuning stage based on group relative policy optimization (GRPO) is introduced. Reward constraints such as voxel joint coverage, bone-mesh inclusion, skin coverage and sparsity, and deformation smoothness enhance the generalization ability and robustness to complex out-of-distribution models. This method achieves unified modeling of skeleton generation and skinning binding, obtaining high-precision, low-noise skin weights and more reasonable skeleton structures on large-scale multi-class 3D models, and significantly improving the deformation quality and stability under animation-driven conditions. Therefore, it solves the problems of related technologies that cannot unify the modeling of skeleton and skin weights and cannot handle sparse skin weights with more stable representations, thus affecting the animation effect.

[0034] Specifically, Figure 1 This is a flowchart illustrating an automatic skeleton binding method for a unified autoregressive 3D model provided in an embodiment of this application.

[0035] like Figure 1 As shown, the unified autoregressive 3D model automatic skeleton binding method includes the following steps: In step S101, the input 3D model is converted into point cloud data.

[0036] In the context of animation production, a 3D model refers to a digital virtual entity with three-dimensional spatial attributes (length, width, and height) that is constructed for an animation scene. It serves as the digital prototype for animation characters, scenes, and props, and is also the core foundation for subsequent processes such as rigging, animation, and rendering.

[0037] The core of the 3D model to point cloud conversion operation for bone and skin weight binding scenarios is to generate point cloud data that can accurately reflect the geometric features of the model and adapt to subsequent binding calculations (such as preserving the point density of joint / deformation key areas), rather than ordinary general point cloud conversion.

[0038] In step S102, the point cloud data is sampled and normalized to extract the local and global geometric features of the 3D model, and the feature embedding of the point cloud data is determined based on the local and global geometric features.

[0039] In actual implementation, the embodiments of this application can convert the three-dimensional model to be bound into point cloud data, normalize the point cloud, extract local and global geometric features based on the point cloud data using a geometric encoder, and use them as conditional inputs for generating a unified autoregressive binding token sequence.

[0040] For example, the input 3D mesh model is denoted as ,in, For vertex set, It is a collection of patches. First, point clouds can be obtained by sampling from the mesh surface. and normalize it to a unified coordinate space (such as Furthermore, a geometric encoder can be used to extract local and global features from the point cloud to obtain shape condition embeddings. This serves as the conditional input for generating subsequent token sequences.

[0041] In the process of binding skeleton and skin weights, sampling and normalization of point cloud data can filter redundant noise and unify geometric scales. Based on this, the extracted local (such as joint neighborhood) and global (such as the overall outline of the model) geometric features can be transformed into structured feature embeddings, accurately representing the spatial morphology and topological key information of the model. This provides a highly recognizable quantitative basis for subsequent skin weight allocation and skeleton matching, greatly improving the accuracy and generalization of binding.

[0042] In step S103, based on feature embedding and a preset autoregressive model, a skeleton tree token sequence and a discrete skinning token sequence are generated to generate a skeleton tree and skinning weights. The skeleton tree and skinning weights are then fused to obtain an automatic 3D mesh binding result suitable for animation-driven conditions.

[0043] The embodiments of this application can use a unified autoregressive generative model to generate a binding token sequence. The binding token sequence includes at least a skeleton tree token sequence and a skin token sequence. This is a novel discrete representation of skin weights learned through FSQ-CVAE. This representation can transform the thorny problem of regressing high-dimensional sparse matrices into an easy-to-process token prediction task. Secondly, this representation can support the construction of TokenRig, which is a unified autoregressive Transformer that learns to generate a single, staggered skeleton parameter and its corresponding SkinToken sequence, thereby jointly modeling the entire binding system.

[0044] The discrete and compact nature of SkinTokens allows it to overcome the limitations of separate, multi-stage processes. Based on this, the entire binding system (i.e., skeleton and skin) can be represented as a single, coherent, discrete sequence of tokens. This enables the binding to be described as a unified sequence generation task. For example, this task can be solved using TokenRig (an autoregressive Transformer model), which first generates the complete skeleton and then generates the corresponding skin weights.

[0045] As one possible approach, embodiments of this application can employ a unified autoregressive generation model to sequentially generate a sequence of binding tokens. It meets the following generation conditions:

[0046] Optionally, in one embodiment of this application, a skeleton tree token sequence and a discrete skinning token sequence are generated based on feature embedding and a preset autoregressive model to generate a skeleton tree and skinning weights, including: discretizing continuous joints using the skeleton tree token sequence and introducing type identifiers to generate topological information; and generating a skeleton tree with a hierarchical structure based on the topological information.

[0047] The advantage of the technical solution provided in this application's embodiments stems from a novel coherent sequence representation that jointly captures skeletal structure and surface skin.

[0048] The skeletal portion employs a skeletal tree tokenization scheme: In this embodiment, the skeletal hierarchy can first be serialized, and the joint coordinates can be uniformly quantized and represented as a discrete integer token sequence. For example, continuous joint coordinates can be... Discretize into And introduce bone type identifiers An example of a skeletal token sequence is as follows:

[0049] The skeletal tree structure can then be obtained by decoding the skeletal token sequence. (Including joint sets and parent-child connection relationships), and serve as structural conditions for skin generation, ensuring that the generation result maintains a reasonable hierarchical topology.

[0050] Through the above-mentioned technical means, the embodiments of this application can achieve a compact and unified representation of the bone structure by serializing the bone hierarchy structure, uniformly quantizing the joint coordinates and converting them into a discrete integer token sequence. This not only preserves the hierarchical relationship and spatial position information of the bones, but also adapts to the sequence modeling logic of the autoregressive model, laying the foundation for subsequent coherent sequence fusion with the surface skin. At the same time, the discretization process reduces the model's sensitivity to bone geometric noise and improves the stability and generalization of bone structure generation.

[0051] Optionally, in one embodiment of this application, a skeleton tree token sequence and a discrete skin token sequence are generated based on feature embedding and a preset autoregressive model to generate a skeleton tree and skin weights, including: compressing high-dimensional sparse skin weights into discrete skin tokens based on a pre-built finite scalar quantization conditional variational autoencoder; and reconstructing a high-quality skin weight matrix based on the discrete skin tokens and a preset decoder to determine the skin weights.

[0052] To overcome the technical drawbacks of large and extremely sparse skin weight matrices, which make direct regression training susceptible to zero values, leading to instability and noise in non-zero regions, this solution proposes to compress the skin weights of each individual bone into a compact discrete representation.

[0053] Specifically, the skinned token sequence can be obtained by discretely compressing the skinned weights using a finite scalar quantized conditional variational autoencoder (FSQ-CVAE), or predicted by a unified autoregressive generative model under skeletal tree conditions.

[0054] The discrete compression of skin weights in FSQ-CVAE includes: using the sparse skin weights corresponding to each bone as the reconstruction target of the conditional variational autoencoder, and using point cloud geometric features as conditional information; using finite scalar quantization of continuous latent variables to obtain discrete SkinTokens, and using a combination of binary cross-entropy loss and Dice loss to reconstruct the sparse non-zero weight region, thereby improving reconstruction accuracy and suppressing weight "leakage" artifacts.

[0055] Furthermore, based on the obtained skeletal tree structure and the SkinTokens sequence, the skin weight matrix of each vertex to each bone can be reconstructed using the FSQ-CVAE decoder.

[0056] For example, for a number of bones The model has a skin weight matrix of... Its sparsity makes direct regression unstable. To address this, SkinTokens are introduced, specifically: Using the Finite Scalar Quantization Conditional Variational Autoencoder (FSQCVAE), each bone... weight vector Compressed into discrete token sequences Furthermore, a unified autoregressive model can continue to generate SkinTokens sequences after the skeleton is generated:

[0057] FSQCVAE can be discretely represented using a quantization function: And by the decoder under conditions Reconstruction .

[0058] During training, combined reconstruction loss is used to reinforce sparse non-zero regions:

[0059] This can suppress weight leakage and improve local consistency, and finally, by stitching together the reconstruction results of each bone, the overall skin prediction can be obtained. , and the bone tree Output them together as the result of automatic binding.

[0060] In summary, regarding the generation of discrete skin representation and unified autoregressive binding, this application proposes SkinTokens as a learned discrete compact representation of skin weights. Addressing the issues of high-dimensionality and extreme sparsity of the skin weight matrix, and the noise and weight leakage inherent in direct continuous regression, a Finite Scalar Quantization Conditional Variational Autoencoder (FSQ-CVAE) is employed to compress and encode the skin influence region of each bone, mapping the continuous skin weight distribution to short-sequence discrete tokens. The decoder then reconstructs the skin weight matrix with high fidelity. Furthermore, the discrete coordinate tokens of the bone tree (including type identifiers) and SkinTokens are jointly modeled in the same autoregressive sequence, enabling skin generation under global bone structure conditions. This explicitly learns the dependency between bone arrangement and surface deformation, improving the local consistency and animability of weights in finer parts (such as fingers and attachments).

[0061] Through the above-mentioned technical means, the embodiments of this application can compress high-dimensional sparse skin weights into discrete skin tokens based on a pre-constructed finite scalar quantization conditional variational autoencoder, and then reconstruct a high-quality skin weight matrix through a preset decoder to determine the skin weights. This transforms the originally difficult-to-optimize high-dimensional sparse regression problem into an efficient discrete token processing task, which can achieve compact compression and accurate reconstruction of skin weights, and avoid the noise and redundancy problems caused by direct regression. At the same time, the discretized representation adapts to the sequence modeling logic, providing technical support for joint modeling with skeletal structures.

[0062] Optionally, in one embodiment of this application, the method further includes: during the model reinforcement learning stage, generating learning parameters based on geometric and deformation rewards and the automatic 3D mesh binding results; and optimizing a preset autoregressive model based on the learning parameters.

[0063] To improve the model's generalization ability on complex 3D assets, non-standard structures, and topologically defective meshes, this application embodiment can introduce reinforcement learning fine-tuning based on Group Relative Policy Optimization (GRPO) on the basis of supervised training. This can directly optimize the autoregressive generation strategy with geometry and animation-driven quality as the objective.

[0064] The reinforcement learning phase incorporates several computable rewards, including voxel joint coverage to encourage bones to fully cover the main body and attached structures, bone-mesh containment to suppress bones from protruding from the mesh surface, skinning coverage and sparsity to avoid unbound vertices and over-bound degradation, and deformation smoothness to penalize edge distortion and spike artifacts in random poses.

[0065] Through the above-mentioned technical means, the embodiments of this application can enable the model to automatically correct common failure modes such as bone omission, weight leakage and deformation instability without additional manual annotation of preference data, thereby outputting more stable bone binding results that can be directly used for animation production.

[0066] Optionally, in one embodiment of this application, the formula for calculating the deformation reward is: , in Rewards for voxel joint coverage. The weights corresponding to the voxel joint coverage reward. The skeleton-mesh contains rewards. The skeleton-mesh contains the weights corresponding to the rewards. For skin coverage and sparsity rewards The weights corresponding to skin coverage and sparsity rewards. The deformation smoothness reward (based on the distortion metric of LBS deformation under random pose). The weights corresponding to the deformation smoothness reward.

[0067] When the generated sequence cannot be decoded into a valid binding, .

[0068] To improve the generalization ability of complex out-of-distribution models, GRPO reinforcement learning fine-tuning can be introduced after supervised training. A set of outputs is sampled from the current policy for the same input point cloud, the reward is calculated and relative optimization within the group is performed, as shown in the above formula. The total reward is in a weighted form.

[0069] Through the aforementioned reward constraints, the unified autoregressive model can correct common failure modes (such as missing bones, exposed bones, unbound vertices and excessively dense weights), and output more stable and usable automatic bone binding results.

[0070] In summary, as Figure 2 As shown, the binding of the skeleton to the skin mainly includes three stages: (a) SkinTokens learning and discretization representation; (b) Unified autoregressive skeleton + SkinTokens generation; (c) GRPO-based reinforcement learning optimization.

[0071] In the following stages: (a) the point cloud and its corresponding skin weights are used as input to train FSQ-CVAE to compress the sparse skin weights into discrete SkinTokens; (b) the point cloud geometric embedding is used as a condition to generate a bone tree token sequence and generate a SkinTokens token sequence under its condition, and decode the bone structure and skin weights; (c) the generation result is evaluated for reward and the generation strategy is optimized so that the binding result meets the requirements of geometric consistency and deformation quality.

[0072] According to the unified autoregressive 3D model automatic skeleton binding method proposed in this application, the input 3D model is first converted into a point cloud representation and sampled and normalized. Global and local geometric features are extracted using a geometric encoder as conditional input. Subsequently, a unified autoregressive generation model generates a single token sequence containing skeleton structure parameters and SkinTokens. The skeleton part uses a skeleton tree tokenization scheme to discretize continuous joint coordinates and introduce type identifiers to generate a hierarchical skeleton tree. The skinning part uses a finite scalar quantization conditional variational autoencoder (FSQ-CVAE) to compress high-dimensional sparse skin weights into discrete SkinTokens, and the decoder reconstructs a high-quality skin weight matrix, transforming skin prediction from a continuous regression problem into a more stable discrete token prediction problem. Furthermore, a reinforcement learning fine-tuning stage based on group relative policy optimization (GRPO) is introduced to improve the generalization ability and robustness to out-of-distribution complex models through reward constraints such as voxel joint coverage, skeleton-mesh inclusion, skin coverage and sparsity, and deformation smoothness. This method enables unified modeling of bone generation and skinning binding, achieving high-precision, low-noise skinning weights and more reasonable bone structures on large-scale, multi-class 3D models, and significantly improving the deformation quality and stability under animation-driven conditions. This solves the problems of related technologies that cannot unify bone and skinning weight modeling and cannot handle sparse skinning weights with more stable representations, thus affecting animation presentation.

[0073] The above method is applicable to 3D models with multiple categories and complex topological structures, and can be used for animation driving in scenarios such as games, movies, and virtual humans.

[0074] Next, refer to the appendix. Figure 3 This application describes an automatic skeleton binding device for a unified autoregressive 3D model proposed in its embodiments.

[0075] Figure 3 This is a block diagram of the unified autoregressive 3D model automatic skeleton binding device according to an embodiment of this application.

[0076] like Figure 3 As shown, the unified autoregressive 3D model automatic skeleton binding device 10 includes: a conversion module 100, an extraction module 200, and a binding module 300.

[0077] The conversion module 100 is used to convert the input 3D model into point cloud data.

[0078] The extraction module 200 is used to sample and normalize the point cloud data to extract the local and global geometric features of the 3D model, and to determine the feature embedding of the point cloud data based on the local and global geometric features.

[0079] The binding module 300 is used to generate a skeleton tree token sequence and a discrete skinning token sequence based on feature embedding and a preset autoregressive model, to generate a skeleton tree and skinning weights, and to fuse the skeleton tree and skinning weights to obtain an automatic 3D mesh binding result suitable for animation-driven conditions.

[0080] Optionally, in one embodiment of this application, the binding module 300 includes: first and second generation units; wherein the first generation unit is used to discretize continuous joints using a bone tree token sequence and introduce type identifiers to generate topology information; and the second generation unit is used to generate a bone tree with a hierarchical structure based on the topology information.

[0081] Optionally, in one embodiment of this application, the binding module 300 further includes: a compression unit and a determination unit; wherein, the compression unit is used to compress high-dimensional sparse skin weights into discrete skin tokens based on a pre-built finite scalar quantization conditional variational autoencoder; and the determination unit is used to reconstruct a high-quality skin weight matrix based on the discrete skin tokens and a preset decoder to determine the skin weights.

[0082] Optionally, in one embodiment of this application, the unified autoregressive 3D model automatic skeleton binding device 10 is further used to: generate learning parameters based on geometry and deformation rewards and the automatic 3D mesh binding results during the model reinforcement learning stage; and optimize the preset autoregressive model based on the learning parameters.

[0083] Optionally, in one embodiment of this application, the formula for calculating the deformation reward is: , in Rewards for voxel joint coverage. The weights corresponding to the voxel joint coverage reward. The skeleton-mesh contains rewards. The skeleton-mesh contains the weights corresponding to the rewards. For skin coverage and sparsity rewards The weights corresponding to skin coverage and sparsity rewards. The deformation smoothness reward (based on the distortion metric of LBS deformation under random pose). The weights corresponding to the deformation smoothness reward.

[0084] Optionally, in one embodiment of this application, the preset conditions for generating the autoregressive model are: , in, For the generated binding token sequence, This is the conditional embedding obtained.

[0085] It should be noted that the foregoing explanation of the embodiment of the unified autoregressive 3D model automatic skeleton binding method also applies to the unified autoregressive 3D model automatic skeleton binding device of this embodiment, and will not be repeated here.

[0086] According to the unified autoregressive 3D model automatic skeleton binding device proposed in the embodiments of this application, the input 3D model can first be converted into a point cloud representation and sampled and normalized. Global and local geometric features are extracted using a geometric encoder as conditional input. Subsequently, a unified autoregressive generation model is used to generate a single token sequence containing skeleton structure parameters and SkinTokens. The skeleton part adopts a skeleton tree tokenization scheme to discretize continuous joint coordinates and introduce type identifiers, thereby generating a skeleton tree with a hierarchical structure. The skin part uses a finite scalar quantization conditional variational autoencoder (FSQ-CVAE) to compress high-dimensional sparse skin weights into discrete SkinTokens, and the decoder reconstructs a high-quality skin weight matrix, so that skin prediction is transformed from continuous regression into a more stable discrete token prediction problem. Furthermore, a reinforcement learning fine-tuning stage based on group relative policy optimization (GRPO) is introduced to improve the generalization ability and robustness to out-of-distribution complex models through reward constraints such as voxel joint coverage, skeleton-mesh inclusion, skin coverage and sparsity, and deformation smoothness. This method enables unified modeling of bone generation and skinning binding, achieving high-precision, low-noise skinning weights and more reasonable bone structures on large-scale, multi-class 3D models, and significantly improving the deformation quality and stability under animation-driven conditions. This solves the problems of related technologies that cannot unify bone and skinning weight modeling and cannot handle sparse skinning weights with more stable representations, thus affecting animation presentation.

[0087] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include: The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0088] When the processor 402 executes the program, it implements the unified autoregressive 3D model automatic skeleton binding method provided in the above embodiments.

[0089] Furthermore, electronic devices also include: Communication interface 403 is used for communication between memory 401 and processor 402.

[0090] The memory 401 is used to store computer programs that can run on the processor 402.

[0091] Memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0092] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0093] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0094] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0095] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for automatic skeleton binding of a unified autoregressive 3D model.

[0096] This application also provides a computer program product storing a computer program that, when executed by a processor, implements the above-described unified autoregressive 3D model automatic skeleton binding method.

[0097] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0098] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0099] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0100] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0101] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0102] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0104] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A unified autoregressive 3D model automatic skeleton binding method, characterized in that, Includes the following steps: Convert the input 3D model into point cloud data; The point cloud data is sampled and normalized to extract the local and global geometric features of the 3D model, and the feature embedding of the point cloud data is determined based on the local and global geometric features. Based on the feature embedding and the preset autoregressive model, a skeleton tree token sequence and a discrete skinning token sequence are generated to generate a skeleton tree and skinning weights. The skeleton tree and the skinning weights are then fused to obtain an automatic 3D mesh binding result suitable for animation-driven conditions.

2. The method according to claim 1, characterized in that, The step of generating a skeleton tree token sequence and a discrete skinning token sequence based on the feature embedding and a preset autoregressive model to generate a skeleton tree and skinning weights includes: The continuous joints are discretized using the skeletal tree token sequence, and type identifiers are introduced to generate topological information; The skeletal tree with a hierarchical structure is generated based on the topology information.

3. The method according to claim 1 or 2, characterized in that, The step of generating a skeleton tree token sequence and a discrete skinning token sequence based on the feature embedding and a preset autoregressive model to generate a skeleton tree and skinning weights includes: Based on a pre-built finite scalar quantization conditional variational autoencoder, high-dimensional sparse skin weights are compressed into discrete skin tokens. A high-quality skin weight matrix is ​​reconstructed based on the discrete skin token and the preset decoder to determine the skin weights.

4. The method according to claim 1, characterized in that, Also includes: During the model reinforcement learning phase, learning parameters are generated based on the geometric and deformation rewards and the automatic 3D mesh binding results. The preset autoregressive model is optimized based on the learning parameters.

5. The method according to claim 4, characterized in that, The formula for calculating the deformation bonus is as follows: ， in Rewards for voxel joint coverage. The weights corresponding to the voxel joint coverage reward. The skeleton-mesh contains rewards. The skeleton-mesh contains the weights corresponding to the rewards. For skin coverage and sparsity rewards The weights corresponding to skin coverage and sparsity rewards. As a reward for deformation smoothness, The weight corresponding to the deformation smoothness reward.

6. The method according to claim 1, characterized in that, The conditions for generating the preset autoregressive model are as follows: ， in, For the generated binding token sequence, This is the conditional embedding obtained.

7. An automatic skeleton binding device for a unified autoregressive 3D model, characterized in that, include: The conversion module is used to convert the input 3D model into point cloud data; An extraction module is used to sample and normalize the point cloud data to extract the local and global geometric features of the 3D model, and to determine the feature embedding of the point cloud data based on the local and global geometric features. The binding module is used to generate a skeleton tree token sequence and a discrete skinning token sequence based on the feature embedding and a preset autoregressive model, so as to generate a skeleton tree and skinning weights, and fuse the skeleton tree and the skinning weights to obtain an automatic 3D mesh binding result suitable for animation-driven conditions.

8. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, the processor executing the program to implement the unified autoregressive 3D model automatic skeleton binding method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the unified autoregressive 3D model automatic skeleton binding method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed to implement the unified autoregressive 3D model automatic skeleton binding method as described in any one of claims 1-6.

Images