Three-dimensional grid model generation method

Abstract

The invention provides a three-dimensional grid model generation method. The three-dimensional grid model generation method comprises the following steps: converting an obtained three-dimensional grid model sample into a sequence; a neural network used for three-dimensional grid model generation is pre-trained based on the sequence, a pre-trained three-dimensional grid generation model is obtained, the neural network used for three-dimensional grid model generation adopts a mixed linear self-attention mechanism, and the mixed linear self-attention mechanism is that an original self-attention layer is inserted behind a linear self-attention layer; the condition information codes are fused into the sequence to train a pre-trained three-dimensional grid generation model to obtain the three-dimensional grid generation model; and generating a three-dimensional mesh model based on the three-dimensional mesh generation model. And the purposes of improving the training and reasoning efficiency and the surface number of the generated three-dimensional grid are achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional digital models, and in particular relates to a three-dimensional grid model generation method. Background Art

[0002] Currently, creating high-quality three-dimensional (3D) mesh models is a labor-intensive task that requires professional 3D modeling knowledge and a significant investment of time. Especially for complex objects with fine geometric details (such as characters, mechanical parts, etc.), it often takes hours or even days to manually create and optimize the mesh. This greatly limits the efficiency and scale of 3D content creation. Although academia and industry have made significant progress in the field of 3D content generation in recent years, and many methods for automatically generating 3D models have emerged, these methods have low training and inference efficiency, and the generated 3D models have a low number of faces, poor topological structure, and lack effective conditional control, resulting in poor practical usability.

[0003] A three-dimensional mesh is a collection of vertices, edges, and faces, and is one of the most commonly used representation methods for 3D models. Meshes can be directly used for rendering, animation, physical simulation, etc., and are easy to edit and manipulate. Meshes can be divided into triangle meshes and quadrilateral meshes, of which triangle meshes are the most common, and quadrilateral meshes are more popular in certain applications (such as subdivision surface modeling). In recent years, deep learning has made significant progress in the field of 3D generation, and many generation methods based on three-dimensional mesh representation have emerged. The existing technical solutions closest to the present invention mainly include: PolyGen: Models mesh generation as a sequence generation problem, using two autoregressive Transformer models to predict vertex sequences and face sequences, respectively. PolyGen is the first method to directly generate meshes, but the number of generated meshes is low (typically less than 1K faces), and the quality of the generated meshes needs to be improved.

[0004] MeshGPT: Uses VQ-VAE to discretize the mesh and then uses an autoregressive Transformer model to predict the encoded sequence. MeshGPT reduces the sequence length and improves generation efficiency through VQ-VAE, but the number of mesh faces generated is still low (usually less than 1K faces).

[0005] MeshAnything: Based on MeshGPT, it adds point cloud conditions and can generate meshes based on point clouds. MeshAnything v2 uses a lossless mesh compression algorithm to generate meshes with up to 1600 faces.

[0006] MeshXL: Uses a single autoregressive Transformer model to directly generate raw mesh sequences. MeshXL improves the quality of generated meshes by training on a large dataset, but the number of generated meshes is still limited (1600 faces).

[0007] In summary, the inventors discovered during the implementation of this embodiment that the existing method of directly generating a grid has the following main disadvantages: 1. Low training and inference efficiency: Most existing methods use the standard Transformer architecture, whose computational complexity is proportional to the square of the sequence length. For high-polygon meshes, the sequence length of the autoregressive generation process is very long, resulting in low training and inference efficiency.

[0008] 2. Face count limitation: The number of mesh faces generated by existing methods is usually less than 1600, which results in a lack of details in the generated mesh model and cannot meet the requirements of high-quality applications.

[0009] 3. Poor topology quality: There is a difference in topology quality between the mesh models generated by existing methods and the mesh models manually modeled by artists, resulting in poor practical usability.

[0010] 4. Lack of controllability: Existing methods are mostly based on point clouds as conditional input, which has high requirements on the quality of input conditions and is not flexible enough. Summary of the Invention

[0011] In response to the problems existing in the prior art, the present invention provides a three-dimensional mesh model generation method, which at least partially solves the problems existing in the prior art of low training and reasoning efficiency and small number of faces of generated three-dimensional mesh.

[0012] The present disclosure provides a method for generating a three-dimensional mesh model, comprising: Convert the acquired three-dimensional mesh model samples into a sequence; Pre-training a neural network for three-dimensional mesh model generation based on the sequence to obtain a pre-trained three-dimensional mesh generation model, wherein the neural network for three-dimensional mesh model generation adopts a hybrid linear self-attention mechanism, wherein the hybrid linear self-attention mechanism is to insert an original self-attention layer after the linear self-attention layer; The conditional information encoding is integrated into the sequence to train the pre-trained 3D mesh generation model to obtain the 3D mesh generation model; Generate a 3D mesh model based on the 3D mesh generation model.

[0013] Optionally, inserting the original self-attention layer after the linear self-attention layer includes: Insert an original self-attention layer after every four linear self-attention layers; The linear self-attention layer converts the similarity calculation between the query and the key into the inner product of the feature map through the kernel function; The original self-attention layer calculates the dot product between the query and the key and normalizes it to obtain the attention weight.

[0014] Optionally, the linear self-attention layer is calculated as: Attention(Q, K, V) = normalize(φ(K)^T×φ(Q)) ×V, Where: Q, K, V represent query, key, and value matrices respectively; φ(x) represents the feature mapping function; normalize(x) represents the normalization function; T represents the matrix transpose; The calculation formula of the original self-attention layer is: Attention(Q, K, V) = softmax(QK^T ÷sqrt(d_k)) ×V Where: Q, K, V represent the query, key, and value matrices respectively; d_k represents the dimension of the key; softmax(x) represents the softmax normalization function; sqrt(d_k) is used to scale the dot product to prevent gradient vanishing; T represents the matrix transpose.

[0015] Optionally, each linear self-attention layer is followed by a feedforward neural network.

[0016] Optionally, encoding the conditional information into the sequence includes: For point cloud conditions, the geometric features of the point cloud are extracted and mapped to high-dimensional feature vectors, which are regarded as conditional embeddings. For image conditioning, the semantic and structural information of the image is extracted as a feature representation, which is mapped to a conditional embedding that matches the size of the network latent space; For numerical conditions, a multi-layer perceptron is used to map the numerical conditions to conditional embeddings that match the size of the network latent space. The numerical conditions include the number of faces and uniformity. The conditional embeddings obtained from the point cloud condition, image condition, and numerical condition are added to the beginning of the sequence.

[0017] Optionally, the sequence-based pre-training of a neural network for three-dimensional mesh model generation to obtain a pre-trained three-dimensional mesh generation model includes: The cross entropy loss function is used as the pre-training objective. By minimizing the cross entropy loss, the pre-trained 3D mesh generation model learns the set discrete label probability distribution.

[0018] Optionally, the sequence-based pre-training of a neural network for three-dimensional mesh model generation to obtain a pre-trained three-dimensional mesh generation model includes: Using short sequences and first precision quantization to pre-train the neural network for 3D mesh model generation, the neural network learns the basic structure and pattern of the 3D mesh and obtains a basic generative model; A pre-trained three-dimensional mesh generation model is obtained by using a full-length sequence and a second precision quantized pre-trained basic generation model; the second precision is greater than the first precision.

[0019] Optionally, converting the acquired three-dimensional mesh model sample into a sequence includes: Decompose the face of the three-dimensional mesh model into three vertices, each vertex is represented by three floating-point numbers, and the floating-point numbers are quantized into discrete identifiers; Each three-dimensional mesh model is represented as a one-dimensional sequence consisting of 9F discrete identifiers, where F is the number of faces in the model. The faces are sorted in zxy order, that is, first sorted by the z coordinate, then by the y coordinate, and finally by the x coordinate; the sorting of the vertices of the face within its face also follows the same zxy order.

[0020] Optionally, before the step of converting the acquired three-dimensional mesh model sample into a sequence, the step further includes: Data screening is performed on 3D mesh model samples. The screening parameters include: independent connected components, number of faces, coefficient of variation, and the proportion of faces whose minimum edge is smaller than the quantization scale.

[0021] Optionally, converting the acquired three-dimensional mesh model samples into a sequence includes: performing a random transformation on the input three-dimensional mesh model samples, wherein the random transformation includes: randomly rotating the three-dimensional mesh model samples around the z-axis, randomly flipping the three-dimensional mesh model samples along the x-axis or y-axis, and / or randomly scaling the size of the three-dimensional mesh model samples.

[0022] The present invention provides a three-dimensional mesh model generation method. By adopting a hybrid linear self-attention mechanism, the computational complexity is reduced to a value proportional to the sequence length, significantly improving the training and inference efficiency. The hybrid linear self-attention mechanism enables the network to process ultra-long sequences during training, and further enables the generation of a three-dimensional mesh model with up to 50,000 faces during inference, thereby achieving the purpose of improving the training and inference efficiency and the number of faces of the generated three-dimensional mesh.

[0023] In addition, the data screening method for topological structure is used to refine the training data and filter out models with low topological quality, so that the generated results have stable and high topological quality, thereby improving the topological quality of the generated results; multiple generation conditions such as point cloud, image, number of faces, uniformity, etc. are introduced to improve the controllability of the generated results. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0025] Figure 1 A flowchart of a three-dimensional mesh model generation method provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0027] It should be clear that the following embodiments of the present disclosure are described through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0028] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that an aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects described herein can be used to implement an apparatus and / or practice a method. In addition, other structures and / or functionalities other than one or more of the aspects described herein can be used to implement this apparatus and / or practice this method.

[0029] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure and are not drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component can be changed at will, and the component layout type may also be more complicated.

[0030] Additionally, in the following description, specific details are provided to provide a thorough understanding of the examples. However, one skilled in the art will appreciate that the aspects described can be practiced without these specific details.

[0031] For ease of understanding, Figure 1 As shown, this embodiment discloses a method for generating a three-dimensional mesh model, including: Step S101: Screening the data of the 3D mesh model samples. Through the screening rules, this embodiment can effectively filter out three-dimensional mesh models that do not meet the training requirements, ensuring that the generated results have good quality and applicability, thereby improving overall efficiency and accuracy.

[0032] Step S102: converting the acquired three-dimensional mesh model samples into a sequence; To enable neural networks to process 3D mesh data, the 3D mesh model must first be converted into a one-dimensional sequence. Specifically, each face in the 3D mesh model is decomposed into three vertices, each of which is represented as three floating-point numbers (corresponding to the x, y, and z coordinates, respectively). These floating-point numbers are then quantized into Q discrete labels, mapping the continuous coordinate values ​​to a finite discrete space.

[0033] Ultimately, each 3D mesh model is represented as a 1D sequence of 9F discrete identifiers, where F is the number of faces in the model. Faces are sorted in zxy order, first by z coordinate, then by y coordinate, and finally by x coordinate. The vertices within a face also follow the same zxy order.

[0034] Step S103: Pre-training a neural network for three-dimensional mesh model generation based on the sequence to obtain a pre-trained three-dimensional mesh generation model, wherein the neural network for three-dimensional mesh model generation adopts a hybrid linear self-attention mechanism, wherein the hybrid linear self-attention mechanism is to insert an original self-attention layer after the linear self-attention layer; Hybrid linear self-attention mechanism: To address the computational complexity of traditional self-attention mechanisms on long sequences, this embodiment primarily uses a linear self-attention mechanism. This reduces the computational complexity from quadratic to linear in the sequence length, significantly improving the model's processing efficiency.

[0035] To maintain the model's generative capabilities, this example adopts a hybrid strategy, inserting a primitive self-attention layer after every four linear self-attention layers. This hybrid structure maintains efficiency while ensuring that the model can capture complex global dependencies.

[0036] Linear self-attention mechanism formula: The linear self-attention mechanism converts the similarity calculation between query (Q) and key (K) into the inner product of feature maps through the kernel function.

[0037] Step S104: integrating the conditional information encoding into the sequence to train the pre-trained 3D mesh generation model to obtain a 3D mesh generation model; By encoding and incorporating different types of conditional information into the sequence, the model is able to generate corresponding 3D mesh models based on the specified conditions.

[0038] Step S105: Generate a three-dimensional mesh model based on the three-dimensional mesh generation model.

[0039] In a specific application scenario, the 3D mesh model generation method is as follows: Data Filtering: To ensure that the 3D mesh model used for subsequent generation has high quality, this embodiment uses multiple screening rules to filter the model based on the following key indicators: Limitation on the number of independent connected components: Screening rules: Limit the number of independent connected components in the model.

[0040] Reasons for selection: Excessive independent connected components often indicate a model composed of multiple unrelated parts. This can be due to errors in data acquisition or reconstruction, or the model itself is too complex to be easily simplified, analyzed, or rendered. Limiting the number of independent connected components ensures a simple and complete model structure.

[0041] Face number limit: Filtering rules: Set the upper and lower limits of the number of faces.

[0042] Reasons for selection: Too many faces: Models with too many faces require more computation than can be processed during training.

[0043] Too few faces: Models with too few faces lack details and cannot accurately express the geometric features of the object, affecting the generation effect.

[0044] Therefore, a reasonable range of face numbers can balance model accuracy and processing efficiency.

[0045] Coefficient of Variation Limits: Screening rules: Set a lower limit for the coefficient of variation.

[0046] Reason for screening: The coefficient of variation measures the degree of dispersion of the patch size distribution.

[0047] A small coefficient of variation indicates that the mesh size distribution is too uniform, which may indicate that the model is overly regularized and lacks natural details, or that it is automatically generated by a program rather than a real scan or modeling result.

[0048] By setting a lower limit, models with a certain patch size difference can be filtered out, retaining more details and natural features.

[0049] Limit on the proportion of faces whose minimum edge is smaller than the quantization scale: Filtering rules: Set an upper limit on the proportion of faces whose minimum edge is smaller than the quantization scale.

[0050] Reasons for selection: The minimum edge is too small: This indicates that there are a large number of extremely small patches in the model. These patches will produce large errors during the quantization process, significantly reducing the accuracy and stability of the model.

[0051] A high ratio indicates severe quantization error and poor model quality.

[0052] By limiting the proportion, we can effectively remove inferior models caused by quantization errors and improve data quality.

[0053] Through the above-mentioned multiple screening rules, this embodiment can effectively filter out three-dimensional mesh models that do not meet the training requirements, ensuring that the generated results have good quality and applicability, thereby improving overall efficiency and accuracy.

[0054] Network pre-training First, we train an unconditional 3D mesh generation model. The specific steps are as follows.

[0055] Network pre-training process This embodiment adopts a phased pre-training strategy to efficiently train the neural network for 3D mesh model generation. The pre-training process is divided into three main parts: 3D mesh serialization, low-precision cold start, and high-precision training.

[0056] 3D mesh serialization: To enable neural networks to process 3D mesh data, the 3D mesh model must first be converted into a one-dimensional sequence. Specifically, each face in the 3D mesh model is decomposed into three vertices, each of which is represented as three floating-point numbers (corresponding to the x, y, and z coordinates, respectively). These floating-point numbers are then quantized into Q discrete labels, mapping the continuous coordinate values ​​to a finite discrete space.

[0057] Ultimately, each 3D mesh model is represented as a 1D sequence of 9F discrete identifiers, where F is the number of faces in the model. Faces are sorted in zxy order, first by z coordinate, then by y coordinate, and finally by x coordinate. The vertices within a face also follow the same zxy order.

[0058] Through this serialization method, the geometric information of the 3D mesh model is effectively encoded into a one-dimensional sequence, making it able to be processed by sequence models such as Transformer.

[0059] Low-precision cold start: To accelerate network convergence, this example first adopts a low-precision cold start strategy. In this stage, short sequences (less than 1024 in length) and low-precision quantization (Q=128) are used for training.

[0060] Short sequences reduce computational complexity, while low-precision quantization reduces the number of discrete tokens, significantly shortening training time.

[0061] The main goal of this stage is to enable the network to quickly learn the basic structure and pattern of the three-dimensional grid, laying the foundation for subsequent high-precision training.

[0062] High-precision training: After the low-precision cold start phase, this embodiment enters the high-precision training phase, in which full-length sequences (up to 50,000) and high-precision quantization (Q=1024) are used for training.

[0063] The full-length sequences allow the network to learn more complex global structures, while high-precision quantization provides finer geometric details.

[0064] Through high-precision training, the network is able to generate high-quality 3D mesh models with more accurate geometry and richer details.

[0065] Network architecture based on hybrid linear self-attention mechanism This embodiment adopts a neural network structure based on a hybrid linear self-attention mechanism to achieve efficient and high-quality three-dimensional mesh model generation.

[0066] Hybrid linear self-attention mechanism: To address the computational complexity of traditional self-attention mechanisms on long sequences, this embodiment primarily uses a linear self-attention mechanism. This reduces the computational complexity from quadratic to linear in the sequence length, significantly improving the model's processing efficiency.

[0067] To maintain the model's generative capabilities, this example adopts a hybrid strategy, inserting a primitive self-attention layer after every four linear self-attention layers. This hybrid structure maintains efficiency while ensuring that the model can capture complex global dependencies.

[0068] Linear self-attention mechanism formula: The linear self-attention mechanism converts the similarity calculation between query (Q) and key (K) into the inner product of feature maps through the kernel function.

[0069] Specifically, the calculation formula of the linear self-attention mechanism is as follows: Attention(Q, K, V) = normalize(φ(K)^T×φ(Q)) ×V, Where: Q, K, V represent query, key, and value matrices respectively.

[0070] φ(x) represents a feature mapping function, such as ReLU or exp function.

[0071] normalize(x) represents a normalization function, such as softmax or other normalization operations.

[0072] T represents the matrix transpose.

[0073] The key to the linear self-attention mechanism is to use the kernel function φ(x) to map queries and keys into a low-dimensional space, thereby achieving linear complexity.

[0074] Original self-attention mechanism formula: The original self-attention mechanism calculates the dot product between the query (Q) and the key (K) and normalizes it using the softmax function to obtain the attention weight.

[0075] Specifically, the calculation formula of the original self-attention mechanism is as follows: Attention(Q, K, V) = softmax(QK^T ÷sqrt(d_k)) ×V, Where Q, K, and V represent the query, key, and value matrices, respectively. d_k represents the dimension of the key. softmax(x) represents the softmax normalization function. sqrt(d_k) is used to scale the dot product to prevent vanishing gradients. T represents the matrix transpose.

[0076] The main structure of the network: The overall structure of the network is based on the Transformer architecture and consists of multiple stacked hybrid linear self-attention layers.

[0077] Each linear self-attention layer is followed by a feed-forward neural network (FFN) for further feature processing.

[0078] After every four linear self-attention layers, a primitive self-attention layer is inserted to enhance the global modeling ability of the model.

[0079] Through this hybrid structure, this embodiment can achieve high-quality three-dimensional mesh model generation while maintaining efficient calculation.

[0080] Data enhancement: Due to the relatively small amount of training data, this embodiment adopts a data enhancement strategy during the training process to improve the generalization ability of the model. Specifically, the following random transformations are performed on the input 3D mesh model: Randomly rotate around the z-axis: Randomly rotate the model to increase the changes of the model at different angles.

[0081] Random Flip: Randomly flip the model along the x or y axis to increase the variation of the model under different symmetries.

[0082] Random Scaling: Randomly scale the model to increase the model's variability at different scales.

[0083] Loss function: This example uses the cross-entropy loss function, commonly used in language modeling, as the training objective. Cross-entropy loss measures the difference between the probability distribution of discrete tokens predicted by the model and the true distribution. Specifically, for each sequence position, the model predicts a probability distribution of discrete tokens, while the true distribution is a one-hot vector representing the true token at that position. The cross-entropy loss is calculated as follows: L = - (1 / N) * Σ [ Σ (y_i * log(p_i)) ], Where: L represents the cross entropy loss, N represents the length of the sequence, y_i represents the one-hot vector of the true distribution, when the i-th discrete identifier is the true identifier, y_i is 1, otherwise it is 0; p_i represents the probability of the i-th discrete identifier predicted by the model; Σ represents the sum.

[0084] By minimizing the cross-entropy loss, the model is able to learn the correct discrete identity probability distribution and thus generate high-quality 3D mesh models.

[0085] Through the above-mentioned staged pre-training strategy, this embodiment can effectively train a high-performance three-dimensional mesh generation model, thereby achieving unconditional high-quality three-dimensional mesh generation.

[0086] Mixed Conditioning Training: This embodiment adopts a hybrid conditional training strategy to achieve 3D mesh model generation based on multiple conditions. Based on pre-training, this stage encodes different types of conditional information and incorporates them into the sequence, enabling the model to generate the corresponding 3D mesh model based on the specified conditions.

[0087] Point cloud conditional encoding: For point cloud conditions, this embodiment uses a pre-trained 3DShape2VecSet encoder, which can effectively extract the geometric features of the point cloud and map them to a high-dimensional feature vector.

[0088] Specifically, the input point cloud data first passes through the 3DShape2VecSet encoder to obtain a fixed-length feature vector.

[0089] This feature vector is then treated as a conditional embedding to guide the model's generation process.

[0090] Image Conditional Encoding: For image conditions, this embodiment uses a pre-trained DINOv2 encoder. DINOv2 is a self-supervised visual feature extractor that can extract semantic and structural information from images.

[0091] The input image data first passes through the DINOv2 encoder to extract the feature representation of the image.

[0092] This feature representation is then mapped to a conditional embedding that matches the size of the network's latent space.

[0093] Surface number and uniformity condition encoding: For numerical conditions such as the number of faces and uniformity (coefficient of variation), this embodiment uses a multi-layer perceptron (MLP) for encoding.

[0094] The number of faces and uniformity are taken as input and mapped to a feature vector that matches the size of the network's latent space through MLP.

[0095] MLP can learn the nonlinear mapping relationship between numerical conditions and the network latent space, enabling the model to understand and utilize this numerical information.

[0096] Conditional pre-population: In order to integrate condition information into the sequence, this embodiment adopts a conditional pre-filling method.

[0097] Specifically, the conditional embedding obtained by the above encoding is added to the beginning of the sequence as the initial part of the sequence.

[0098] When generating a sequence, the model can first receive conditional information and generate subsequent discrete identifiers based on this information.

[0099] In this way, the model can effectively utilize a variety of conditional information to generate a 3D mesh model that meets the specified requirements.

[0100] Through the above-mentioned mixed condition training strategy, this embodiment can realize the generation of three-dimensional mesh models based on multiple conditions such as point cloud, image, number of faces and uniformity, thereby expanding the application scope and flexibility of the model.

[0101] The method disclosed in this embodiment has the following advantages: 1. Efficient hybrid linear self-attention mechanism to achieve high-polygon model generation: Existing methods based on the original self-attention mechanism face the challenge of quadratic computational complexity when processing long sequences (high-polygon models), resulting in low training and inference efficiency.

[0102] This embodiment adopts a hybrid linear self-attention mechanism to transfer most of the calculations to linear complexity, significantly improving the training and inference efficiency.

[0103] By inserting a primitive self-attention layer every four layers, the proposed method takes into account global modeling capabilities, enabling the model to efficiently process high-polygon 3D mesh models while ensuring generation quality, thus overcoming the model complexity limitations of existing methods. This enables the proposed method to generate more detailed and complex 3D models, meeting the needs of higher-precision applications.

[0104] 2. Refined data screening to ensure high topology quality: Existing methods often ignore the impact of data quality on model generation results, resulting in the generated models containing topological errors, noise or unreasonable geometric features.

[0105] This embodiment effectively filters out low-quality input data by implementing strict, multi-dimensional screening rules, including the number of components, the range of the number of faces, the coefficient of variation, and the proportion of quantization error.

[0106] This refined data screening strategy ensures the purity and high quality of model training data, significantly improving the topological quality and geometric accuracy of the generated models. The generated models are closer to real-world objects, reducing the workload of subsequent repair and optimization.

[0107] 3. Powerful multi-condition generation capability to expand application scope: Existing methods are usually limited to the generation of a single condition or a few conditions, which restricts the application scenarios of the model.

[0108] This embodiment supports mixed input of multiple conditions (such as point clouds, images, number of faces, uniformity, etc.), and effectively integrates different types of information through pre-filling and conditional coding technology.

[0109] This multi-condition generation capability enables the present invention to flexibly generate 3D models that meet specific constraints based on the user's specific needs. For example, a similar 3D model can be generated based on an image, or a complete 3D shape can be reconstructed from a point cloud.

[0110] This flexible multi-conditional control greatly expands the application scope of the model and is suitable for many fields such as design, manufacturing, games, virtual reality, etc.

[0111] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0112] In the present disclosure, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "including," "comprising," "having," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0113] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0114] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0115] Various changes, substitutions, and modifications may be made to the technology described herein without departing from the teachings defined by the appended claims. Moreover, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of things, means, methods, and actions described above. Currently existing or later developed processes, machines, manufactures, compositions of things, means, methods, or actions that perform substantially the same function or achieve substantially the same results as the corresponding aspects described herein may be utilized. Accordingly, the appended claims include within their scope such processes, machines, manufactures, compositions of things, means, methods, or actions.

[0116] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0117] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A three-dimensional mesh model generation method, characterized in that: include: Convert the acquired three-dimensional mesh model samples into a sequence; Pre-training a neural network for three-dimensional mesh model generation based on the sequence to obtain a pre-trained three-dimensional mesh generation model, wherein the neural network for three-dimensional mesh model generation adopts a hybrid linear self-attention mechanism, wherein the hybrid linear self-attention mechanism is to insert an original self-attention layer after the linear self-attention layer; The conditional information encoding is integrated into the sequence to train the pre-trained 3D mesh generation model to obtain the 3D mesh generation model; Generate a 3D mesh model based on the 3D mesh generation model.

2. The three-dimensional mesh model generation method according to claim 1, characterized in that: The inserting of the original self-attention layer after the linear self-attention layer includes: Insert an original self-attention layer after every four linear self-attention layers; The linear self-attention layer converts the similarity calculation between the query and the key into the inner product of the feature map through the kernel function; The original self-attention layer calculates the dot product between the query and the key and normalizes it to obtain the attention weight.

3. The three-dimensional mesh model generation method according to claim 2, characterized in that: The calculation formula of the linear self-attention layer is: Attention(Q, K, V) = normalize(φ(K)^T×φ(Q)) ×V, Where: Q, K, V represent query, key, and value matrices respectively; φ(x) represents the feature mapping function; normalize(x) represents the normalization function; T represents the matrix transpose; The calculation formula of the original self-attention layer is: Attention(Q, K, V) = softmax(QK^T ÷sqrt(d_k)) ×V Where: Q, K, V represent the query, key, and value matrices respectively; d_k represents the dimension of the key; softmax(x) represents the softmax normalization function; sqrt(d_k) is used to scale the dot product to prevent gradient vanishing; T represents the matrix transpose.

4. The three-dimensional mesh model generation method according to claim 1, characterized in that: Each linear self-attention layer is followed by a feed-forward neural network.

5. The three-dimensional mesh model generation method according to claim 1, characterized in that: The encoding of the conditional information into the sequence includes: For point cloud conditions, the geometric features of the point cloud are extracted and mapped to high-dimensional feature vectors, which are regarded as conditional embeddings. For image conditioning, the semantic and structural information of the image is extracted as a feature representation, which is mapped to a conditional embedding that matches the size of the network latent space; For numerical conditions, a multi-layer perceptron is used to map the numerical conditions to conditional embeddings that match the size of the network latent space. The numerical conditions include the number of faces and uniformity. The conditional embeddings obtained from the point cloud condition, image condition, and numerical condition are added to the beginning of the sequence.

6. The three-dimensional mesh model generation method according to claim 1, characterized in that: The sequence-based pre-training of a neural network for three-dimensional mesh model generation to obtain a pre-trained three-dimensional mesh generation model includes: The cross entropy loss function is used as the pre-training objective. By minimizing the cross entropy loss, the pre-trained 3D mesh generation model learns the set discrete label probability distribution.

7. The three-dimensional mesh model generation method according to claim 1, characterized in that: The sequence-based pre-training of a neural network for three-dimensional mesh model generation to obtain a pre-trained three-dimensional mesh generation model includes: Using short sequences and first precision quantization to pre-train the neural network for 3D mesh model generation, the neural network learns the basic structure and pattern of the 3D mesh and obtains a basic generative model; A pre-trained three-dimensional mesh generation model is obtained by using a full-length sequence and a second precision quantized pre-trained basic generation model; the second precision is greater than the first precision.

8. The three-dimensional mesh model generation method according to claim 1, characterized in that: The step of converting the acquired three-dimensional mesh model sample into a sequence includes: Decompose the face of the three-dimensional mesh model into three vertices, each vertex is represented by three floating-point numbers, and the floating-point numbers are quantized into discrete identifiers; Each three-dimensional mesh model is represented as a one-dimensional sequence consisting of 9F discrete identifiers, where F is the number of faces in the model. The faces are sorted in zxy order, that is, first sorted by the z coordinate, then by the y coordinate, and finally by the x coordinate; the sorting of the vertices of the face within its face also follows the same zxy order.

9. The three-dimensional mesh model generation method according to claim 1, characterized in that: The step of converting the acquired three-dimensional mesh model sample into a sequence also includes: Data screening is performed on 3D mesh model samples. The screening parameters include: independent connected components, number of faces, coefficient of variation, and the proportion of faces whose minimum edge is smaller than the quantization scale.

10. The three-dimensional mesh model generation method according to claim 1, characterized in that: The converting of the acquired three-dimensional mesh model samples into a sequence includes: performing a random transformation on the input three-dimensional mesh model samples, wherein the random transformation includes: randomly rotating the three-dimensional mesh model samples around the z-axis, randomly flipping the three-dimensional mesh model samples along the x-axis or y-axis, and / or randomly scaling the size of the three-dimensional mesh model samples.

Images