General three-dimensional shape repairing method and device based on efficient hierarchical generation model
By building a unified data set and designing a hierarchical latent variable diffusion model, the problem of poor repair of multiple types of defects in the existing technology is solved, efficient and robust three-dimensional shape repair is achieved, and the generalization ability and repair quality of the model are improved.
Patent Information
- Application Number
- CN202510414176.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-03
- Publication Date
- 2025-07-11
AI Technical Summary
The existing three-dimensional shape repair technology is limited in handling multiple defect types, and the diversity of training data is insufficient, which limits the generalization ability of the model.
Build a unified three-dimensional shape repair dataset, design an efficient and noise-robust hierarchical latent variable diffusion model, and combine it with scalable training strategies to achieve high-quality repair of multiple types of defects through multi-scale latent variable coding mechanism and pre-compression strategy.
It improves the generalization ability and repair effect of the model, enhances the robustness of irregular noise, ensures high fidelity and comprehensiveness of the repair results, and optimizes training efficiency.
Smart Images
Figure CN120298636A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of three-dimensional shape repair, and particularly relates to a general three-dimensional shape repair method and device based on an efficient hierarchical generation model. Background Art
[0002] In the existing three-dimensional shape repair technologies, existing methods mostly focus on specific shape repair tasks, such as shape completion, super-resolution, or denoising, and have achieved remarkable results on a single repair target using regression models or generation models. However, these methods are limited in dealing with shapes with multiple types of defects simultaneously, and the diversity of training data is insufficient, which limits the generalization ability of the models. Summary of the Invention
[0003] The present invention aims at the deficiencies of the existing technology in general three-dimensional shape repair, and proposes a general three-dimensional shape repair method and device based on an efficient hierarchical generation model. This method realizes high-quality repair of three-dimensional shapes with multiple types of defects, improves the generalization ability of the model and the actual application effect by constructing a unified repair data set, designing an efficient and noise-robust repair model, and combining an extensible training strategy.
[0004] The object of the present invention is achieved by the following technical solutions: A three-dimensional shape repair method based on an efficient hierarchical generation model, the method comprising:
[0005] (1) Constructing a unified three-dimensional shape repair data set, the data set obtaining defect-complete shape pairs by generating various simulated defects for different repair tasks;
[0006] (2) Designing and training a hierarchical latent variable diffusion model, encoding the defective shape based on a multi-scale latent variable encoding mechanism, and gradually generating the complete shape by using a hierarchical diffusion process to realize three-dimensional shape repair;
[0007] (3) Based on the defective shape input to the model, adopting a pre-compression strategy to compress the defective shape features into a unified feature space, and enhancing the noise robustness of the defective shape through a feature alignment strategy;
[0008] (4) Using the trained model to repair the defective shape and generate the corresponding complete shape.
[0009] Further, in step (1), the construction process of the data set includes: for different repair tasks of completion, super-resolution, and denoising, adopting corresponding defect generation methods; generating training samples with various defect morphologies by adding simulated noise, missing parts, or reducing the resolution to the original shape data.
[0010] Further, in step (2), the specific design of the model includes: adopting a multi-scale latent variable encoding mechanism to encode the defect shape and capture geometric features at different levels; using a hierarchical diffusion process to gradually generate a complete shape, from a rough contour to detailed features, to achieve 3D shape repair.
[0011] Further, in step (3), the pre-compression and feature alignment strategy specifically includes: learning a unified feature representation on a large-scale complete shape data; training a defect shape encoder so that its output features are aligned with the feature representation of the complete shape.
[0012] Further, in step (4), the repair process further includes: based on the compressed defect shape features, inputting them into the hierarchical latent variable diffusion model to generate a complete shape layer by layer; calculating the loss by comparing the generated result with the features of the original defect shape and backpropagating to optimize the model parameters.
[0013] The model generates the complete shape layer by layer, ensuring that the repair result achieves high-quality shape restoration while retaining details.
[0014] In a second aspect, the present invention also provides a 3D shape repair device based on an efficient hierarchical diffusion model, including a memory and one or more processors. An executable code is stored in the memory. When the processor executes the executable code, the above-mentioned 3D shape repair method based on the efficient hierarchical diffusion model is implemented.
[0015] In a third aspect, the present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the above-mentioned 3D shape repair method based on the efficient hierarchical diffusion model is implemented.
[0016] In a fourth aspect, the present invention also provides a computer program product, including computer programs / instructions. When the computer programs / instructions are executed by a processor, the above-mentioned 3D shape repair method based on the efficient hierarchical diffusion model is implemented.
[0017] Advantages of the present invention: The present invention proposes a general three-dimensional shape repair method based on an efficient hierarchical generation model, which effectively realizes the unified repair of various defect types of shapes. By constructing a large-scale, real and diverse defect data set, the generalization ability of the model in different repair tasks is improved; by designing a unified defect-complete shape feature alignment mechanism, the robustness of the model to irregular noise is enhanced, supporting the understanding and high-quality repair of complex defect shapes, improving the comprehensiveness and accuracy of the repair, and ensuring the high fidelity of the repair results; at the same time, the proposed pre-compression strategy optimizes the training efficiency and reduces the impact of high-resolution data on the training speed. Through the present invention, the unified repair of various types of defects can be realized, the repair quality and application range of three-dimensional shapes are improved, and the actual needs of multiple fields such as virtual reality, robotics, and content generation are met. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] Figure 1 FIG. is a flowchart of a general three-dimensional shape repair method based on an efficient hierarchical generation model provided by the present invention.
[0019] Figure 2 FIG. is a schematic diagram of an example of a large-scale defect-complete shape pair data set constructed by the present invention.
[0020] Figure 3 FIG. is a training schematic diagram of obtaining a complete shape from a defective shape through an efficient hierarchical generation model according to the present invention.
[0021] Figure 4 FIG. is a structural diagram of a general three-dimensional shape repair device based on an efficient hierarchical generation model according to the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0022] The following further describes the technical details and principles of the present invention with reference to the accompanying drawings:
[0023] As Figure 1 shown, the present invention proposes a general three-dimensional shape repair method based on an efficient hierarchical generation model. The method includes:
[0024] (1) Data set construction: For different repair subtasks, a large-scale defect-complete shape pair data set is constructed using a customized synthesis method. This data set realistically simulates various shape defects in actual scenarios, covering various forms such as incomplete, noise-polluted, and low-resolution, ensuring that the model has good generalization ability;
[0025] (2) Model Design and Training: An efficient and noise-robust hierarchical latent variable diffusion model was designed, which achieved multi-scale encoding of defect shapes and efficient generation of complete shapes. A unified defect-complete shape representation method was proposed to address the irregular noise problem in defect shapes. First, shape feature representations were learned from a large-scale complete shape dataset, and then a defect shape encoder was trained to align its encoding results with the corresponding complete shape features. Through this feature alignment step, pre-compression and denoising enhancement of defect shapes were achieved, thereby improving the model's noise robustness and training efficiency. Specifically as follows:
[0026] Hierarchical Noise-Robust Encoder: Specifically, a hierarchical noise-robust encoder was used to extract multi-scale features from defect shapes. The encoder adopted a block structure, dividing the 3D shape into multiple blocks and extracting their local features respectively to ensure good robustness in the face of complex defects (such as noise and incompleteness). The encoded high-resolution feature grid was used for the subsequent conditional generation stage.
[0027] Hierarchical Latent Variable Diffusion Model: Specifically, in this stage, a hierarchical latent variable diffusion model was used to achieve layer-by-layer reconstruction of shapes. The model received conditional inputs and noise data in each layer and used a sparse network structure for feature extraction and fusion. At low-resolution conditions, the model generated a preliminary sparse feature grid and gradually decoded it through high-resolution conditions to enhance the shape details. Through the multi-level sparse network structure, the model effectively retained the geometric features of the original shape while achieving high-quality reconstruction of the defect area. Finally, the sparse feature grid output by the model was converted by a decoder to generate a complete 3D shape, realizing high-fidelity repair of defects.
[0028] (3) Training Strategy: Since the input of high-resolution defect shapes would significantly reduce the training speed, a pre-compression strategy for defect shapes was proposed. First, the defect shapes were compressed into a unified feature space and then input into the model for training, effectively improving the training efficiency. During model training, the alignment features between defect shapes and complete shapes were used to enhance the model's ability to retain details and ensure the high-fidelity of the repair results.
[0029] As Figure 2 shown, the present invention proposed to construct a large-scale defect-complete shape pair dataset. For different shape repair sub-tasks (such as completion, super-resolution, and denoising), diverse defect simulation methods were adopted to generate a large number of defect-complete shape pairs, ensuring that the dataset covered various real defect types and improving the model's generalization ability. The specific process is as follows:
[0030] 1. Simulation of Defect Shapes:
[0031] Noise-free completion: Randomly sample a part of the complete shape to generate defective samples with different degrees of incompleteness.
[0032] Super-resolution: Downsample the complete shape to generate low-quality shapes with different resolutions for training the model's ability to restore high-resolution shapes.
[0033] Noise completion: Render and fuse depth maps through sparse viewpoints, introducing different degrees of noise to simulate defective data obtained by sensors in reality.
[0034] Noise refinement: Collect more viewpoint data and fuse them to generate complete but noisy shape samples for training the model to eliminate surface noise.
[0035] 2. Unified signed distance field grid representation: All defective shapes and complete shapes are represented using a high-resolution truncated signed distance field (TSDF) to ensure data consistency and the generalization ability of the model.
[0036] 3. Diverse data augmentation: By diversifying the settings of viewpoints, resolutions, and defect degrees, the diversity of data is enhanced, and the model's adaptability to complex defective scenarios is strengthened.
[0037] As Figure 3 shown, the proposed efficient hierarchical generation model in the present invention efficiently encodes defective shapes through a multi-scale latent variable encoder and generates complete shapes layer by layer using a hierarchical latent variable diffusion model, ensuring high-quality shape restoration while retaining details in the repair results. The specific process of shape repair is as follows:
[0038] 1. Probabilistic modeling and block coding:
[0039] A cascaded variational autoencoder is used to jointly learn multi-level latent variable representations, simplifying the training process and enhancing the dependencies between levels. The hierarchical variational autoencoder consists of multiple layers of encoders E = {E 1 , …, E L} and decoders D = {D 1 , …, D L}, where E L and D L represent the encoder and decoder of the L-th layer respectively; the encoder learns the approximate posterior probability through the formula q φ (z|x) = ∏ i q φ (z i |z ＜i , x), where x represents the input complete shape, and the latent variable z i of the current i-th layer depends on the latent variable z of the previous layer＜i The decoder learns the likelihood function p ψ (x|z) = ∏ i p ψ (x i |z ≥i ), indicating that the decoding of each layer depends on the latent variables of the current layer and higher layers. x i represents the shape decoded at the i-th layer. Meanwhile, to enhance the generalization ability of the encoder for complete and defective shapes, a block coding strategy is adopted. The three-dimensional shape is divided into multiple non-overlapping blocks, and feature extraction is performed separately through a hierarchical encoder. The block feature encodings of each layer are concatenated to form the latent representation of the entire shape, thereby reducing the risk of anomalies caused by noise. The decoders of different layers are not restricted by the block structure during the decoding process, avoiding the problem of inconsistent boundaries between blocks.
[0040] 2. Multi-scale feature extraction of defective shapes: Through a hierarchical noise-robust encoder, the defective shape is encoded into a multi-level sparse feature grid which is the L-th layer sparse feature grid of the encoding, extracting key geometric and semantic information to ensure the robustness of the features to defective noise. To improve the encoding quality, first use a frozen complete shape to train a basic hierarchical variational autoencoder to encode the complete shape z = {z 1 , …, z L}, where z L is the complete shape encoded by the L-th layer variational autoencoder, and then the defective shape encoder is refined and fine-tuned through a feature alignment strategy.
[0041] 3. Efficient hierarchical diffusion generation: Using a hierarchical latent variable diffusion model, conditioned on the sparse features of the defective shape, a sparse latent variable grid of the complete shape is generated layer by layer from low resolution to high resolution. At each layer, the model uses a multi-layer sparse U-Net and follows the following probability distribution formula to gradually recover finer shape details, ensuring the consistency and high quality of the generated results at each level.
[0042]
[0043] where c is an optional condition other than the defective shape, such as a class label or an image.
[0044] 4. Rendering and Loss Optimization: Compare the generated complete shape with the original defective shape to calculate the pixel-level and feature-level loss values. Use the backpropagation algorithm to optimize the model parameters layer by layer to ensure that the repaired result is highly consistent with the original shape in terms of details and structure. The latent variables of all levels of the hierarchical variational autoencoder are jointly trained and optimized using the standard ELBO objective function. The hierarchical latent variable diffusion model is trained using a continuous-time diffusion model, combined with v-parameterization and a simplified training objective function. For the optimization of the noise-robust encoder, use the AlignmentLoss function to fine-tune the noise-robust encoder to ensure that the multi-level encoding of the defective shape is aligned with the complete shape. The formula is as follows:
[0045]
[0046] where represents the expected value of the distribution q. Here, q is parameterized by φ c to parameterize.
[0047] Corresponding to the embodiments of the general three-dimensional shape repair method based on the efficient hierarchical generation model described above, the present invention also provides embodiments of a general three-dimensional shape repair device based on the efficient hierarchical generation model. Refer to Figure 4 For the general three-dimensional shape repair device based on the efficient hierarchical generation model provided by the embodiments of the present invention, it includes a memory and one or more processors. An executable code is stored in the memory. When the processor executes the executable code, it is used to implement the general three-dimensional shape repair method in the above embodiments.
[0048] The embodiments of the general three-dimensional shape repair device based on the efficient hierarchical generation model provided by the present invention can be applied to any device with data processing capabilities. This device with data processing capabilities can be a device or apparatus such as a computer. The device embodiments can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by the processor of any device with data processing capabilities reading the corresponding computer program instructions in the non-volatile memory into the memory for operation. From the hardware level, as Figure 4 shown, it is a hardware structure diagram of any device with data processing capabilities where the general three-dimensional shape repair device based on the efficient hierarchical generation model provided by the present invention is located. In addition to Figure 4 the processors, memory, network interfaces, and non-volatile memories shown, the device where the embodiments are located in any device with data processing capabilities usually includes other hardware according to the actual functions of the device with data processing capabilities, which will not be elaborated here.
[0049] For the implementation processes of the functions and roles of each unit in the above device, please refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated here.
[0050] For the device embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The device embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative efforts.
[0051] The embodiment of the present invention also provides a computer-readable storage medium, on which a program is stored. When the program is executed by a processor, a general three-dimensional shape repair method based on an efficient hierarchical generation model in the above embodiment is implemented.
[0052] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the foregoing embodiments, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a Smart Media Card (SMC), an SD card, a Flash Card, etc. equipped on the device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store the data that has been output or will be output.
[0053] The present invention also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, a general three-dimensional shape repair method based on an efficient hierarchical generation model is implemented.
[0054] The above embodiments are used to explain the present invention rather than limit the present invention. Any modifications and changes made within the spirit and scope of the protection of the present invention fall within the protection scope of the present invention.
Claims
1. A general three-dimensional shape repair method based on an efficient hierarchical generation model, characterized in that The method includes: (1) Construct a unified 3D shape repair dataset, which obtains defect-complete shape pairs by generating various simulated defects for different repair tasks; (2) Design and train a hierarchical latent variable diffusion model, encode the defective shape based on the multi-scale latent variable encoding mechanism, and gradually generate the complete shape using the hierarchical diffusion process to achieve 3D shape repair; (3) Based on the defective shape input to the model, adopt a pre-compression strategy to compress the defective shape features into a unified feature space, and enhance the noise robustness of the defective shape through a feature alignment strategy; (4) Use the trained model to repair the defective shape and generate the corresponding complete shape.
2. The general three-dimensional shape repair method based on an efficient hierarchical generation model according to claim 1, characterized in that In step (1), the construction process of the dataset includes: adopting corresponding defect generation methods for different repair tasks of completion, super-resolution, and denoising; generating training samples with various defect morphologies by adding simulated noise, missing parts, or reducing the resolution to the original shape data.
3. A general three-dimensional shape repair method based on an efficient hierarchical generation model according to claim 1, characterized in that In step (2), the specific design of the model includes: adopting a multi-scale latent variable encoding mechanism to encode the defective shape and capture geometric features at different levels; using the hierarchical diffusion process to gradually generate the complete shape from the rough contour to the detailed features to achieve 3D shape repair.
4. A general three-dimensional shape repair method based on an efficient hierarchical generation model according to claim 1, characterized in that, In step (3), the pre-compression and feature alignment strategies specifically include: learning a unified feature representation on a large-scale complete shape data; training a defective shape encoder to align its output features with the feature representation of the complete shape.
5. A general three-dimensional shape repair method based on an efficient hierarchical generation model according to claim 1, characterized in that, In step (4), the repair process further includes: based on the compressed defective shape features, input them into the hierarchical latent variable diffusion model to generate the complete shape layer by layer; calculate the loss by comparing the features of the generated result with the original defective shape and backpropagate to optimize the model parameters.
6. A three-dimensional shape repair device based on an efficient hierarchical generation model, comprising a memory and one or more processors, wherein executable code is stored in the memory, characterized in that, When the processor executes the executable code, it implements the 3D shape repair method based on the efficient hierarchical generation model according to any one of claims 1 to 5.
7. A computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, it implements the 3D shape repair method based on the efficient hierarchical generation model according to any one of claims 1 to 5.
8. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, it implements the 3D shape repair method based on the efficient hierarchical generation model according to any one of claims 1 to 5.