Zero-shot relief monocular 3D reconstruction method and system

By employing a diffusion refinement method with dual-teacher pseudo-supervision and weighted noise injection, the problem of synchronous restoration of global geometry and local texture in the 3D reconstruction of a single relief image is solved, achieving efficient and automated 3D reconstruction of reliefs. This method is applicable to multi-source heterogeneous relief scenarios and meets the needs of digital protection of cultural heritage.

CN122223083BActive Publication Date: 2026-11-10UNIV OF SCI & TECH BEIJING
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202610333846.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-03-18
Publication Date
2026-11-10
Estimated Expiration
2046-03-18

AI Technical Summary

Technical Problem

Existing technologies cannot get rid of the dependence on physical entities or depth truth in the 3D reconstruction of single relief images. They are difficult to ensure global geometric rationality and local texture authenticity simultaneously under zero-sample conditions, and their cross-style generalization ability is weak, which cannot meet the needs of digital protection of cultural heritage.

Method used

A dual-teacher pseudo-supervision mechanism and a diffusion refinement method with weighted noise injection are adopted. A consensus depth map is generated through a multi-architecture pre-trained model. Noise injection and spatial awareness loss function optimization are combined with edge prior maps to achieve targeted enhancement of high-frequency details of relief and constraint of global geometric structure.

Benefits of technology

It achieves high-precision, fully automated 3D reconstruction of reliefs, reduces digitization costs, adapts to multi-source heterogeneous relief scenarios, has excellent zero-sample cross-domain generalization capabilities, and meets the dual requirements of cultural heritage digitization for craft texture and geometric accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122223083B_ABST
    Figure CN122223083B_ABST
Patent Text Reader

Abstract

The application discloses a zero-sample relief monocular three-dimensional reconstruction method and system, comprising the following steps: constructing a double-teacher pseudo-supervised signal, generating a consensus depth map through pixel-level median fusion of multiple pre-trained depth models to guarantee the global geometric structure, and combining an edge prior map generated by surface normal mapping to locate a micro-carving area; in the diffusion refinement stage, constructing a spatial self-adaptive noise intensity mask according to the edge prior, implementing a weighted noise injection strategy, and guiding a denoising network to directionally enhance the details of the texture area; and designing a spatial perception loss function to dynamically balance the depth consistency and edge detail recovery. The application can realize high-fidelity three-dimensional reconstruction only by using a single relief RGB image without real depth labels and entity scanning data, significantly improves the restoration precision of carving textures, maintains the accuracy of the global geometric structure, and has excellent zero-sample cross-style generalization capability and engineering practicability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the fields of computer vision and computer graphics, and in particular to a method and system for zero-sample relief monocular 3D reconstruction based on a dual-teacher mechanism and weighted noise injection. Background Technology

[0002] As a unique art form situated between two-dimensional painting and three-dimensional sculpture, relief sculpture is widely used in architectural decoration, craft design, digital asset creation, and the digital preservation of cultural heritage. Its three-dimensional digital reconstruction is urgently needed in scenarios such as virtual museum construction, artifact restoration and archiving, and digital cultural and creative product development. However, traditional three-dimensional acquisition technologies have significant bottlenecks: three-dimensional laser scanning requires expensive equipment and physical contact with the object, while multi-view photogrammetry relies on multi-angle image acquisition and complex post-processing. For damaged, immovable (such as cliff carvings) or relief artifacts with only historical photographs remaining, the above methods are completely ineffective because they cannot acquire the physical entity or meet the data acquisition conditions, severely hindering the digital preservation of cultural heritage.

[0003] To overcome hardware and data dependencies, monocular depth estimation technology has become a research focus in single-image 3D reconstruction. This technology mines monocular cues such as perspective, occlusion, and lighting in images, uses deep learning models to predict pixel-level depth maps, and then generates a 3D model through backprojection. However, existing solutions face multiple challenges in relief sculpture scenes:

[0004] 1. Supervised learning methods rely on ground truth depth data: For example, end-to-end depth estimation models based on CNN or Transformer require training with pairs of "image-laser scan depth ground truth values". Obtaining high-precision depth ground truth values ​​for relief artifacts is extremely costly, and the relief styles of different dynasties, regions, and materials vary significantly, resulting in weak model generalization ability and easy geometric distortion when applied across domains; at the same time, the inherent smoothing effect of regression loss functions makes it difficult to effectively recover minute carving textures (such as folds in clothing, hair strands, and patterns).

[0005] 2. Poor Domain Adaptability of General Zero-Shot Depth Models: While models like Marigold and DepthAnything, pre-trained on large-scale natural scenes, possess zero-shot inference capabilities, their training data primarily consists of indoor / outdoor natural scenes, resulting in a significant domain shift compared to the unique 2.5D structure of relief sculptures—"shallow relief on a flat surface." These models are prone to misinterpreting surface lighting and shadows as planar textures, leading to global geometric errors. Furthermore, due to a lack of prior modeling of high-frequency details in the relief, the generated depth maps are overly smoothed, resulting in a severe loss of key artistic features. As closed "black box" systems, they also lack controllable optimization mechanisms tailored to the characteristics of relief sculptures.

[0006] 3. Misalignment between point cloud completion and relief generation technologies: Point cloud completion methods (such as 3D-GANs) require residual point clouds as input and cannot start the reconstruction process from pure two-dimensional images; relief generation technologies (such as methods based on grayscale mapping or 3D model compression) focus on visual stereoscopic simulation, and the generated results lack real geometric scale constraints, resulting in distorted depth proportions, which cannot meet the stringent requirements of geometric accuracy for the digitization of cultural relics.

[0007] 4. Difficulty in balancing detail restoration and structural stability: Existing generative methods (including diffusion models) are prone to introducing structural artifacts when enhancing details, while strengthening global constraints leads to the suppression of high-frequency information. Relief surfaces have the dual requirements of macroscopic compositional accuracy and microscopic carving precision. Existing technologies lack a collaborative optimization mechanism for "low-frequency structure" and "high-frequency details," making it difficult to achieve high-fidelity reconstruction.

[0008] In summary, current technologies for 3D reconstruction of single-image reliefs suffer from fundamental contradictions: they cannot escape dependence on physical entities or ground truth depth values, struggle to simultaneously ensure global geometric plausibility and local texture realism under zero-sample conditions, and exhibit weak cross-style generalization capabilities. Therefore, a novel zero-sample 3D reconstruction method is urgently needed that does not rely on real depth supervision, can adaptively integrate multi-source prior knowledge, and provides targeted enhancement for high-frequency details in reliefs, thereby overcoming key technological bottlenecks in the digital preservation of cultural heritage. Summary of the Invention

[0009] This invention addresses the shortcomings of existing technologies by providing a zero-sample monocular 3D reconstruction method, system, device, and storage medium for relief sculpture.

[0010] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0011] A zero-shot relief monocular 3D reconstruction method, comprising: a training phase and an inference phase;

[0012] The training phase includes:

[0013] Constructing a dual-teacher pseudo-supervised signal: The embossed RGB image is input into multiple pre-trained monocular depth estimation models with different architectures. The depth prediction maps output by each model are fused at the pixel level to generate a consensus depth map as the first teacher signal. At the same time, the surface normals of the embossed RGB image are extracted based on the pre-trained depth estimation model, and the surface normals are mapped to edge intensity maps through a geometry-edge adapter to generate an edge prior map as the second teacher signal.

[0014] Diffusion refinement with weighted noise injection is performed as follows: the embossed RGB image and the consensus depth map are encoded into the latent space; a spatially adaptive noise intensity mask is constructed based on the edge prior map; and non-uniform noise injection is performed on the latent space representation during the forward noise addition process, so that the noise intensity in the edge texture region is higher than that in the flat region; a predicted depth map is generated through the denoising network of the diffusion model; and a spatially perceptual loss function is calculated based on the predicted depth map, the consensus depth map, and the edge prior map; the diffusion model is optimized using the loss function.

[0015] The reasoning phase includes:

[0016] The RGB image of the relief to be reconstructed is input into the trained diffusion model, which outputs a fine depth map.

[0017] The fine depth map is converted into a relief 3D point cloud or mesh model by back projection.

[0018] Furthermore, the construction of the noise intensity mask satisfies the following: regions with high intensity values ​​in the edge prior map are assigned high noise weights, and regions with low intensity values ​​are assigned low noise weights. The weighted noise injection formula is:

[0019]

[0020] in, This represents the latent space of the consensus depth graph. For normalized noise intensity mask, This indicates element-wise multiplication.

[0021] Furthermore, the spatial perception loss function includes depth consistency loss. With edge detail loss The total loss is expressed as:

[0022]

[0023] in, and These are the weighting coefficients for depth consistency loss and edge detail loss, respectively.

[0024] The depth consistency loss The scale-invariant log loss, weighted by edge awareness, is defined as follows:

[0025]

[0026] in, To predict depth map at pixels Depth value at that location, For consensus depth maps at pixels Depth value at that location, For scale-invariant control parameters, the preferred value is... .

[0027] The The weighted average operator is defined as follows:

[0028] in, For the spatial weight map constructed from the edge prior map at the pixel The weight values ​​are assigned to locations near the edge and higher weights to non-edge areas, in order to reduce the constraint of the consensus depth map on the prediction results in the edge areas and enhance the global structural consistency in the non-edge areas.

[0029] The loss of edge details The combination of weighted binary cross-entropy loss and Dice loss is expressed as follows:

[0030]

[0031] in, The edge map is extracted from the predicted depth map. For edge prior maps, As the balance coefficient, the preferred value is... Among them, the weighted binary cross-entropy loss is used to alleviate the class imbalance problem between edge pixels and non-edge pixels, and the Dice loss is used to enhance the overall overlap and connectivity of edge regions.

[0032] Furthermore, the edge detail loss is based on the predicted edge map. The predicted edge map is generated by calculating the difference between the predicted edge map and the edge prior map E using the following formula:

[0033]

[0034] in, Normals calculated from the predicted depth map Quantity, This is the sharpening factor.

[0035] Furthermore, the geometry-edge adapter includes convolutional and projection layers for converting surface normal features into edge intensity representations that are sensitive to embossed micro-scratches.

[0036] Furthermore, the number of the multiple pre-trained monocular depth estimation models Furthermore, the model architectures are distinct to enhance the robustness of the consensus depth graph to the global geometry of the relief.

[0037] Furthermore, the diffusion model is a latent diffusion model, and the encoding / decoding of the image and depth map is achieved through a variational autoencoder.

[0038] This invention discloses a relief 3D digital reconstruction system for performing the above-described relief monocular 3D reconstruction method, comprising:

[0039] The dual-teacher signal generation module is configured to: input the embossed RGB image into multiple pre-trained monocular depth estimation models with different architectures, perform pixel-level median fusion on the depth prediction maps output by each model to generate a consensus depth map; and extract the surface normals of the embossed RGB image based on the pre-trained depth estimation model, and map the surface normals to an edge intensity map through a geometry-edge adapter to generate an edge prior map.

[0040] The weighted noise diffusion refinement module is configured to: encode the embossed RGB image and the consensus depth map into the latent space; construct a spatially adaptive noise intensity mask based on the edge prior map; inject non-uniform noise into the latent space representation during the forward noise addition process; generate a predicted depth map through the denoising network of the diffusion model; and calculate a spatially perceptual loss function based on the predicted depth map, the consensus depth map, and the edge prior map to optimize the diffusion model.

[0041] The 3D model generation module is configured to: input the RGB image to be reconstructed into the trained diffusion model to output a fine depth map, and convert the fine depth map into a relief 3D point cloud or mesh model through back projection.

[0042] Compared with the prior art, the advantages of the present invention are as follows:

[0043] 1. Completely break through the dependence on deep truth data, significantly reducing the threshold and cost of digitizing cultural heritage.

[0044] This invention utilizes a dual-teacher pseudo-supervision mechanism to achieve high-precision 3D reconstruction using only a single RGB image of the relief, completely eliminating reliance on expensive equipment such as laser scanning for obtaining accurate depth labels. This feature allows the technology to be directly applied to damaged, immovable (such as cliff carvings), or inaccessible relief artifacts for which only historical photographs exist. It overcomes the application bottleneck of traditional supervised learning methods caused by difficulties in data acquisition, significantly reducing the economic cost and technical threshold of digital preservation, and providing a feasible path for large-scale rescue archiving of cultural heritage.

[0045] 2. Achieve high-fidelity reconstruction of high-frequency engraving details and global geometry.

[0046] This invention innovatively integrates a weighted noise injection strategy with a spatially perceptive loss function to perform targeted enhancement on edge texture regions during the diffusion refinement process. This mechanism effectively overcomes the inherent smoothing effect of depth estimation, accurately recovering the micro-carving features of the relief surface (such as the direction of clothing folds, hair texture, decorative patterns, etc.), while strictly constraining the overall geometric structure through a consensus depth map. Experimental verification shows that the reconstruction results significantly outperform existing methods in terms of detail sharpness and spatial scale accuracy, meeting the stringent requirements of cultural heritage digitization for both the restoration of craft textures and geometric precision.

[0047] 3. Possesses excellent zero-shot cross-domain generalization capability, adapting to multi-source heterogeneous relief sculpture scenarios.

[0048] This invention generates a robust consensus depth by median fusion of multi-architecture pre-trained models and incorporates low-level edge priors that do not depend on specific domains, thus mechanistically avoiding the risk of overfitting to a single data distribution. Without any fine-tuning, it can stably be applied to relief images from different dynasties (such as Tang and Song dynasty grottoes), different materials (stone carvings / wood carvings / metal reliefs), different artistic styles, and different lighting conditions. The reconstruction results maintain a high level of geometric consistency and detail clarity, completely solving the industry pain points of weak generalization ability and structural distortion in cross-style applications of existing models.

[0049] 4. Enables a fully automated end-to-end reconstruction process, improving project usability and deployment efficiency.

[0050] This invention provides a complete automated inference pipeline from a single 2D image to a 3D point cloud / mesh model, requiring no manual intervention, initial 3D data, or complex post-processing. This feature significantly simplifies the operational process, reduces reliance on technical personnel, and enables rapid deployment in resource-constrained scenarios such as museums, archaeological sites, and digital cultural and creative industries, providing an efficient, easy-to-use, and scalable solution for cultural heritage digitization projects.

[0051] 5. Enhance the controllability and robustness of the reconstruction process in complex scenarios.

[0052] Based on an edge-prior spatial adaptive noise injection and loss weight allocation mechanism, the model can focus on detailed regions and suppress artifact generation. It maintains detail recovery capability and structural stability even when processing blurred, low-resolution, or unevenly lit historical photographs. Furthermore, users can controllably adjust the detail recovery intensity by modifying the noise weight parameters, significantly improving the method's adaptability and reliability in real-world, complex application scenarios. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of a zero-sample relief monocular 3D reconstruction method in one embodiment of the present invention;

[0055] Figure 2 This is a structural block diagram of a zero-sample relief monocular 3D reconstruction method according to an embodiment of the present invention. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] This invention provides a zero-sample monocular 3D reconstruction method for relief sculpture, such as... Figure 1 and 2 As shown, the specific implementation steps are as follows:

[0058] Step 1: Constructing a dual-teacher pseudo-supervision signal

[0059] Due to the lack of true depth labels, this invention first extracts two types of complementary geometric cues from a single RGB image as "dual-teacher" signals. The first type is a consensus depth map, used to provide an accurate global structural benchmark. A single general model is prone to specific biases in relief scenes; therefore, this invention inputs the input image separately into... Among the various pre-trained monocular depth estimation models with different architectures, the following results were obtained: A rough depth prediction map is generated. Subsequently, a pixel-level median fusion algorithm is used to calculate the consensus depth. Median fusion effectively eliminates extreme mispredictions from individual models, ensuring a smooth and accurate global geometry at the initial depth. The second type is edge prior, used to provide guidance on high-frequency details. This invention utilizes a pre-trained Metric3Dv2 model as a feature extractor because its predicted surface normals are more sensitive to curvature changes, making it suitable for characterizing the "soft edges" of reliefs. A cascaded geometry-edge adapter (containing convolutional and projection layers) maps the normal features to edge intensity maps. This edge map can accurately locate minute scratches and textures on the relief surface, compensating for the lack of detail in the consensus depth map.

[0060] Step 2: Diffusion Refinement Based on Weighted Noise Injection

[0061] This invention employs a latent diffusion model as the student network for deep generative processing. First, a variational autoencoder (VAE) is used to encode the RGB image and consensus depth map into a low-dimensional latent space. Then, the core weighted noise injection operation is performed: based on the edge prior map obtained in step S1... Construct a normalized noise intensity mask Higher weights are assigned to textured edge regions and lower weights to flat regions. A non-uniform injection formula is used during the forward noise addition process: This strategy artificially introduces high-intensity noise interference into edge regions, forcing the U-Net denoising network to focus on redrawing the damaged edge textures during the reverse generation process. This allows it to recover lost high-frequency details while preserving the structure of flat regions. Finally, a fine-grained depth prediction map is obtained through VAE decoding. .

[0062] Step 3: Calculation and optimization of the spatial perception loss function

[0063] To resolve the potential conflict between "consensus structure" and "edge details," this invention designs a spatially aware adaptive loss function. First, to calculate the edge loss, a differentiable gradient is used to calculate the predicted depth map. Transform into predicted edge map The calculation formula is: ,in For the Z-component of the normal, The sharpening factor is used as the second factor. Next, a reliability weighting graph based on edge information is constructed. When calculating the loss, for flat regions, a depth-consistency loss is assigned (based on consensus depth). Higher weights are applied to lock in the global structure; for textured regions, edge detail loss is assigned (based on edge priors). Higher weights are used to enhance the clarity of details. Total loss function Through joint optimization, the model can adaptively balance global structure and local details.

[0064] Step 4: Generation of the relief 3D model

[0065] During the inference phase, an arbitrary input relief image is fed into the trained network, which directly outputs a detailed depth map. The pixel coordinates in the depth map are then processed using a backpropagation algorithm. and its depth value Mapped to three-dimensional spatial coordinates It generates high-precision relief 3D point cloud or mesh model to complete digital reconstruction.

[0066] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0067] In another embodiment, a relief 3D digital reconstruction system is provided, which corresponds one-to-one with the relief monocular 3D reconstruction method in the above embodiments, including:

[0068] The dual-teacher signal generation module is configured to: input the embossed RGB image into multiple pre-trained monocular depth estimation models with different architectures, perform pixel-level median fusion on the depth prediction maps output by each model to generate a consensus depth map; and extract the surface normals of the embossed RGB image based on the pre-trained depth estimation model, and map the surface normals to an edge intensity map through a geometry-edge adapter to generate an edge prior map.

[0069] The weighted noise diffusion refinement module is configured to: encode the embossed RGB image and the consensus depth map into the latent space; construct a spatially adaptive noise intensity mask based on the edge prior map; inject non-uniform noise into the latent space representation during the forward noise addition process; generate a predicted depth map through the denoising network of the diffusion model; and calculate a spatially perceptual loss function based on the predicted depth map, the consensus depth map, and the edge prior map to optimize the diffusion model.

[0070] The 3D model generation module is configured to: input the RGB image to be reconstructed into the trained diffusion model to output a fine depth map, and convert the fine depth map into a relief 3D point cloud or mesh model through back projection.

[0071] The system environment of the relief 3D digital reconstruction system is as follows:

[0072] Hardware environment: NVIDIA Quadro RTX 6000 GPU (24GB VRAM), 64GB RAM, Ubuntu 20.04 operating system.

[0073] Software environment: PyTorch 1.12, Diffusers library, OpenCV 4.5, Open3D 0.17.

[0074] Input data: A single embossed RGB image (resolution recommended ≥512×512), in JPEG / PNG format;

[0075] Pre-trained model preparation:

[0076] Monocular depth estimation teacher model: Six mainstream pre-trained models are used, including Lotus (Diffusion-based), Metric3Dv2 (Swin-T backbone), Depth Anything v2 (ViT-L backbone), UniDepth (ViT-L backbone), Pixel-Perfect (ViT-L backbone) and Marigold (Diffusion-based);

[0077] Surface normal extraction model: Metric3Dv2 (pre-trained on general scenarios);

[0078] VAE codec: Uses KL-f8 VAE compatible with Stable Diffusion v1.4;

[0079] Diffusion backbone network: Based on Stable Diffusion U-Net, with input channels configured according to the latent variable dimensions.

[0080] Implementation effectiveness verification: Verification was performed on a test set containing grotto reliefs (stone).

[0081] Detail restoration: DBE for detail extraction effects (hair strands, clothing wrinkles, etc.) is improved by 44.08% (vs DepthAnythingv2).

[0082] Structural fidelity: Global geometric error (RMSE) was reduced by 16.64%, and the relief undulation ratio conformed to the records of cultural relics archives;

[0083] Robustness: It can still preserve the main sculpted structure even in blurry, low-light historical photographs;

[0084] Efficiency: Single image (slice) inference time ≈ 3.2 seconds (A100 GPU), which meets the needs of actual engineering.

[0085] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0086] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0087] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for monocular 3D reconstruction of relief sculpture, comprising a training phase and an inference phase, characterized in that: The training phase includes: Constructing a dual-teacher pseudo-supervised signal: The embossed RGB image is input into multiple pre-trained monocular depth estimation models with different architectures. The depth prediction maps output by each model are fused at the pixel level to generate a consensus depth map as the first teacher signal. At the same time, the surface normals of the embossed RGB image are extracted based on the pre-trained depth estimation model, and the surface normals are mapped to edge intensity maps through a geometry-edge adapter to generate an edge prior map as the second teacher signal. Diffusion refinement with weighted noise injection is performed as follows: the embossed RGB image and the consensus depth map are encoded into the latent space; a spatially adaptive noise intensity mask is constructed based on the edge prior map; and non-uniform noise injection is performed on the latent space representation during the forward noise addition process, so that the noise intensity in the edge texture region is higher than that in the flat region; a predicted depth map is generated through the denoising network of the diffusion model; and a spatially perceptual loss function is calculated based on the predicted depth map, the consensus depth map, and the edge prior map; the diffusion model is optimized using the loss function. The spatial perception loss function includes depth consistency loss. With edge detail loss The total loss is expressed as: , in, and These are the weighting coefficients for depth consistency loss and edge detail loss, respectively. The depth consistency loss The scale-invariant log loss, weighted by edge awareness, is defined as follows: , in, To predict depth map at pixels Depth value at that location, For consensus depth maps at pixels Depth value at that location, These are scale-invariant control parameters; The The weighted average operator is defined as follows: in, For the spatial weight map constructed from the edge prior map at the pixel The weight values ​​are assigned to locations near the edge and higher weights to locations outside the edge, in order to reduce the constraint of the consensus depth map on the prediction results in the edge region and enhance the global structural consistency in the non-edge region. The loss of edge details The combination of weighted binary cross-entropy loss and Dice loss is expressed as follows: in, The edge map is extracted from the predicted depth map. For edge prior maps, The is the balance coefficient; where the weighted binary cross-entropy loss is used to alleviate the class imbalance problem between edge pixels and non-edge pixels, and the Dice loss is used to enhance the overall overlap and connectivity of edge regions; The reasoning phase includes: The RGB image of the relief to be reconstructed is input into the trained diffusion model, which outputs a fine depth map. The fine depth map is converted into a relief 3D point cloud or mesh model by back projection.

2. The method for monocular three-dimensional reconstruction of relief sculpture according to claim 1, characterized in that, The construction of the noise intensity mask satisfies the following: regions with high intensity values ​​in the edge prior map are assigned high noise weights, and regions with low intensity values ​​are assigned low noise weights. The weighted noise injection formula is: , in, This represents the latent space of the consensus depth graph. For normalized noise intensity mask, This indicates element-wise multiplication.

3. The method for monocular three-dimensional reconstruction of relief sculpture according to claim 1, characterized in that, The edge detail loss is based on the predicted edge map. The predicted edge map is generated by calculating the difference between the predicted edge map and the edge prior map E using the following formula: , in, Normals calculated from the predicted depth map Quantity, This is the sharpening factor.

4. The method for monocular three-dimensional reconstruction of relief sculpture according to claim 1, characterized in that, The geometry-edge adapter includes convolutional and projection layers to convert surface normal features into edge intensity representations that are sensitive to embossed micro-scratches.

5. The method for monocular three-dimensional reconstruction of relief sculpture according to claim 1, characterized in that, The number of pre-trained monocular depth estimation models Furthermore, the model architectures are distinct to enhance the robustness of the consensus depth graph to the global geometry of the relief.

6. The method for monocular three-dimensional reconstruction of relief sculpture according to claim 1, characterized in that, The diffusion model is a latent diffusion model, and the encoding / decoding of the image and depth map is achieved through a variational autoencoder.

7. A three-dimensional digital reconstruction system for relief sculpture, characterized in that, A method for performing relief monocular 3D reconstruction as described in any one of claims 1 to 6, comprising: The dual-teacher signal generation module is configured to: input the embossed RGB image into multiple pre-trained monocular depth estimation models with different architectures, perform pixel-level median fusion on the depth prediction maps output by each model to generate a consensus depth map; and extract the surface normals of the embossed RGB image based on the pre-trained depth estimation model, and map the surface normals to an edge intensity map through a geometry-edge adapter to generate an edge prior map. The weighted noise diffusion refinement module is configured to: encode the embossed RGB image and the consensus depth map into the latent space; construct a spatially adaptive noise intensity mask based on the edge prior map; inject non-uniform noise into the latent space representation during the forward noise addition process; generate a predicted depth map through the denoising network of the diffusion model; and calculate a spatially perceptual loss function based on the predicted depth map, the consensus depth map, and the edge prior map to optimize the diffusion model. The 3D model generation module is configured to: input the RGB image to be reconstructed into the trained diffusion model to output a fine depth map, and convert the fine depth map into a relief 3D point cloud or mesh model through back projection.

Citation Information

Patent Citations

  • Three-dimensional bas-relief reconstruction method based on monocular image

    CN119399370A