A U-OPT-based unsupervised single-view three-dimensional reconstruction method

Abstract

The application provides a U-OPT-based unsupervised single-view three-dimensional reconstruction method, and belongs to the technical field of computer vision and three-dimensional reconstruction. The method is based on the Unicorn unsupervised reconstruction core paradigm, a global-local dual-branch structured autoencoder architecture is constructed, a decoupled collaborative phased controllable optimization strategy and a multi-dimensional unsupervised loss function system are designed, the decoupled modeling of global shape prior and local fine-grained features is realized, and the performance bottleneck of the pure global model in the aspect of insufficient local detail description is broken. The method of the application can complete end-to-end training and high-precision three-dimensional reconstruction through only single-view RGB images without relying on any 3D true value, contour mask, key point, camera pose and other supervision information.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and 3D reconstruction technology, specifically involving an unsupervised single-view method based on U-OPT (Unicorn-Optimized, optimized original model). Figure 3 Dimensional reconstruction method. Background Technology

[0002] single vision Figure 3 3D reconstruction is one of the core challenges in the field of computer vision. Its goal is to recover the complete 3D geometry and texture information of an object from a given 2D image. This technology is widely used in scenarios such as autonomous driving, virtual reality, augmented reality, and robot autonomous navigation, and has significant theoretical and engineering value.

[0003] Early monocular vision Figure 3 3D reconstruction methods often use voxels as 3D shape representations, employing deep learning models to learn the mapping relationship between images and voxel meshes. However, voxel representations themselves suffer from limitations in resolution, high computational and storage costs, and difficulty in reproducing fine-grained details of objects. Therefore, 3D meshes, which provide continuous surface geometry and high-quality visual rendering, are gradually becoming the preferred method for single-view reconstruction. Figure 3 The mainstream representation form for 3D reconstruction. Existing single-view reconstructions based on 3D meshes... Figure 3 Although the 3D reconstruction method has been continuously innovating in terms of representation form and network structure, the following common bottlenecks still need to be addressed: (1) The accuracy of shape restoration needs to be improved: Most existing methods are difficult to take into account both the overall topological outline of the object and fine-grained geometric details. The reconstructed 3D mesh generally has problems such as rough shape, edge distortion, and missing details; (2) The generalization ability of the model has inherent shortcomings: The above methods all belong to the fully supervised learning paradigm, which strongly depends on large-scale RGB images and 3D mesh ground truth annotation datasets to complete the training. When facing complex free shapes, special pose objects or low-quality input images outside the training set, the reconstruction performance will drop sharply.

[0004] Therefore, in recent years, completely unsupervised single-view Figure 3 Unsupervised single-view RGB image reconstruction methods have been proposed. These methods do not require paired 3D labeled data and can complete training and reconstruction using only a single-view RGB image. Among them, Unicorn represents an unsupervised single-view reconstruction method. Figure 3 3D reconstruction models achieve end-to-end image reconstruction and unsupervised learning through structured autoencoders and differentiable rendering. However, while such methods improve generalization performance to some extent, they still have the following new technical shortcomings:

[0005] (1) Pure global deformation modeling is not good at depicting the local micro-geometry of objects, and the reconstruction results are prone to problems such as excessive smoothing of edges and loss of fine-grained structure;

[0006] (2) There is an inherent coupling contradiction between global shape consistency and local detail optimization. Synchronous optimization is prone to local optimization destroying the global topology and generating geometric illusions that do not conform to the real physical laws.

[0007] (3) In the process of unsupervised training, there are common problems such as mode collapse and unstable gradient backpropagation. The module freezing mechanism (such as dynamically modifying the requires_grad attribute) is prone to causing computation graph breakage and gradient backpropagation abnormalities, resulting in model training divergence.

[0008] Therefore, it is necessary to develop an unsupervised single-view algorithm that can balance global shape topological consistency and local fine-grained detail reproduction in an unsupervised paradigm, while also possessing a stable and controllable training process. Figure 3 The new 3D reconstruction method is of great significance for lowering the technical threshold for 3D reconstruction and expanding its generalization capabilities across categories and scenarios. Summary of the Invention

[0009] To address the problems existing in the background technology, the purpose of this invention is to provide an unsupervised single-view algorithm based on U-OPT. Figure 3 This invention proposes a 3D reconstruction method. Based on the Unicorn unsupervised reconstruction paradigm, this method constructs a global-local dual-branch structured autoencoder architecture, designs a decoupled and collaborative phased controllable optimization strategy and a multi-dimensional unsupervised loss function system, achieving decoupled modeling of global shape priors and local fine-grained features, thus overcoming the performance bottleneck of insufficient local detail representation in pure global models. This method achieves end-to-end training and high-precision 3D reconstruction using only a single-view RGB image without relying on any 3D ground truth, contour masks, keypoints, camera pose, or other supervisory information.

[0010] To achieve the above objectives, the technical solution of the present invention is as follows:

[0011] An unsupervised single-view based U-OPT Figure 3 The 3D reconstruction method includes the following steps:

[0012] Step 1. Obtain a single-view RGB image, and then perform preprocessing on the image, including size normalization and pixel value normalization;

[0013] Step 2. Construct a topology-aware global-local dual-branch structured autoencoder to predict four explicit factors of an object: shape, texture, camera pose, and background, from end to end.

[0014] Step 3. Construct a multi-dimensional unsupervised composite loss function system consisting of reconstruction loss, neighborhood reconstruction loss, regularization loss, and pose uniformity loss;

[0015] Step 4. Design a decoupled and collaborative phased controllable optimization strategy, specifically, alternate optimization of 3D structure and camera pose to achieve safe decoupling of global and local optimization;

[0016] Step 5. Complete the reconstructed image generation and gradient backpropagation through differentiable rendering. After a preset number of iterations of training, a trained topology-aware global-local dual-branch structured autoencoder is obtained.

[0017] Step 6. Input the preprocessed image from Step 1 into the trained autoencoder, and output a 3D mesh that retains local fine-grained features.

[0018] Furthermore, the preprocessing in step 1 includes:

[0019] Step 1.1. Image size standardization: All input images are uniformly processed into m×m pixel RGB three-channel images to avoid geometric deformation of objects caused by non-uniform scaling; at the same time, channel verification is performed on the images to remove invalid samples with abnormal grayscale images and transparent channels;

[0020] Step 1.2. Pixel value normalization: Convert the uint8 format pixel values ​​to float32 format and map the pixel values ​​uniformly to the range [-1, 1], so that the pixel values ​​are aligned with the numerical input requirements of the VGG16 network used for calculating the perceptual loss in the pre-training process, and obtain a normalized image.

[0021] Furthermore, the topology-aware global-local dual-branch structured autoencoder in step 2 includes a progressive shape deformation module, a channel-level texturing module, an affine transformation module, and a background modeling and differentiable rendering module, which are used to extract features from the preprocessed standard image.

[0022] The progressive shape deformation module is based on the three-dimensional point coordinates of the reference ellipsoid and the shape code extracted from the input image. It is used to model the object shape as a combination of global deformation and local residual micro-deformation of the reference ellipsoid, complete the construction of the object's geometry, and obtain a shape mesh that has both global topological consistency and local fine-grained features.

[0023] The channel-level texturing module generates and binds texture features consistent with the input image to the shape mesh based on the texture code extracted from the input image and the shape mesh output by the progressive shape deformation module, resulting in a textured shape mesh.

[0024] The affine transformation module predicts the 6D pose and anisotropic scaling parameters of the object relative to the camera based on a textured full shape mesh, and performs corresponding spatial transformations on the textured shape mesh to obtain a posed mesh transformed to the pose of the target camera.

[0025] The background modeling and differentiable rendering module is based on the background latent code and pose mesh extracted from the input image. In the background modeling stage, it generates a background image with the same resolution as the input image. In the differentiable rendering stage, it completes the fusion of the pose mesh and the 2D generated background. Finally, it outputs a 2D reconstructed image containing foreground objects and the generated background, which is used for subsequent unsupervised gradient backpropagation.

[0026] Furthermore, the specific working process of the progressive shape deformation module is as follows:

[0027] Given a 3D vertex x of an ellipsoid, the shape deformation is defined as:

[0028]

[0029] in, For a multilayer perceptron with global branches, the input is a 3D point x and a shape code extracted from the input image via an autoencoder. The stitching result; the global displacement predicted by the multilayer perceptron. After being applied to all ellipsoidal vertices, a shaped mesh is generated, enabling surface mapping at any resolution;

[0030] Building upon this, a delayed-start local residual deformation branch is introduced. This branch only performs minute displacement adjustments on the vertices after global deformation, and is responsible for characterizing fine-grained features such as the object's edges and bumps. The mathematical definition of the local residual deformation branch is shown in the following equation:

[0031]

[0032] in, These are the vertex coordinates after global deformation. Lightweight MLP for local branches These are local shape dynamic weight coefficients, whose values ​​vary with the training epochs. Controllable growth;

[0033] The complete shape deformation result of the progressive shape deformation module is the superposition of global deformation and local micro-deformation, as shown in the following formula:

[0034]

[0035] in, The reference ellipsoid;

[0036] The specific processing flow from the global branch to the local branch is as follows: First, the reference ellipsoid... The original three-dimensional vertices and their corresponding shape codes The concatenated data is then input into the global branch of the multilayer perceptron. In the process, the global displacement is predicted and applied to the original 3D vertices. Above, a global deformable mesh with a basic topological profile and its vertex coordinates are generated. Subsequently, the coordinates of the globally deformed vertices are... and their corresponding shape codes Lightweight MLPs that splice and feed local branches In the calculation, the local residual micro-displacement is obtained; finally, the local shape dynamic weighting coefficient is used. The amplitude of the local residual micro-displacement is constrained and superimposed onto the global deformed vertex coordinates. The output is a final shape mesh S that combines global topological consistency with local fine-grained features. The design of local branches strictly limits the displacement to a small range, ensuring that local optimization is based entirely on the converged global shape prior.

[0037] Furthermore, the specific process of controllable growth of the local shape dynamic weight coefficient is as follows:

[0038] The local residual deformation branch defaults to the first... Wheel start, front The effective output of local branches is completely frozen (with a weight of 0), and only the global branches are optimized to ensure that the model learns a stable category-level global shape prior first.

[0039] From the At the beginning of each training round, the local weights increase linearly with each training round until they reach the preset weight cap. As shown in the following formula:

[0040]

[0041] in, This is the current training round; To initiate the round; Terminating the round for growth; This represents the upper limit of shape weight.

[0042] Furthermore, the specific working process of the channel-level texturing module is as follows:

[0043] First, given the texture code Using convolutional networks Generate a texture image that associates each two-dimensional point with a vertex of a shape mesh through a spherical coordinate mapping;

[0044] Then, a local texture fine-tuning branch is designed to achieve decoupled modeling of category-level global texture prior and instance-level fine-grained texture. The calculation process is shown in the following formula:

[0045] ,

[0046] ,

[0047] in, For global texture channels, MLP for local texture fine-tuning This refers to the local texture dynamic weight coefficient; the linear and controllable growth rule of this coefficient is completely consistent with that of the local shape dynamic weight coefficient, and its value increases linearly with the training epoch t until it reaches the preset texture weight upper limit. ; Indicates UV mapping;

[0048] The specific processing flow of the local texture fine-tuning branch is as follows: A two-layer linear multilayer perceptron architecture is adopted, using global texture channel features of dimension C and texture code of dimension D as joint inputs. Through a hidden layer, the output is a feature fine-tuning amount of dimension C; simultaneously, local texture dynamic weight coefficients are utilized. The intensity of the feature fine-tuning is constrained and limited; finally, the fine-tuned feature is fused with the global texture, and the fused final texture map is bound to the shape mesh S obtained by the progressive shape deformation module through UV mapping to generate a textured complete shape mesh T.

[0049] Furthermore, the specific working process of the affine transformation module is as follows:

[0050] The affine transformation module generates K candidate poses using rotation matrices and translation vectors, and assigns them corresponding probabilities p. k The pose with the highest probability is selected and integrated with the corresponding scaling parameters into a single affine transformation. Subsequently, the affine transformation was performed. Applying the action to all 3D points x on a textured, full-shape mesh T, a poseted mesh P is generated;

[0051] Affine transformation The formula for coordinate transformation of a three-dimensional point x is:

[0052] ;

[0053] in, For rotation matrix, is an anisotropic scaling diagonal matrix, and t is the translation vector.

[0054] Furthermore, in the background modeling and differentiable rendering module, differentiable rendering is implemented using layered aggregation rendering, and the layered aggregation rendering formula is as follows:

[0055]

[0056] in, Image pixel coordinates, This is the final rendered color for that pixel. The sampled color of the foreground at this pixel is used to render the foreground, which is a posed mesh; To generate the background The color of the pixel; N is the number of foreground triangles in the view corresponding to that pixel; The transparency value of the i-th triangle in this pixel; The cumulative transparency of the pose grid;

[0057] Furthermore, an exponential occupancy function is adopted instead of the traditional sigmoid occupancy function. By adjusting the decay rate of the occupancy probability, problems such as in-plane edge blurring and artifacts in the rendering results are effectively suppressed, significantly improving the edge sharpness of the reconstructed image. The calculation formula is as follows:

[0058]

[0059] Where d is the normalized distance from the pixel sampling point to the center of the triangle face. is the effective radius of influence of the triangular face, and k is the attenuation coefficient used to control the attenuation rate of transparency.

[0060] Furthermore, in step 3, the multi-dimensional unsupervised composite loss function The calculation formula is:

[0061]

[0062] In the formula, The reconstruction loss is calculated at the pixel level. Loss consists of perceived loss; For the reconstruction losses in the surrounding area, This is the regularization loss; The pose uniformity loss is defined as the sum of the average probability of pose candidates and the uniform distribution. Errors should be minimized to prevent mode collapse during multi-pose prediction. , and All three are preset weight coefficients ranging from 0 to 1, and their sum equals 1. They are used to balance the contribution level of the corresponding loss items in the total loss.

[0063]

[0064] For pixel Loss, guaranteeing reconstructed image Align with the input image I at the pixel level; Perceived loss These are the weighting coefficients;

[0065] Regularization loss Loss of normal consistency Laplace smoothing loss constitute, ;

[0066] Among them, the normal consistency loss constrains the continuity of the normal vectors of adjacent triangular faces to avoid sharp surface abrupt changes; the Laplacian smoothing loss constrains the local smoothness of mesh vertices to avoid outlier vertices and surface wrinkles.

[0067] Furthermore, the alternating optimization of 3D structure and camera pose is achieved through non-intrusive tensor operations, the specific process of which is as follows:

[0068] Step 4.1. Perform module freezing: Module freezing is achieved solely through tensor detach operations, removing operations that dynamically modify gradient calculation properties (requires_grad); in the pose optimization step, the output results of the 3D mesh and background are separated, and only the pose prediction branch is updated; in the 3D structure optimization step, the pose parameters are separated, and only the shape, texture, and background branches are updated, completely avoiding computation graph breakage and gradient backpropagation anomalies;

[0069] Step 4.2. Perform alternating step size control: switch the optimization stage every 2 iterations. By using a fixed step size period, the 3D structure and pose optimization complement each other, avoiding training imbalance caused by over-optimization of a single module.

[0070] Step 4.3. Perform global-local separation gradient clipping: Set the gradient clipping threshold for global parameters to w1 and the gradient clipping threshold for local branch parameters to w2, where w1 > w2 > 0. While ensuring the effectiveness of local optimization, strictly limit the magnitude of local gradients to avoid training divergence.

[0071] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0072] 1. This invention proposes an unsupervised single-view method based on U-OPT. Figure 3 The method for 3D reconstruction decouples the modeling process of object shape and texture into two stages: global topology modeling and local detail enhancement, by constructing a global-local dual-branch structured autoencoder architecture. Under the condition that it does not rely on any 3D ground truth, contour mask, key points or other supervision information, it achieves the synergistic improvement of global shape consistency and local fine-grained details, and breaks through the performance bottleneck of insufficient local detail depiction of pure global models.

[0073] 2. This invention designs a 3D structure and camera pose alternating optimization framework based on tensor separation. It completely avoids computation graph breakage and gradient backpropagation anomalies through a non-intrusive module freezing mechanism. Combined with a global-local separation gradient pruning strategy, it effectively ensures the stability of unsupervised training. At the same time, it introduces a local weight linearly controllable growth strategy to achieve an ordered training process of "global convergence first, local enhancement later".

[0074] 3. Comparative experiments on the publicly available benchmark datasets ShapeNet NMR and CUB-200-2011 demonstrate that the method of this invention comprehensively outperforms the baseline model Unicorn in the unsupervised paradigm, and achieves Chamfer-level superiority in 11 out of 13 test categories on ShapeNet NMR. Compared to Unicorn, the overall average relative accuracy is improved by 4.48%, especially for irregular object categories such as speakers, vessels, and displays, where the improvement in local detail accuracy is more significant. At the same time, under completely unsupervised constraints, its performance surpasses the classic supervised method DVR (relative improvement of 23.2%), which relies on camera pose and contour mask supervision. On the CUB-200-2011 fine-grained bird dataset, the PCK@0.1 index is improved to 49.7 and the Mask IoU is improved to 71.4, verifying the effectiveness and robustness of the method in synthetic and real-world scenes. Attached Figure Description

[0075] Figure 1 This invention is based on U-OPT single-view Figure 3 A technical roadmap for dimensional reconstruction methods.

[0076] Figure 2 This is a structural diagram of the global-local dual-branch structured autoencoder of the present invention.

[0077] Figure 3 These are images of common objects from the ShapeNet NMR dataset in Embodiment 1 of this invention.

[0078] Figure 4 This is a graph showing the loss change during the U-OPT training process in Embodiment 1 of the present invention.

[0079] Figure 5 This is a visualization comparison of some models in the ShapeNet NMR dataset in Embodiment 1 of the present invention.

[0080] Figure 6 This is a visualization comparison chart of some models in the CUB-200-2011 dataset in Embodiment 1 of the present invention. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings.

[0082] An unsupervised single-view based U-OPT Figure 3 3D reconstruction method, the technical roadmap of which is as follows: Figure 1 As shown, it includes the following steps:

[0083] Step 1. Obtain a single-view RGB image, and then perform preprocessing on the image, including size normalization and pixel value normalization;

[0084] Step 2. Construct a topology-aware global-local dual-branch structured autoencoder to predict four explicit factors—object shape, texture, camera pose, and background—end-to-end; the autoencoder structure diagram is shown below. Figure 2 As shown; it includes a progressive shape deformation module, a channel-level texturing module, an affine transformation module, and a background modeling and differentiable rendering module;

[0085] Step 3. Construct a multi-dimensional unsupervised composite loss function system consisting of reconstruction loss, neighborhood reconstruction loss, regularization loss, and pose uniformity loss;

[0086] Step 4. Design a decoupled and collaborative phased controllable optimization strategy, specifically, alternate optimization of 3D structure and camera pose to achieve safe decoupling of global and local optimization;

[0087] Step 5. Complete the reconstructed image generation and gradient backpropagation through differentiable rendering. After a preset number of iterations of training, a trained topology-aware global-local dual-branch structured autoencoder is obtained.

[0088] Step 6. Input the preprocessed image from Step 1 into the trained autoencoder, and output a 3D mesh that retains local fine-grained features.

[0089] Example 1

[0090] An unsupervised single-view based U-OPT Figure 3 The 3D reconstruction method includes the following steps:

[0091] Step 1: Single-view RGB image acquisition and preprocessing

[0092] Step 1.1. Dataset Acquisition: This embodiment uses two publicly available datasets, ShapeNet NMR and CUB-200-2011, for model training and validation. The ShapeNet NMR dataset is built based on the large-scale 3D model library ShapeNetCore, selecting 13 of the largest rigid object categories (including airplanes, cars, chairs, lamps, sofas, etc.), which are single-view... Figure 3Classic benchmark datasets for 3D reconstruction tasks, common object images such as Figure 3 As shown, the CUB-200-2011 dataset focuses on North American birds, covering 200 bird species and containing 11,788 real-world RGB images of birds. The images exhibit rich intra-class differences and scene diversity, effectively validating the model's ability to extract subtle visual features.

[0093] Step 1.2. Image Preprocessing: All input images are uniformly processed into 64×64 pixel RGB three-channel images to avoid geometric deformation caused by non-uniform scaling; the 0-255 pixel values ​​in uint8 format are converted to float32 format and uniformly mapped to the [-1, 1] interval to align with the numerical input requirements of the pre-trained VGG16 network. Simultaneously, channel validation is performed on all images to remove invalid samples with abnormal grayscale images and transparent channels. The training process uses only the training set data, and the test set is only used for model performance verification. No supervised annotation information such as 3D ground truth, object contours, keypoints, or category semantics is used throughout the process.

[0094] Step 2: Construct a global-local dual-branch structured autoencoder:

[0095] The global-local dual-branch structured autoencoder includes a progressive shape deformation module, a channel-level texturing module, an affine transformation module, and a background modeling and differentiable rendering module.

[0096] The progressive shape deformation module models the object shape as a combination of global deformation and local residual micro-deformation of a reference ellipsoid, including global branches and local residual deformation branches. The global branch applies deformation to an icosahedron slightly stretched into an ellipsoidal mesh by fixing anisotropic scaling, utilizing a multilayer perceptron. Global displacement is predicted by concatenating 3D point coordinates and shape codes. Based on this, a local residual deformation branch with delayed initiation is introduced, employing a two-layer linear MLP architecture (input dimension 3+D, hidden layer dimension 128, output dimension 3) to perform minute displacement adjustments on the globally deformed vertices. The output displacement is then weighted by coefficients. Strictly limited to a tiny range, avoiding local optimization from disrupting the global topology, the final output is a shape mesh S;

[0097] The channel-level texturing module generates texture images using convolutional networks and maps them onto a sphere using spherical coordinates. It includes a global branch and a local texture fine-tuning branch. The global branch generates the global texture; based on this, a local texture fine-tuning branch is designed, employing a two-layer linear MLP architecture (input dimension C+D, hidden layer dimension 64, output dimension C). It only fine-tunes the channel features of the global texture, limiting the intensity of local fine-tuning through dynamic weight coefficients. In other words, the channel-level texturing module binds texture features consistent with the input image to the shape mesh. By fusing the fine-tuned local features with the global texture and binding the fused final texture map to the final shape mesh S output by the progressive shape deformation module through UV mapping, a textured complete shape mesh T is generated.

[0098] The affine transformation module, based on a textured full-shape mesh T, predicts the 6D pose of the object relative to the camera and anisotropic scaling parameters, generating K candidate poses and assigning them corresponding probabilities. During the inference phase, the pose with the highest probability is used, and the scaling parameters and the most likely six-dimensional pose are integrated into a single affine transformation matrix, which is applied to all three-dimensional points x on the textured full-shape mesh T to generate a poseified mesh.

[0099] The background modeling and differentiable rendering module processes the poseable mesh obtained from the affine transformation module. In the background modeling stage, the background latent code obtained from the input image encoding is used as input, and a progressive background generator generates a background image with the same resolution as the input image. In the differentiable rendering stage, an improved soft rasterization differentiable renderer is employed, using a layered aggregation rendering formula based on alpha synthesis and an exponential occupancy function to complete the fusion rendering of the poseable textured mesh and the generated background.

[0100] Step 3: Construct a multi-dimensional unsupervised loss function:

[0101]

[0102] Reconstruction loss Ensure pixel-level alignment and texture awareness; neighborhood reconstruction loss Cross-instance consistency constraints are achieved through feature memory and cross-instance latent code exchange; regularization loss. Mesh geometry rationality is ensured through normal uniformity and Laplace smoothing constraints; attitude uniformity loss. To avoid the collapse of multi-pose prediction patterns, four types of losses work together through preset weights to provide complete supervised constraints for unsupervised training.

[0103] Step 4: Perform phased, controllable optimization training on the autoencoder:

[0104] Step 4.1. Training Environment Configuration: All experiments were conducted on a 40GB NVIDIA A100 graphics card, using Python 3.9, PyTorch 1.13, and CUDA 11.6. The batch size was set to 32, and the base learning rate was 1×10⁻⁶. -4 The Adam optimizer is used, and the learning rate is fine-tuned by dividing it by 5 at the end of training.

[0105] Step 4.2. Alternating Optimization Execution: The 3D structure and camera pose are frozen in modules using tensor separation operations. The optimization stage is switched every two iterations. The gradient clipping threshold for global parameters is set to 1.0, and the gradient clipping threshold for local branch parameters is set to 0.5.

[0106] Step 4.3. Local Weight Control: Local branches start by default at round 30 (taking the ShapeNet NMR dataset as an example, the total number of training rounds is 224), and the local weights are 0 for the first 30 rounds. Starting from round 30, the local shape weights... and local texture weights It increases linearly according to equation (10) until a preset upper limit is reached. During training, the loss change curve is as follows: Figure 4 As shown, the changes exhibit typical characteristics of rapid convergence in the early stage and stable optimization in the later stage, which is highly consistent with the phased progressive training strategy: the total loss (loss_total) drops rapidly from 0.06 to 0.035 within 0-50 steps, corresponding to the model prioritizing the learning of class-level shared features under low-dimensional latent code; the RGB reconstruction loss (loss_rgb) converges the fastest to ensure pixel alignment; the controllable fluctuations within the middle 50-150 steps stem from the strategy switching between progressive conditional upgrade and 3D-pose alternating optimization, at which point the neighborhood reconstruction loss (loss_neighbor) intervenes to constrain cross-instance geometric rules, while the perceptual loss (loss_perceptual) focuses on high-level semantic consistency; in the later stage, the total loss stabilizes at around 0.028, and there are no signs of drastic fluctuations or overfitting, indicating that the training process is stable and controllable;

[0107] After training, the desired global-local bi-branch structured autoencoder is obtained;

[0108] Step 5: Input the single image to be reconstructed into the trained global-local bi-branch structured autoencoder to obtain the reconstruction result.

[0109] Model performance verification

[0110] Visualization comparison of some models in the ShapeNet NMR dataset is shown below. Figure 5 As shown, a comparative experiment using the ShapeNet NMR dataset: with Chamfer- Distance was used as the core evaluation metric, and a horizontal comparison was made with Unicorn (unsupervised baseline), DVR, and SDF-SRN (supervised method). The experimental results are shown in Table 1.

[0111] Table 1. Comparison results of ShapeNet NMR

[0112] airplane 0.109 0.110 0.128 0.114 bench 0.149 0.159 - 0.255 storage cabinet 0.133 0.137 - 0.254 car 0.159 0.168 0.150 0.203 Chair 0.246 0.253 0.262 0.371 monitor 0.201 0.220 - 0.257 Light bulb 0.488 0.523 - 0.363 cell phone 0.122 0.127 - 0.191 rifle 0.099 0.097 - 0.130 sofa 0.186 0.192 - 0.321 speaker 0.188 0.224 - 0.312 table 0.269 0.243 - 0.303 Boat 0.141 0.155 - 0.180 average value 0.192 0.201 - 0.250

[0113] As shown in Table 1, U-OPT achieves comprehensive performance superiority over the baseline model Unicorn under completely unsupervised conditions. In 11 out of 13 test categories, U-OPT outperforms the Chamfer-... At distances lower than Unicorn, the overall average score decreased from 0.201 to 0.192, with a relative accuracy improvement of 4.48%. The improvement was particularly significant for complex, irregular objects: the relative accuracy improved by 16.07% for the speaker category, 9.03% for the boat category, and 8.64% for the monitor category. Furthermore, under completely unsupervised constraints, U-OPT achieved a 23.2% relative improvement in overall average score compared to the DVR method, which relies on camera pose and contour mask supervision, and outperformed DVR in accuracy across all 12 categories.

[0114] The CUB-200-2011 dataset partial model visualization comparison chart is as follows: Figure 6 As shown, the CUB-200-2011 dataset comparison experiment: using PCK@0.1 and Mask IoU as the core evaluation metrics, a horizontal comparison was conducted with CMR, UMR, SMR (supervised method), and Unicorn (unsupervised baseline). The experimental results are shown in Table 2, and some model visualization comparison charts are shown below. Figure 6 As shown.

[0115] Table 2 Comparison Results of CUB-200-2011 Datasets

[0116] CMR √ √ × 48.3 70.6 UMR √ × √ 58.2 73.4 SMR √ × √ 62.2 80.6 Unicorn × × × 49.0 70.4 This invention model × × × 49.7 71.4

[0117] As shown in Table 2, U-OPT achieves comprehensive performance superiority over the baseline model Unicorn in the fully unsupervised paradigm: the PCK@0.1 index improves from 49.0 to 49.7, a relative improvement of 1.43%; the Mask IoU index improves from 70.4 to 71.4, a relative improvement of 1.42%. Without relying on any prior assumptions or supervision information, U-OPT's Mask IoU index surpasses that of the CMR method, which relies on symmetry assumptions and camera pose annotations, and its PCK@0.1 index is essentially on par with the CMR method. Global branches ensure stable convergence and geometric consistency of the overall object shape, driving the improvement in Mask IoU; delayed local branches achieve accurate characterization of fine-grained features, enabling simultaneous optimization of the PCK@0.1 index.

[0118] The above description is merely a specific embodiment of the present invention. Any feature disclosed in this specification may be replaced by other equivalent or similar features unless otherwise specified. All disclosed features, or steps in all methods or processes, may be combined in any way except for mutually exclusive features and / or steps.

Claims

1. A U-OPT based unsupervised single view 3D reconstruction method, characterized in that, Includes the following steps: Step 1. Obtain a single-view RGB image, and then perform preprocessing on the image, including size normalization and pixel value normalization; Step 2. Construct a topology-aware global-local dual-branch structured autoencoder to predict four explicit factors of an object: shape, texture, camera pose, and background, from end to end. Step 3. Construct a multi-dimensional unsupervised composite loss function system consisting of reconstruction loss, neighborhood reconstruction loss, regularization loss, and pose uniformity loss; Step 4. Design a decoupled and collaborative phased controllable optimization strategy, specifically, alternate optimization of 3D structure and camera pose to achieve safe decoupling of global and local optimization; Step 5. Complete the reconstructed image generation and gradient backpropagation through differentiable rendering. After a preset number of iterations of training, a trained topology-aware global-local dual-branch structured autoencoder is obtained. Step 6. Input the preprocessed image from Step 1 into the trained autoencoder, and output a 3D mesh that retains local fine-grained features.

2. The unsupervised single-view 3D reconstruction method as described in claim 1, characterized in that, The preprocessing in step 1 includes: Step 1.

1. Image size normalization: Standardize all input images to the same size. The RGB three-channel image of the pixel avoids geometric deformation of the object caused by non-uniform scaling; at the same time, the image is verified by channel to remove invalid samples with abnormal grayscale and transparency channels. Step 1.

2. Pixel value normalization: Convert the uint8 format pixel values ​​to float32 format and map the pixel values ​​uniformly to the [-1,1] interval, so that the pixel values ​​are aligned with the numerical input requirements of the VGG16 network used for calculating the perceptual loss in the pre-training process, and obtain a normalized image.

3. The unsupervised single-view 3D reconstruction method as described in claim 1, characterized in that, In step 2, the topology-aware global-local bi-branch structured autoencoder includes a progressive shape deformation module, a channel-level texturing module, an affine transformation module, and a background modeling and differentiable rendering module, which are used to extract features from the preprocessed standard image. The progressive shape deformation module is based on the three-dimensional point coordinates of the reference ellipsoid and the shape code extracted from the input image. It is used to model the object shape as a combination of global deformation and local residual micro-deformation of the reference ellipsoid, complete the construction of the object's geometry, and obtain a shape mesh that has both global topological consistency and local fine-grained features. The channel-level texturing module generates and binds texture features consistent with the input image to the shape mesh based on the texture code extracted from the input image and the shape mesh output by the progressive shape deformation module, resulting in a textured shape mesh. The affine transformation module predicts the 6D pose and anisotropic scaling parameters of the object relative to the camera based on a textured full shape mesh, and performs corresponding spatial transformations on the textured shape mesh to obtain a posed mesh transformed to the pose of the target camera. The background modeling and differentiable rendering module is based on the background latent code and pose mesh extracted from the input image. In the background modeling stage, it generates a background image with the same resolution as the input image. In the differentiable rendering stage, it completes the fusion of the pose mesh and the 2D generated background. Finally, it outputs a 2D reconstructed image containing foreground objects and the generated background, which is used for subsequent unsupervised gradient backpropagation.

4. The unsupervised single-view 3D reconstruction method as described in claim 3, characterized in that, The specific working process of the progressive shape deformation module is as follows: Given a 3D vertex x of an ellipsoid, the shape deformation is defined as: ， in, For a multilayer perceptron with global branches, the input is a 3D point x and a shape code extracted from the input image via an autoencoder. The stitching result; the global displacement predicted by the multilayer perceptron. After being applied to all ellipsoidal vertices, a shaped mesh is generated, enabling surface mapping at any resolution; Building upon this, a delayed-start local residual deformation branch is introduced. This branch only performs minute displacement adjustments on the vertices after global deformation, and is responsible for characterizing fine-grained features such as the object's edges and bumps. The mathematical definition of the local residual deformation branch is shown in the following equation: ， in, These are the vertex coordinates after global deformation. Lightweight MLP for local branches The local shape dynamic weight coefficient has a value that increases controllably with the number of training rounds t. The complete shape deformation result of the progressive shape deformation module is the superposition of global deformation and local micro-deformation, as shown in the following formula: ， in, Let S be the reference ellipsoid and S be the final shape grid.

5. The unsupervised single-view 3D reconstruction method as described in claim 4, characterized in that, The specific process of controllable growth of the local shape dynamic weight coefficient is as follows: The local residual deformation branch defaults to the first... Wheel start, front The effective output of the local branch is completely frozen with a weight of 0, and only the global branch is optimized to ensure that the model learns a stable category-level global shape prior first. From the At the beginning of each training round, the local weights increase linearly with each training round until they reach the preset weight cap. As shown in the following formula: ， in, This is the current training round; To initiate the round; Terminating the round for growth; This represents the upper limit of shape weight.

6. The unsupervised single-view 3D reconstruction method as described in claim 3, characterized in that, The specific working process of the channel-level texturing module is as follows: First, given the texture code Using convolutional networks Generate a texture image that associates each two-dimensional point with a vertex of a shape mesh through a spherical coordinate mapping; Then, a local texture fine-tuning branch is designed to achieve decoupled modeling of category-level global texture prior and instance-level fine-grained texture. The calculation process is shown in the following formula: ， ， in, For global texture channels, MLP for local texture fine-tuning This refers to the local texture dynamic weight coefficient; the linear and controllable growth rule of this coefficient is completely consistent with that of the local shape dynamic weight coefficient, and its value increases linearly with the training epoch t until it reaches the preset texture weight upper limit. T represents a textured, full-shape mesh.

7. The unsupervised single-view 3D reconstruction method as described in claim 3, characterized in that, The specific working process of the affine transformation module is as follows: The affine transformation module generates K candidate poses using rotation matrices and translation vectors, and assigns them corresponding probabilities p. k The pose with the highest probability is selected and integrated with the corresponding scaling parameters into a single affine transformation. ; Then, the affine transformation Applying the action to all 3D points x on a textured, full-shape mesh T, a poseted mesh P is generated; Affine transformation The formula for coordinate transformation of a 3D point x is: ； in, Let be a rotation matrix. is an anisotropic scaling diagonal matrix, and t is the translation vector.

8. The unsupervised single-view 3D reconstruction method as described in claim 3, characterized in that, In the background modeling and differentiable rendering module, differentiable rendering is implemented using layered aggregation rendering. The formula for layered aggregation rendering is: ， in, Image pixel coordinates, This is the final rendered color for that pixel. The sampled color of the foreground at this pixel is used to render the foreground, which is a posed mesh; To generate the background The color of the pixel; N is the number of foreground triangles in the view corresponding to that pixel; The transparency value of the i-th triangle in this pixel; The cumulative transparency of the pose grid; An exponential occupancy function is used instead of the traditional sigmoid occupancy function. By adjusting the decay rate of the occupancy probability, in-plane edge blurring and artifacts in the rendering results are effectively suppressed, significantly improving the edge sharpness of the reconstructed image. The calculation formula is as follows: ； Where d is the normalized distance from the pixel sampling point to the center of the triangle face. is the effective radius of influence of the triangular face, and k is the attenuation coefficient used to control the attenuation rate of transparency.

9. The unsupervised single-view 3D reconstruction method as described in claim 1, characterized in that, Step 3: Multi-dimensional unsupervised composite loss function The calculation formula is: ， In the formula, The reconstruction loss is calculated at the pixel level. Loss consists of perceived loss; For the reconstruction losses in the surrounding area, This is the regularization loss; The pose uniformity loss is defined as the sum of the average probability of pose candidates and the uniform distribution. Errors should be minimized to prevent mode collapse during multi-pose prediction. , and All three are preset weight coefficients ranging from 0 to 1, and their sum equals 1. They are used to balance the contribution level of the corresponding loss items in the total loss. ， For pixel Loss, guaranteeing reconstructed image Align with the input image I at the pixel level; Perceived loss These are the weighting coefficients; Regularization loss Loss of normal consistency Laplace smoothing loss constitute, ; Among them, the normal consistency loss constrains the continuity of the normal vectors of adjacent triangular faces to avoid sharp surface abrupt changes; the Laplacian smoothing loss constrains the local smoothness of mesh vertices to avoid outlier vertices and surface wrinkles.

10. The unsupervised single-view 3D reconstruction method as described in claim 1, characterized in that, Alternating optimization of 3D structure and camera pose is achieved through non-intrusive tensor operations, the specific process of which is as follows: Step 4.

1. Module Freeze: Module freeze is achieved only through tensor separation operation, removing operations that dynamically modify gradient calculation properties; in the pose optimization step, the output results of 3D mesh and background are separated, and only the pose prediction branch is updated; in the 3D structure optimization step, the pose parameters are separated, and only the shape, texture, and background branches are updated, completely avoiding computation graph breakage and gradient backpropagation anomalies. Step 4.

2. Perform alternating step size control: switch the optimization stage every 2 iterations. By using a fixed step size period, the 3D structure and pose optimization complement each other, avoiding training imbalance caused by over-optimization of a single module. Step 4.

3. Perform global-local separation gradient clipping: Set the gradient clipping threshold for the global parameters to w1, and the gradient clipping threshold for the local branch parameters to w2, and While ensuring the effectiveness of local optimization, the magnitude of local gradients is strictly limited to avoid training divergence.

Images