Sparse view angle three-dimensional reconstruction pose optimization mode
By constructing a ring-shaped virtual coordinate system in sparse viewpoint 3D reconstruction and combining depth-guided pose projection and curvature-color joint mask, the feature dependency and dynamic interference problems in sparse viewpoint 3D reconstruction are solved, achieving efficient and stable pose optimization and reconstruction results.
Patent Information
- Application Number
- CN202511438904.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-10
- Publication Date
- 2026-01-27
AI Technical Summary
Existing sparse-view 3D reconstruction techniques suffer from poor feature matching dependency, pose drift amplification, dynamic boundary interference, and sensitivity to symmetrical scenes in data-driven methods, resulting in low reconstruction integrity.
We employ a ring-shaped virtual coordinate system design, depth-guided pose projection, generation of curvature-color joint masks, and multimodal differentiable rendering optimization. By constructing a ring-shaped virtual coordinate system, introducing Marigold monocular depth consistency constraints, and combining LAB color space and local curvature features, we integrate the depth-normal-virtual viewpoint three constraints of LM-Gaussian for multimodal regularization optimization.
It achieves efficient pose optimization without feature matching in sparse viewpoints, suppresses dynamic interference, improves reconstruction completeness and inference speed, and meets real-time requirements.
Smart Images
Figure FT_1 
Figure SMS_3 
Figure SMS_4
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and 3D reconstruction technology, specifically relating to a sparse viewpoint 3D reconstruction pose optimization method. Background Technology
[0002] Currently, existing sparse-viewpoint 3D reconstruction pose optimization methods suffer from the following problems: 1. Feature matching dependency: Traditional SFM relies on SIFT / ORB feature point matching, and the matching failure rate is >60% when the viewpoint overlap rate is <15%. 25 2. Pose drift amplification: Incremental SFM translation error accumulates per device under sparse viewing angles (measured 0.7cm / device). 2。 3. Dynamic boundary interference: Non-rigid deformation of the human body / object causes mask boundary jitter (±15px fluctuation). 11 4. Limitations of data-driven methods: Data-driven methods such as ADen require pre-training and are sensitive to symmetrical scenes (rotation error > 10°). 5. To address the aforementioned issues, this solution was developed. Summary of the Invention
[0003] In view of the shortcomings of the prior art, the technical problem to be solved by the present invention is to provide a sparse viewpoint 3D reconstruction pose optimization method, which has high reconstruction integrity after optimization.
[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a sparse viewpoint 3D reconstruction pose optimization method, comprising the following steps: S1. Input a sparse image; S2. Construct a circular virtual coordinate system; S3, Depth-guided pose projection; S4. Generate a curvature-color joint mask; S5, multimodal differentiable rendering optimization; S6. Optimize pose output.
[0005] Furthermore, the design parameters of the annular virtual coordinate system in step S2 are as follows: the mean value of the ground plane normal vector is Z-axis 5, the center of the scene bounding box is the origin, and the longest spatial diagonal is X-axis 7.
[0006] Furthermore, the pose projection enhancement in step S3 employs the following method: Marigold monocular depth consistency constraints are introduced to optimize the projection matrix, using the following formula: ; Where k is the current viewpoint, P is the 3D point set, N is the number of configurable viewpoints, and D is the depth map.
[0007] Furthermore, the curvature-color joint mask in step S4 is as follows: Using Python, the LAB color space and local curvature features are combined, resulting in the following features: mask=(LAB_chroma>130)&(LAB_lightness<20)&(curvature>0.25).
[0008] Furthermore, in step S5, the multimodal regularization integrates the depth-normal-virtual viewpoint three constraints of LM-Gaussian, with the following characteristics: Lreg=0.7Ldepth+0.2Lnormal+0.1Lvirtual.
[0009] Furthermore, the virtual viewpoint production method in step S2 is as follows: equally spaced virtual viewpoints are generated on a ring plane.
[0010] Furthermore, the multimodal regularization weight allocation rule in step S5 is as follows: when the feature point is greater than or equal to 100, γ=0.9; when the feature point is less than 100 or greater than or equal to 50, γ=0.5; when the feature point is less than 50, γ=0.1; where γ represents the weight.
[0011] Compared with the prior art, the present invention has the following beneficial effects: 1. Physical constraints overcome feature dependence: Virtual coordinate system enables zero-overlap pose reuse (no SIFT / ORB feature matching required).
[0012] 2. Dynamic interference suppression: Curvature-color joint masking solves the boundary jitter caused by clothing wrinkles.
[0013] 3. Lightweight real-time optimization: Multimodal regularization replaces NeRF-like implicit coding, improving inference speed by 13 times. Attached Figure Description
[0014] Figure 1 This is a flowchart of a sparse viewpoint 3D reconstruction pose optimization method according to the present invention. Detailed Implementation
[0015] To make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings for detailed explanation.
[0016] like Figure 1 As shown, this embodiment provides a sparse viewpoint 3D reconstruction pose optimization method, including the following steps: S1. Input a sparse image; S2. Construct a circular virtual coordinate system; S3, Depth-guided pose projection; S4. Generate a curvature-color joint mask; S5, multimodal differentiable rendering optimization; S6. Optimize pose output.
[0017] The design parameters of the ring-shaped virtual coordinate system in step S2 are as follows: the mean of the ground plane normal vector is the Z-axis, meaning all cameras perceive the ground as flat, and the average orientation of the ground plane is the "up" direction in the virtual world. The center of the scene bounding box is the origin, and the geometric center of the entire scene is taken as the (0,0,0) point in the virtual world. The longest spatial diagonal is the X-axis (right-handed system). The longest direction in the scene is found as the X-axis, thus establishing a complete right-handed coordinate system.
[0018] The virtual viewpoints are generated in step S2 as follows: equally spaced virtual viewpoints are generated in the annular plane, with an interval angle θ = 360° / M, where M is set according to requirements. In the annular virtual coordinate system, some virtual camera viewpoints (M) are generated evenly to provide more directional constraints for optimization.
[0019] In step S3, pose projection enhancement is achieved using the following method: Marigold monocular depth consistency constraints are introduced to optimize the projection matrix, using the following formula: ; Where k is the current viewpoint, P is the 3D point set, N is the number of configurable viewpoints, and D is the depth. render To generate a depth map using a 3D model based on the currently estimated pose, D mono This is a depth map predicted from a single image using AI models such as Marigold.
[0020] Step S3 involves projecting existing historical poses from other devices or time periods onto this unified virtual coordinate system as an initial estimate of the new camera pose. This initial pose is then fine-tuned using the formula described above to make it more accurate.
[0021] The optimization objective in step S3 is to adjust the pose T so that the rendered depth map and the AI-predicted depth map are as consistent as possible (minimizing the L1 norm difference). Simultaneously, a regularization term (F norm) is added to prevent the adjusted 3D points (TP) from... hist ) from its initial position (P) in the virtual coordinate system virtual Too far, maintain stability.
[0022] The curvature-color joint mask in step S4 is as follows: using Python, the LAB color space and local curvature features are fused together, and the features are as follows: mask=(LAB_chroma>130)&(LAB_lightness<20)&(curvature>0.25).
[0023] Where mask represents the precise outline, LAB_chroma>130: areas with high color saturation; LAB_lightness<20: areas with very low brightness; curvature>0.25: areas with high curvature.
[0024] This feature combines the above conditions to generate a mask, extracting pixels that are brightly colored, not too dark, and located in high-curvature (edge / wrinkle) areas. This effectively eliminates color interference from flat areas (such as a solid color background) and interference from dark shadows.
[0025] In step S5, multimodal regularization is performed by integrating the depth-normal-virtual view three constraints of LM-Gaussian, with the following characteristics: Lreg = 0.7Ldepth + 0.2Lnormal + 0.1Lvirtual.
[0026] L_depth (weight γ=0.7): Depth consistency loss, ensuring geometric correctness.
[0027] L_normal (weight γ=0.2): Normal consistency loss. Ensures that the optimized surface normal direction is smooth and reasonable (calculated through cosine similarity), thus improving surface quality.
[0028] L_virtual (weight γ=0.1): Virtual view constraint loss. Prevents optimization from deviating excessively from the strong priors provided by the virtual coordinate system.
[0029] Using weighted summation to jointly guide the direction of pose optimization is more effective.
[0030] The multimodal regularization weight allocation rules in step S5 are as follows: when the feature points are greater than or equal to 100, γ=0.9; when the feature points are less than 100 or greater than or equal to 50, γ=0.5; when the feature points are less than 50, γ=0.1; where γ represents the weight.
[0031] The parameterized configuration in this embodiment is as follows:
[0032] The operation process of this embodiment is as follows: First, perform synchronous data acquisition: trigger synchronous shooting with the circular studio camera (uniform illumination from a line light source). Load the historical pose library (≥3 poses for the same scene).
[0033] Then, pose initialization is performed using Python tools.
[0034] Part of the code is as follows: virtual_poses = align_to_ring(hist_poses, ring_axis) # Project to the ring axis optimized_poses = depth_aware_refine(virtual_poses, mono_depth)#Marigolddepth optimization(@ref) Finally, iterative optimization is performed, using 3DGS to render the mask and depth map; this is a loop process that gradually optimizes the pose until convergence.
[0035] The specific steps are as follows: A1: Render the depth map and mask using the current pose and 3D Gaussian model (3DGS).
[0036] A2: Calculate the multimodal loss L_reg and update (fine-tune) the camera pose through backpropagation.
[0037] A3: Iterate every 5 times: recalculate the curvature based on the latest geometric model and update the mask once, because the understanding of the object shape is more accurate during the optimization process, and the mask also needs to be updated to adapt to the new geometry.
[0038] The technical renderings are as follows:
[0039] Single-frame pose optimization takes ≤0.5s (meets the real-time requirement of 30fps).
[0040] This scheme achieves the following for sparse viewpoint 3D reconstruction pose optimization: 1. Physical constraint breakthrough feature dependence: virtual coordinate system enables zero-overlap pose reuse (no SIFT / ORB feature matching required); 2. Dynamic interference suppression: curvature-color joint mask solves boundary jitter caused by clothing wrinkles; 3. Lightweight real-time optimization: multimodal regularization replaces NeRF-like implicit coding, improving inference speed by 13 times.
[0041] The foregoing has shown and described the basic principles and main features of the present invention, as well as its advantages. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from its spirit and scope. All such changes and modifications fall within the scope of the present invention as claimed, which is defined by the appended claims and their equivalents.
Claims
1. A sparse-viewpoint 3D reconstruction pose optimization method, characterized in that: Includes the following steps: S1. Input a sparse image; S2. Construct a circular virtual coordinate system; S3, Depth-guided pose projection; S4. Generate a curvature-color joint mask; S5, multimodal differentiable rendering optimization; S6. Optimize pose output.
2. The sparse viewpoint 3D reconstruction pose optimization method according to claim 1, characterized in that: The design parameters of the ring virtual coordinate system in step S2 are as follows: the mean value of the ground plane normal vector is the Z-axis, the center of the scene bounding box is the origin, and the longest spatial diagonal is the X-axis.
3. The sparse viewpoint 3D reconstruction pose optimization method according to claim 1, characterized in that: In step S3, pose projection enhancement is achieved using the following method: Marigold monocular depth consistency constraints are introduced to optimize the projection matrix, using the following formula: ； Where k is the current viewpoint, P is the 3D point set, N is the number of configurable viewpoints, and D is the depth map.
4. The sparse viewpoint 3D reconstruction pose optimization method according to claim 1, characterized in that: The curvature-color joint mask in step S4 is as follows: Using Python, the LAB color space and local curvature features are combined, resulting in the following features: mask=(LAB_chroma>130)&(LAB_lightness<20)&(curvature>0.25).
5. The sparse viewpoint 3D reconstruction pose optimization method according to claim 1, characterized in that: In step S5, multimodal regularization integrates the depth-normal-virtual viewpoint three constraints of LM-Gaussian, with the following characteristics: Lreg=0.7Ldepth+0.2Lnormal+0.1Lvirtual.
6. The sparse viewpoint 3D reconstruction pose optimization method according to claim 2, characterized in that: The virtual viewpoint is generated in step S2 as follows: equally spaced virtual viewpoints are generated on a circular plane.
7. The sparse viewpoint 3D reconstruction pose optimization method according to claim 5, characterized in that: The multimodal regularization weight allocation rules in step S5 are as follows: when the feature point is greater than or equal to 100, γ=0.9; when the feature point is less than 100 or greater than or equal to 50, γ=0.5; when the feature point is less than 50, γ=0.1; where γ represents the weight.