A three-dimensional boundary representation reconstruction method based on multi-view image application dome type curved surface fitting

Abstract

The application discloses a three-dimensional boundary representation reconstruction method based on a dome type curved surface fitting of multi-view images, which comprises the following steps: 1, extracting edge masks and patch masks from multi-view images; 2, using a two-dimensional Gaussian sputtering to construct a Gaussian representation with learnable features, each Gaussian cell of the Gaussian representation is a directional elliptical disc aligned with a surface, and edge features and patch features are additionally added; 3, using a two-stage learning framework to train the Gaussian representation according to the edge masks, patch masks and multi-view images obtained in step 1; 4, performing elliptical sampling according to the Gaussian distribution obtained in step 3 to generate a point cloud aligned with a real surface and having edge and patch labels; and 5, applying specific cell fitting and dome type curved surface fitting to restore a parameterized curved surface according to the point cloud obtained in step 4, gradually extracting edges and corner points through geometric intersection lines, and obtaining a clear CAD model through pruning and fusion optimization.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional boundary representation, and to a three-dimensional boundary representation reconstruction method based on multi-view images and dome-like surface fitting. Background Technology

[0002] Boundary representation (B-rep) represents a 3D solid through its explicit boundaries, including trimmed corners, edges, and faces.

[0003] Modeling is then performed. Recovering B-rep representations from unstructured data is a challenging and valuable task in the field of computer vision and graphics.

[0004] In recent years, advancements in deep learning technology have significantly improved the ability to reconstruct 3D geometric shapes, but it still mainly relies on...

[0005] It relies on dense and clean point cloud data, but is difficult to generalize to new forms. Currently, most CAD reconstruction research focuses on...

[0006] These methods take dense point clouds as input, first segmenting them to extract regions defined by edges (i.e.,

[0007] Point clouds are labeled with "patch" data and then parametrically reconstructed. Early learning-based methods introduced supervised primitive segmentation networks. Subsequent research improved this paradigm by learning feature embeddings for point grouping. Recent extensions combine edge and surface features to improve reconstruction quality and consistency with the fitted geometry. Despite these advances, existing CAD reconstruction techniques remain constrained by two key challenges: the high cost of acquiring high-quality point clouds and the heavy reliance on extensive manual annotation. In contrast, image data is more readily available and scalable, but a significant gap remains between image data and parametric 3D modeling.

[0008] In recent years, with the development of neural rendering technologies such as Neural Radiation Field (NeRF) and 3D Gaussian Sputtering (3DGS)...

[0009] With the emergence of [these technologies], multi-view 3D reconstruction has received increasing attention. These methods can directly reconstruct high-fidelity [3D structures] from images.

[0010] The geometry and appearance of truth. However, how to further extract structured information from these implicit or explicit neural representations?

[0011] The development of editable parametric CAD models remains a problem that urgently needs to be solved. Summary of the Invention

[0012] Purpose of the invention: The technical problem to be solved by the present invention is to address the shortcomings of the prior art by providing a three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting.

[0013] To address the aforementioned technical problems, this invention discloses a three-dimensional boundary representation reconstruction method based on multi-view images and dome-like surface fitting, comprising the following steps:

[0014] Step 1: Extract edge masks and patch masks from multi-view images;

[0015] Step 2: Construct a Gaussian representation with learnable features using two-dimensional Gaussian sputtering; each Gaussian element of the Gaussian representation is an oriented elliptical disk aligned with the surface, with additional edge and patch features.

[0016] Step 3: Based on the edge mask, patch mask, and multi-view images obtained in Step 1, a two-stage learning box is used.

[0017] The Gaussian representation is trained.

[0018] Step 4: Based on the Gaussian distribution obtained in Step 3, perform elliptic sampling to generate a distribution aligned with the real surface and featuring...

[0019] Point cloud of edge and patch labels;

[0020] Step 5: Based on the point cloud obtained in Step 4, apply specific primitive fitting and dome-like surface fitting to restore the parametric surface. Gradually extract edges and corners through geometric intersections, and after pruning and fusion optimization, obtain a clear CAD model.

[0021] The step 1, which involves extracting edge masks and patch masks from multi-view images, includes:

[0022] An edge detector is used to extract edge masks from images at each viewpoint;

[0023] A segmentation model is used to generate a face mask for each viewpoint.

[0024] Step 2 specifically includes:

[0025] Step 2-1: Using two-dimensional Gaussian sputtering as the basic framework, the three-dimensional scene is represented as a set of learnable two-dimensional Gaussian primitives; each of the two-dimensional Gaussian primitives is represented by its center point. Two principal tangent vectors and scaling factor Parametric definition; for a point u = (u,v) on the local tangent plane, its two-dimensional Gaussian distribution function is defined as:

[0026]

[0027] Where u and v are the two-dimensional coordinates of point u on the local tangent plane, respectively representing the coordinates along the tangent plane. The relative offset of the direction (dimensionless), scaled by a scaling factor Convert to actual spatial distance;

[0028] Step 2-2, for each Gaussian element Extended learnable feature embeddings This forms an enhanced Gaussian representation:

[0029] =

[0030] in, The coordinates of the center point, These are two mutually perpendicular principal tangent vectors in a 2D local coordinate system. For the corresponding scaling factor, For opacity, For color, Learnable high-dimensional feature vectors (i.e., adding a randomly initialized mathematical tensor to each Gaussian element for subsequent training) are used to encode edge and patch semantic information;

[0031] Steps 2-3: Render the 2D Gaussian distribution sputtered to screen space using weighted alpha blending.

[0032]

[0033] in, Represents the two-dimensional pixel coordinates in screen space (image plane). Represents screen coordinates The two-dimensional coordinates (u,v) are mapped inversely to the local tangent plane, where N represents the total number of all two-dimensional Gaussian primitives in the scene, and c(x) represents the final rendered color value at pixel x in screen space.

[0034] The weighted alpha blending process is related to color and edge value parameters. It calculates the color and edge value of each Gaussian unit in 3D space using a specific formula to obtain a two-dimensional image (RGB image and edge image, where the value of each pixel after calculation comes from the combination of the color and edge value parameters of the Gaussian unit).

[0035] Step 3 specifically includes:

[0036] Step 3-1, the first stage, jointly learn the 3D geometric structure and edge semantics of Gaussian representation;

[0037] Step 3-2, the second stage, freezes the geometric parameters and edge features learned in the first stage, and optimizes only their high-dimensional feature vectors. By comparing and learning, the surface features of Gaussian elements are optimized to obtain surface instance labels.

[0038] The first stage of learning described in step 3-1, which jointly learns the 3D geometric structure and edge semantic features, includes:

[0039] During the sputtering rendering process in steps 2-3, for each Gaussian element... Assign a scalar edge value ∈ [0,1] to encode edge semantics, the edge values ​​are rendered in the same way as colors, and the edge map is rendered by alpha blending weighted by transparency:

[0040]

[0041] =

[0042] in, It is the cumulative visibility weight of the i-th Gaussian element along the line of sight. It is its learnable marginal probability. The opacity of the i-th Gaussian edge is the alpha-weighted cumulative sum of the Gaussian edge values.

[0043] The overall loss function in the first stage for:

[0044] = +

[0045] in, To supervise the loss function of geometric reconstruction, To supervise the loss function for edge prediction, This is the balance coefficient, and its value ranges from [0,1].

[0046] The geometry part follows the optimization objective of the original Gaussian sputtering:

[0047]

[0048] in, and Supervised by the original image;

[0049]

[0050] in This represents the predicted image obtained by weighted alpha blending rendering; Represents a real image (Ground Truth); and These represent the local means of the predicted and actual plots, respectively. and These represent the local variances of the predicted and actual plots, respectively. This represents the local covariance between the predicted and actual plots. and is the numerical stability constant, where =0.01、 =0.03 is the default parameter. Indicates the dynamic range of pixels (e.g., in an 8-bit image). =255).

[0051]

[0052] in This represents the calculation result of the overall L1 loss function; This represents the predicted image obtained by weighted alpha blending rendering; Represents the ground truth image corresponding to the predicted image. and These represent the coordinate indices of image pixels in the horizontal and vertical directions, respectively. Indicates the width of the image (number of pixels); Indicates the height of the image (number of pixels); Represents the color channel index of the image; Indicates the total number of color channels in an image (e.g., C=3 for an RGB image); The total number of pixels in the image is represented and used to normalize the loss value to the mean; | represents the absolute value operator, used to calculate the absolute error between the predicted value and the true value for a single pixel or a single channel.

[0053] Edge-aware loss is used to determine whether a Gaussian object is located at an object boundary.

[0054] =

[0055] in This represents the set of Gaussian meta-indexes located in the boundary region of an object; Represents a single Gaussian meta-index in set E; Indicates the first The actual image pixel values ​​corresponding to each boundary Gaussian element; Indicates the first The model renders and predicts the image pixel values ​​corresponding to each boundary Gaussian element;

[0056] By constraining the geometric position and edge value of each Gaussian element through the total loss function in the first stage, the accurate and reasonable geometric position and edge value of each Gaussian element can be obtained after a certain number of training rounds, which is to say that the three-dimensional geometric structure and edge semantic features have been learned.

[0057] The second stage of learning, as described in step 3-2, introduces the triplet loss function. Let's learn about noodle examples and tags:

[0058] Based on the geometric parameters and edge features obtained in the first stage, each Gaussian element is... Assign a high-dimensional feature vector Using encoded patch information, the rendering method for each feature channel during sputtering rendering is similar to that of the edge values ​​in the first stage. The feature vector of each pixel is obtained by accumulating alpha blending weighted by opacity. ;

[0059] Since the two-dimensional patch masks extracted from different viewpoints may lack consistency, this invention employs a contrastive learning strategy for feature learning, where the cosine distance between features is used to measure the similarity between samples:

[0060] For pixels ,That The normalized eigenvectors are denoted as The distance between two points is defined as:

[0061]

[0062] To learn discriminative patch features, after excluding background (mask region instance label is 0) and corner noise (mask region includes bottom right corner pixels), two-dimensional mask regions with more than a minimum threshold (set to 128) and less than a maximum threshold (set to 10000) of covered pixels are considered. For each valid two-dimensional mask region... Randomly sample an anchor point A positive sample { }, and sample a set of negative samples from other mask regions. From the candidate set The sample closest to the anchor point is selected as the most difficult negative sample. Triple loss function Defined as:

[0063] =

[0064] Where T is the set of sampled triples, m is the margin hyperparameter, and d(·, ·) is the cosine distance between features. This loss function makes features within the same patch more compact and features between different patches more separated, thus obtaining clear patch instance labels.

[0065] The second stage of learning described in step 3-2 simultaneously introduces a contrastive learning loss function to optimize the discriminativeness of patch features:

[0066] Based on the above normalized feature vectors and cosine distance The contrastive learning loss function consists of two parts: intra-mask consistency loss and inter-mask separation loss.

[0067] Intramask consistency loss: for each valid 2D patch mask region that is not background Randomly sampled pixel set Calculate the mean cosine distance of all pixel pairs within the set:

[0068]

[0069] For all valid mask regions The average value is used to obtain the overall consistency loss within the mask:

[0070]

[0071] in The optimization objective of this loss term is to minimize the number of effective mask regions. This makes the features of the same facet compact;

[0072] Inter-mask separation loss: Samples a set of pixels across the mask from all non-background pixels. For any two different mask regions , corresponding pixel set and pixel set Calculate the mean cosine distance across mask pixel pairs and construct the loss term:

[0073]

[0074] For all cross-mask pairs The average value is used to obtain the overall mask separation loss:

[0075]

[0076] The optimization objective of this loss term is to minimize... This allows for the separation of different facet features.

[0077] The final contrastive learning loss function is: In the formula, λ=0.1 is the weighting coefficient of the inter-mask separation loss. This loss function is achieved through simultaneous constraints. and This enables discriminative learning of patch features, where "compressed same-mask and separated different-mask".

[0078] The second stage of learning described in step 3-2 employs a dynamic smoothing weight mechanism to fuse the triplet loss and the contrastive learning loss:

[0079] Let the current training iteration number be t, and the total number of iterations be T. Define the iteration progress. The training process is divided into three stages based on α, and the weights of the contrastive learning loss are dynamically adjusted. With triplet loss weights :

[0080] Base period ( ): Optimize the basic consistency of patch features solely through contrastive learning loss;

[0081] transition period ( Define the transition coefficient. The cosine annealing strategy is used to smoothly transition the weights. , , where π is the constant of pi, this method avoids training oscillations caused by sudden changes in weights;

[0082] Strengthening period ( ): The key is to enhance the discriminative power of patch instance labels through triplet loss.

[0083] The final total loss function is: In the formula The aforementioned triplet loss is used. This dynamic weighting mechanism enables a gradual transition from basic feature learning to instance discrimination learning, improving model training stability and reconstruction accuracy.

[0084] Step 4, which describes elliptic sampling based on the trained Gaussian units, includes:

[0085] For each Gaussian element, sample its center point;

[0086] For ellipses whose major-to-minor axis ratio is greater than a preset threshold, take n additional points (e.g., 4 points) along the surface of the ellipse.

[0087] The edge values ​​and patch instance labels corresponding to Gaussian elements are assigned to all sampling points, where the edge values ​​are binarized by setting a threshold and then assigned to all sampling points.

[0088] Perform patch instance labeling post-processing on the point cloud formed by all sampled points, specifically including the following steps:

[0089] Labeled point cloud grouping: Group all sampled points according to the patch instance label value to obtain several point cloud clusters. ,in This represents the set of sampling points with label value k, i.e. ;

[0090] Point cloud downsampling: for each point cloud cluster If it contains a number of sampling points Then randomly downsample to From these points, we obtain the downsampled point cloud cluster. ;

[0091] Chamfer distance calculation: Calculates the distance between any two point cloud clusters. , The Chamfer distance (CD distance) between (i!=j) is defined as:

[0092]

[0093] In the formula, |||2 is the L2 norm. , These are the number of points in the two cloud clusters, respectively.

[0094] Label clustering and merging: A distance matrix is ​​constructed based on the CD distance of all point cloud clusters. Hierarchical clustering (average linking method) is used to cluster the labels, and a CD distance threshold is set. The distance between CD is less than The labels corresponding to point cloud clusters are merged into the same label;

[0095] Label remapping: The labels after clustering and merging are renumbered with consecutive integers to obtain the final unified patch instance labels. ,in This represents the total number of tags after merging.

[0096] The primitive fitting described in step 5 includes:

[0097] Step 5-1: Define three primitive models: plane, cylinder, and sphere, and represent them as parametric surfaces for subsequent geometric fitting and optimization; for each point cloud cluster with patch instance labels, use the random sampling consensus algorithm.

[0098] (RANSAC) is used to fit the above primitive models respectively.

[0099] Step 5-2: Calculate the geometric intersection between adjacent primitives to obtain the corresponding straight lines and curves in geometric space.

[0100] For two planes The direction of their intersection is And solve using least squares Obtain the anchor point on the intersection line. Thus, a straight line is obtained. .

[0101] Subsequently, these intersections are constrained using the edge point cloud, and their parameter range is determined by projecting the edge points onto straight lines and curves. This allows us to extract valid line segments and curve segments as edges in the boundary representation (B-rep).

[0102] When a plane intersects a cylinder, the plane is... The cylinder is ,in Represents the plane normal vector. Represents the plane constant term. This represents the direction vector of the cylinder's axis. This represents a point on the axis of the cylinder. This represents the radius of the cylinder.

[0103] When a plane intersects a cylinder, the line of intersection is a circle embedded within the plane Π. Its center... The projection of the cylinder's axis onto plane Π is calculated as follows: ,in Let be any point on the plane Π. The radius of the intersection circle and the radius of the cylinder... Maintain consistency.

[0104] The edge points are then projected onto the plane. The data is then filtered using planar and radial distance thresholds to retain valid points that form the circular curve segment.

[0105] When a plane intersects a sphere, the plane is... , sphere ,in: Indicates the center of the ball. This represents the radius of the sphere.

[0106] When a plane intersects a sphere, the line of intersection is also embedded in the plane. The inner circle. Its center. For the center of the ball In plane Orthographic projection on: ,radius It is determined by both the radius of the sphere and the distance from the center of the sphere to the plane: .in It is the projection of the center of the sphere onto the plane. It is the radius of the intersection circle.

[0107] After projecting the edge points onto plane Π, valid points are retained to form the circular curve segment by filtering through a distance threshold.

[0108] Step 5-3: Using the intersection line of two adjacent planes obtained in Step 5-2, candidate corner points are obtained through the intersection of the two lines, and then these candidate points are clustered to generate the final corner points;

[0109] Step 5-4: Under the constraints of the extracted edges and corner points, trim and refine each parametric surface:

[0110] Projecting planar primitives onto a local coordinate system The cylindrical primitives use cylindrical coordinates. The spherical primitives use spherical coordinates. Within their respective parameter domains, the intersection lines are connected into closed boundary loops based on shared corner points. Through signed area calculation and inclusion relationship analysis, the outer boundary loop and the inner hole loop are distinguished, and the effective area of ​​the surface is clarified.

[0111] The plane is constrained triangulated, and the cylinder and sphere are meshed in the parameter domain to generate watertight patches.

[0112] Finally, all reconstructed patches are merged into a unified CAD mesh, vertex normals are aligned, topology adjustments are made through Boolean operations, and finally assembled into a watertight boundary representation (B-rep) CAD model.

[0113] The fitting parameterized primitive model mentioned in step 5-1 is specifically as follows: the Random Sample Consensus Algorithm (RANSAC) is used to fit one or more of the plane, cylinder and sphere respectively. By iteratively sampling the minimum point set, fitting candidate models and evaluating the proportion of interior points based on geometric distance metrics, the model with the best fitting effect is selected as the parameterized surface of the patch, thereby achieving robust parameter estimation.

[0114] The parametric representation and fitting core formulas for planes, cylinders, and spheres are defined as follows:

[0115] The parametric equation of the plane is: After normalization, it is expressed as , where the normal vector It is a unit vector. This is the offset of the plane from the origin;

[0116] For point Its perpendicular distance to the plane is: ,satisfy Points are identified as interior points, and the plane corresponding to the maximum number of interior points is the optimal fitting result.

[0117] The parameter set of a cylinder is defined as follows: ,in These are the centers of the upper and lower bases of the cylinder, respectively. Let be the radius of the cylinder. Projecting the point cloud onto a plane perpendicular to the candidate axis yields two-dimensional coordinates. Solving for circle parameters using least squares :

[0118]

[0119] The center and radius of the projected circle are obtained from the above parameters:

[0120] Satisfy radial distance error The point is determined as an interior point, and the cylinder height h is estimated by combining the projection range of the axis, and finally the cylinder parameters in the world coordinate system are obtained.

[0121] The parameterized equation for a sphere is: , among which the center of the ball The equation has a radius of r. After expanding and rearranging the equation into a linear form, the following system of linear equations can be solved using the least squares method:

[0122] in .

[0123] The final formulas for calculating the center and radius of the sphere can be obtained from the solutions of the above system of equations:

[0124]

[0125] For any point in the point cloud Its residual relative to the fitted sphere is:

[0126]

[0127] satisfy Points with a value less than τ (τ=0.01) are considered interior points, and the sphere corresponding to the maximum number of interior points is the optimal fitting result.

[0128] The dome-like surface fitting described in step 5 includes the following steps:

[0129] Step 5-2-1, Point cloud coordinate system alignment: Calculate the range of the point cloud obtained in step 4 in the three coordinate axes x, y, and z, determine the dimension of the smallest range as the vertical direction of the dome, rotate the point cloud so that this dimension is aligned with the Z-axis of the coordinate system, and make the fitting plane approximately parallel to the XY plane.

[0130] Step 5-2-2, Control point mesh generation: Generate a regular control point mesh covering the point cloud area on the rotated point cloud;

[0131] Step 5-2-3, Determining the Concavity and Convexity of the Dome: Find the maximum and minimum points of the point cloud in the vertical direction, calculate the center of the tangent plane, and compare the distances of the maximum and minimum points to the center of the tangent plane. If the minimum point is closer, the dome is determined to be concave towards the center (the vertical value first decreases and then increases). If the maximum point is closer, the dome is determined to be convex towards the outside (the vertical value first increases and then decreases). This provides a basis for the parameterized direction adaptation of the dome curvature in the subsequent Bezier patch fitting.

[0132] Step 5-2-4, Bézier surface patch fitting: Divide the control point grid into multiple non-overlapping 4×4 control point blocks according to the initial index interval of 3. Adjust the parameterization direction of the Bézier surface based on the concavity and convexity determination result in step 5-2-3. Perform two-parameter direction (u / v) parameterization fitting on each control point block using the cubic Bézier surface formula. Generate the corresponding surface patch lattice by sampling at equal intervals along the two parameter directions.

[0133] Step 5-2-5, Surface patch combination and triangular meshing: Piece together all surface patch lattices in the order of mesh to form a complete fitted surface lattice, divide each quadrilateral unit in the lattice into triangles, and construct a triangular mesh surface;

[0134] Step 5-2-6, Coordinate System Restoration: Transform the triangular mesh surface back to the original coordinate system using the inverse of the rotation matrix in Step 5-2-1, and output the final dome-fitted surface.

[0135] The control point mesh generation described in step 5-2-2 includes the following steps:

[0136] Step 5-2-2-1, Calculate the local tangent plane coordinate system of the point cloud: Calculate the centroid of the point cloud. ,in Indicates the first The coordinate vector of a point, The total number of points; decentralized point cloud is obtained Calculate the covariance matrix ; through eigenvalue decomposition Three feature vectors were obtained. and the corresponding eigenvalues The eigenvector corresponding to the smallest eigenvalue As the normal vector of the tangent plane, the eigenvectors corresponding to the largest and second largest eigenvalues. The two orthogonal tangent axes that form the tangent plane constitute a local tangent plane coordinate system;

[0137] Step 5-2-2-2, Calculate the bounding box of the tangent plane: Project the point cloud onto the tangent plane coordinate system to obtain the projected coordinates of each point on the two tangent axes. , Calculate the maximum value of the projected coordinates. Minimum value as well as , And add preset proportions in each direction. The margin is used to expand the enclosure: , ,right Similarly, the coordinates are as follows, where The value range is 0.01 to 0.05, with 0.02 being preferred, to ensure that the control point grid completely covers the point cloud area;

[0138] Step 5-2-2-3, Generate regular mesh nodes: Within the bounding box of the tangent plane, generate regular mesh nodes according to the preset mesh resolution.

[0139] (Preferred) The grid is uniformly divided to generate a series of two-dimensional grid node coordinates:

[0140]

[0141] ,

[0142] in, , , , ;

[0143] Step 5-2-2-4: Map the mesh nodes back to 3D space: Convert the coordinates of each 2D mesh node to 3D world coordinates. Construct a KD-tree index for the point cloud, for each 3D coordinate point. Search for the nearest neighbor in the point cloud and use the 3D coordinates of the nearest neighbor as the control point value of the grid node. This ultimately forms a regularly arranged three-dimensional control point mesh with a size of [size missing]. .

[0144] The Bézier surface patch fitting described in step 5-2-4 includes the following steps:

[0145] Step 5-2-4-1, Control point mesh patching: Let the control point mesh be... The grid is divided with a step size of 3 starting indexes along both the row and column directions, and the starting index for the row-direction partitioning is [index missing]. The starting index for column-wise sharding is For each Cut Within range Control point block Ensure that all segments are non-overlapping and completely cover the entire control point grid;

[0146] Step 5-2-4-2, Cubic Bézier surface formula fitting: For each Control point block The core formula of cubic Bézier surface is adopted. Perform fitting, where: for The cubic parameter vector of the direction, ; for The cubic parameter vector of the direction, ; It is a cubic Bessel basis matrix. ; for The transpose of the matrix; For the current partition Control point blocks, each element of which is a three-dimensional coordinate vector, and matrix multiplication is performed on components;

[0147] Step 5-2-4-3, Generate surface patch matrix by equidistant sampling: Set the number of equal sampling divisions. Then the parameter step size ,along Direction parameter value ,along Direction parameter value All Substituting into the formula in step 5-2-4-2, we can calculate each... Control point block corresponding Dimensional curved surface lattice (preferred) );

[0148] The root mean square of the fitting residuals between the surface patch lattice generated by the above steps and the original dome cloud is... Furthermore, the various curved surfaces are smoothly connected without any seams, providing a high-precision foundation for the subsequent construction of the complete dome surface.

[0149] Beneficial effects:

[0150] This invention can directly recover the parametric boundary representation of 3D entities from 2D images, eliminating the reliance on dense, high-quality point cloud data and extensive manual annotation found in previous techniques. Compared to traditional methods that use the same multi-view images to reconstruct point clouds as input, this invention exhibits significant performance advantages and can be used for downstream tasks such as 3D reconstruction and reverse engineering. Attached Figure Description

[0151] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0152] Figure 2 This is a schematic diagram of the final three-dimensional boundary representation reconstruction result in Example 1.

[0153] Figure 3 This is a schematic diagram of the final three-dimensional boundary representation reconstruction result in Example 2. Detailed Implementation

[0154] This invention proposes a 3D boundary representation reconstruction method based on multi-view images and dome-like surface fitting. This method extracts geometric, edge, and patch information from multi-view images through two-stage Gaussian sputtering learning, and assembles a parametric CAD model using constraint-guided primitive fitting, overcoming the dependence of traditional methods on high-quality point clouds and extensive manual annotation. The 3D boundary representation (B-rep) reconstruction method based on multi-view images described in this invention includes the following steps:

[0155] Step 1: 2D Label Extraction: Extract edge masks and patch masks from multi-view images to prepare for subsequent 3D label extraction.

[0156] Learning provides supervisory signals.

[0157] Step 2: Constructing a feature-aware Gaussian representation: Representing the 3D scene using 2D Gaussian sputtering (2DGS).

[0158] It is a set of Gaussian primitives with learnable features, each primitive containing position, orientation, scaling, color, opacity, and edge and patch features.

[0159] Step 3: Two-stage Gaussian training: The first stage jointly learns the geometric structure and edge semantics; the second stage freezes the geometric parameters and uses a triplet loss function. We learn the patch instance labels, optimize patch features through comparative learning, and obtain clear 3D patch segmentation.

[0160] Step 4: Gaussian Sampling to Generate Labeled Point Clouds: Sampling is performed based on the shape of Gaussian elements, and edges and...

[0161] The sampling points are labeled with patch labels, resulting in a dense point cloud with semantic labels.

[0162] Step 5: Primitive Fitting and B-rep Reconstruction: Parametric primitive fitting and dome-like surface fitting are performed on the point cloud to extract edges and corners. Through constraint optimization and topology adjustment, the points are assembled into a watertight boundary representation CAD model.

[0163] Example 1:

[0164] Taking a real-world 3D printed part model as an example to illustrate the application of this method in practical situations, suppose we need to use this invention to reconstruct the three-dimensional boundary representation of this mechanical part for downstream tasks, then:

[0165] Step 1: 2D Label Extraction: Take 50 multi-view images around the 3D printed part, and then use the PidNet edge detector to extract the edge mask of each image. Then, use the Segment Anything Model (SAM) to generate the patch mask of each view image.

[0166] Step 2: Construct a feature-aware Gaussian representation: Initialize a set of 2D Gaussian primitives, each Gaussian consisting of a center point,

[0167] Two tangent vectors, scaling factor, color, opacity, and learnable edge scalars and 16-dimensional feature vectors The Gaussian initial position is randomly initialized, and the remaining parameters are randomly assigned values.

[0168] Step 3: Two-stage Gaussian training:

[0169] Step 3-1, First Stage (Geometry and Edge Learning): Training is performed using 50 images for 30,000 iterations. The optimizer updates all Gaussian parameters simultaneously. Loss Function = + Simultaneously supervise the consistency between the rendered image and the real image, as well as the consistency between the rendered edge map and the PidNet edge mask. After training, accurate object geometric contours and sharp edge Gaussian distributions are obtained.

[0170] Step 3-2, Second Stage (Patch Segmentation Learning): Freeze all Gaussian geometric parameters and optimize only the 16-dimensional feature vector. Continue training for 15,000 iterations, using triplet loss. Contrastive learning is employed, using a dynamic smoothing weight mechanism to fuse triplet loss and contrastive learning loss. Anchor points and positive samples are randomly sampled from each 2D facet mask, while negative samples are sampled from other faces, bringing similar features closer together and pushing dissimilar features further away. After training, Gaussian representations of faces belonging to the same part (such as a plane or cylinder) have similar feature vectors.

[0171] Step 4: Gaussian Sampling to Generate Labeled Point Clouds: Traverse all Gaussian elements. For each Gaussian, sample its center point; if the scaling factor of the Gaussian is... and If the ratio is greater than 3 (i.e., the ellipse is slender), then four additional points are uniformly sampled on its elliptical surface. The edge values ​​of the Gaussian are then... All sampling points are assigned with (threshold binarization) and patch labels (discrete labels obtained by clustering based on CD distance). The final result is a dense point cloud of approximately 20,000 points with edge and patch labels.

[0172] Step 5: Constraint-Guided B-rep Fitting: Group the point cloud by patch label. For each point cloud cluster, use RANSAC to attempt fitting planes, cylinders, and spheres respectively, selecting the optimal primitive type and parameters based on the proportion of interior points. For each point cloud cluster, simultaneously use dome-type surface fitting and compare it with the primitive fitting results, selecting the one with smaller error as the final fitting result. Calculate the intersection equation of adjacent primitive fitting surfaces (e.g., two planes). Project the edge point cloud onto this intersection line, determining the start and end points of line segments based on the distribution of projection points, forming valid edges. Calculate the intersection point of three edges (e.g., three planes intersecting at one point) as corner points. Finally, using these parameterized surfaces, line segments, and corner points, generate a watertight boundary representation CAD model through Boolean operations such as trimming and stitching.

[0173] Through the above steps, this embodiment successfully reconstructed multi-view images of 3D printed parts captured in a real-world scene into a complete, watertight 3D boundary representation CAD model. For example... Figure 2 The first row and first column represent the actual captured image. The first row and second column represent the edge mask and patch mask extracted from the multi-view image after processing in step 1. The second row and first column represent the patch instance segmentation obtained after the second stage training in step 3. The second row and second column represent the CAD model obtained by the final combination. Different colors represent different patch instances.

[0174] This part contains various geometric features such as planar and cylindrical surfaces, and faces challenges in real-world shooting environments, including varying lighting, cluttered backgrounds, and areas with weak textures. Despite these challenges, the present invention can still robustly extract accurate geometric structure and semantic information from images, ultimately outputting a parametric model that can be directly used for CAD editing. This embodiment fully demonstrates the feasibility and practicality of the invention in real-world reverse engineering, digital twins, and rapid prototyping scenarios. Compared to traditional methods, this invention eliminates the need for expensive 3D scanning equipment, achieving high-precision structured reconstruction using only images captured by ordinary consumer-grade cameras, significantly lowering the technical application threshold and possessing significant creative and industrial application value.

[0175] Example 2:

[0176] Taking a typical mechanical part with a dome-like surface from the ABC-NEF dataset as an example, suppose we need to use this invention to reconstruct the 3D boundary representation of this mechanical part for downstream tasks, then:

[0177] Step 1: First, acquire 50 multi-view images of the part, and use the PidNet edge detector to extract edge masks from each image. Input the 50 multi-view images into the SAM model to generate a patch mask for each image.

[0178] Step 2: Initialize a set of 2D Gaussian primitives. Each Gaussian primitive consists of a center point, two tangent vectors, a scaling factor, color, opacity, and a learnable edge scalar. and 16-dimensional feature vectors The Gaussian initial position is randomly initialized, and the remaining parameters are randomly assigned values.

[0179] Step 3: Using 50 multi-view RGB images and corresponding edge mask images as supervision, the first stage of training is performed to learn geometric and edge semantics. The training iterations are 30,000 times, with the optimizer simultaneously updating all Gaussian parameters. The loss function consists of geometric reconstruction loss and edge perception loss, supervising the consistency between the rendered image and the real image, as well as the consistency between the rendered edge map and the edge mask. After training, accurate object geometric contours and edge Gaussian distributions are obtained. The second stage of training, using 50 multi-view RGB images and corresponding patch masks, is then performed to learn patch segmentation. All Gaussian geometric parameters are frozen, and only the 16-dimensional feature vector is optimized. Triple loss is used for contrastive learning, ensuring that Gaussian features within the same patch are similar, while features between different patches are different, thus obtaining clear patch segmentation.

[0180] Step 4: Traverse all Gaussian elements and sample the center point of each Gaussian; for slender ellipses (the ratio of the major axis to the minor axis is greater than 3), sample four additional points. Assign the edge values ​​(binarized with a threshold of 0.8) and patch labels (discretized by clustering based on a CD distance threshold of 0.05) of the Gaussian elements to all sampled points, ultimately obtaining a dense point cloud of approximately 20,000 points, each point with edge and patch labels.

[0181] Step 5: Group the point cloud by patch label. For each group, fit a plane, cylinder, or sphere using RANSAC, select the optimal primitive, and simultaneously fit a dome-like surface. Compare the fitting results with the primitive fitting results, and select the one with the smaller error as the final fitting result. Then calculate the intersection lines of adjacent primitives, extract effective edges using edge point cloud constraints, and obtain corner points through intersection point clustering. Finally, under the constraints of edges and corner points, trim and optimize the surface, and assemble it into a watertight boundary representation CAD model through Boolean operations.

[0182] The final three-dimensional boundary representation reconstruction result of the mechanical part obtained from the above steps is as follows: Figure 3 As shown, the first column represents the patch instance segmentation obtained after the second stage of training in step 3, the second column represents the CAD model obtained by the final combination, and the third column is the GT model, where different colors represent different patch instances.

[0183] In Example 2, the fitting process for industrial parts fully demonstrates the core advantages of this method. First, through the automated coordinate system alignment and adaptive control point generation mechanism in step 5-2-2, this method can stably generate a control point mesh that fits the geometry of the original point cloud even when the point cloud density is uneven, there is noise, or there are local missing points. Example 2 requires no manual intervention or extensive annotation throughout the process, which significantly reduces the dependence of traditional methods on manually adjusting parameters such as control point density and position, saving a lot of time and economic costs.

[0184] Secondly, this method is customized and optimized for dome-shaped surfaces that are widely found in industrial parts. The convex and concave shape of the dome is pre-identified through the concavity and convexity determination in step 5-2-3, and combined with the piecewise fitting strategy in step 5-2-4, the smooth connection and curvature continuity of each surface piece are ensured. In Example 2, the root mean square of the fitting residual is controlled within 0.50 cm, and the generated triangular mesh can be directly imported into 3D modeling software (such as Rhino software) for parameter editing. This meets the stringent requirements of factory digital modeling for parameter accuracy and model editability, and provides a high-precision CAD model foundation for practical applications such as reverse engineering and CNC machining of industrial parts.

[0185] This invention provides a three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting, characterized in that, Includes the following steps: Step 1: Extract edge masks and patch masks from multi-view images; Step 2: Construct a Gaussian representation with learnable features using 2D Gaussian sputtering. Each Gaussian primitive in the Gaussian representation is an oriented elliptical disk aligned with the surface, with additional edge and patch features. Step 3: Based on the edge mask, patch mask, and multi-view images obtained in Step 1, a two-stage learning box is used. Training the Gaussian representation on the frame: Step 4: Based on the Gaussian distribution obtained in Step 3, perform elliptic sampling to generate a distribution aligned with the real surface and featuring... Point cloud of edge and patch labels; Step 5: Based on the point cloud obtained in Step 4, apply specific primitive fitting and dome-like surface fitting to restore the parametric surface. Extract edges and corners step by step through geometric intersections, and obtain a clear CAD model through pruning and fusion optimization.

2. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting as described in claim 1, characterized in that, Step 2 specifically includes: Step 2-1: Using two-dimensional Gaussian sputtering as the basic framework, the three-dimensional scene is represented as a set of learnable two-dimensional Gaussian primitives; Step 2-2, for each Gaussian element Extended learnable feature embeddings This forms an enhanced Gaussian representation; Steps 2-3: Sputter the two-dimensional Gaussian distribution to screen space using weighted alpha blending.

3. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 2, characterized in that, The first stage of the two-stage learning described in step 3 involves jointly learning the 3D geometric structure and edge semantic features, including: During the sputtering rendering process in steps 2-3, for each Gaussian element... Assign a scalar edge value ∈ [0, 1] to encode edge semantics, edge values ​​are rendered by alpha blending weighted by transparency to render the edge map.

4. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 3, characterized in that, The second stage of the two-stage learning described in step 3 introduces a triplet loss function. Let's learn about noodle examples and tags: During sputter rendering, each feature channel is rendered by accumulating the feature vector of each pixel through opacity-weighted alpha blending. ; For pixels ,That The normalized eigenvectors are denoted as The distance between two points is defined as: Confirm valid two-dimensional mask regions that meet the conditions For each valid two-dimensional mask region Randomly sample an anchor point A positive sample { }, and sample a set of negative sample candidates from other mask regions. From the candidate set The sample closest to the anchor point is selected as the most difficult negative sample. Triple loss function Defined as: = Where T is the set of sampled triples, m is the margin hyperparameter, and d(·, ·) is the cosine distance between features.

5. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 4, characterized in that, The second stage of learning involves simultaneously introducing a contrastive learning loss function to optimize the discriminativeness of patch features. This contrastive learning loss function consists of intra-mask consistency loss and inter-mask separation loss. Intramask consistency loss: for each valid 2D patch mask region that is not background Randomly sampled pixel set Calculate the mean cosine distance of all pixel pairs in the set, and take the mean cosine distance of all valid mask regions to obtain the overall mask consistency loss; Inter-mask separation loss: Samples a set of pixels across the mask from all non-background pixels. For any two different mask regions, calculate the mean cosine distance between the cross-mask pixel pairs, and take the mean of the cosine distance between all cross-mask pairs to obtain the overall mask separation loss.

6. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 5, characterized in that, The second stage of learning employs a dynamic smoothing weight mechanism to fuse triplet loss and contrastive learning loss. The training process is divided into three stages based on the iteration progress, and the weights of the contrastive learning loss and triplet loss are dynamically adjusted. During the initial phase, the consistency of patch features is optimized solely through contrastive learning loss. During the transition period, a cosine annealing strategy is used to smooth the transition weights. The enhancement phase focuses on improving the discriminative power of patch instance labels through triplet loss.

7. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 3, characterized in that, Step 4, which describes elliptic sampling based on the trained Gaussian units, includes: For each Gaussian element, sample its center point; For ellipses whose major-to-minor axis ratio is greater than a preset threshold, n additional points are sampled along the ellipse surface. Assign the edge values ​​and patch instance labels corresponding to the Gaussian elements to all its sampling points; Perform patch instance labeling post-processing on the point cloud composed of all sampling points to remap the labels and obtain the final unified patch instance labels.

8. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 7, characterized in that, The primitive fitting described in step 5 includes: Step 5-1-1: Define three primitive models: plane, cylinder, and sphere, and express them as parametric surfaces. For each point cloud cluster with patch instance labels, fit the above primitive models respectively. Step 5-1-2: Calculate the geometric intersection between adjacent primitives to obtain the corresponding straight lines and curves in the geometric space; Step 5-1-3: Using the intersection line of two adjacent planes obtained in step 5-1-2, candidate corner points are obtained through the intersection of the two lines, and then these candidate points are clustered to generate the final corner points; Step 5-1-4: Under the constraints of the extracted edges and corners, the effective area of ​​each parametric surface is defined. The plane is subjected to constrained triangulation, and the cylinder and sphere are meshed in the parameter domain to generate watertight patches. Finally, all reconstructed patches are merged into a unified CAD mesh, and the vertex normals are aligned. Boolean operations are used for topology adjustment, and finally, a watertight boundary representation CAD model is assembled.

9. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 1, characterized in that, The dome-like surface fitting described in step 5 includes the following steps: Step 5-2-1: Align the coordinates of the point cloud obtained in step 4; Step 5-2-2: Generate a regular control point mesh covering the point cloud region on the aligned point cloud. Step 5-2-3: Determine the concavity / convexity of the dome; Step 5-2-4, Bézier surface patch fitting: Divide the control point mesh into multiple control point blocks, adjust the parameterization direction of the Bézier surface based on the concavity and convexity determination results, perform parameterization fitting of each control point block in a two-parameter direction using the cubic Bézier surface formula, and generate the corresponding surface patch lattice by sampling at equal intervals along the two parameter directions. Step 5-2-5: Piece together all the surface patch lattices in the order of the grid to form a complete fitted surface lattice. Divide each quadrilateral unit in the lattice into triangles to construct a triangular mesh surface. Step 5-2-6, Coordinate System Restoration: Transform the triangular mesh surface back to the original coordinate system using the inverse of the rotation matrix in Step 5-2-1, and output the final dome-fitted surface.

10. The three-dimensional boundary representation reconstruction method based on multi-view image application of dome-like surface fitting according to claim 9, characterized in that, The control point mesh generation described in step 5-2-2 includes the following steps: Step 5-2-2-1: Calculate the local tangent plane coordinate system of the point cloud; Step 5-2-2-2: Project the point cloud onto the tangent plane coordinate system to obtain the projected coordinates of each point on the two tangent axes. Calculate the maximum and minimum values ​​of the projected coordinates in the u and v directions respectively, and add a preset margin in each direction to expand the bounding box. Step 5-2-2-3, Generate regular mesh nodes: Within the bounding box of the tangent plane, uniformly divide the mesh according to the preset mesh resolution to generate a series of two-dimensional mesh node coordinates; Step 5-2-2-4: Map the mesh nodes back to 3D space: Convert the coordinates of each 2D mesh node to 3D world coordinates, construct the KD tree index of the point cloud, search for the nearest neighbor in the point cloud for each 3D coordinate point, and use the 3D coordinates of the nearest neighbor as the control point value of the mesh node, finally forming a regularly arranged 3D control point mesh.

Images