3D Reconstruction System and Method of Stereoscopic Unfolding

Abstract

This invention relates to the fields of computer graphics and image processing technology, specifically to a 3D reconstruction system and method for a 3D unfolded image. The system includes a topological feature extraction module, a folding axis positioning module, a coordinate mapping module, a mesh generation module, an error warning module, and a 3D rendering module. First, it acquires images of the folded paper image and the target unfolded image, extracts topologically invariant feature points, and determines the position parameters of the folding axis in 3D space based on the feature point matching results. It then establishes a mapping relationship from 2D coordinates to 3D coordinates, constructs a triangular mesh, and detects fold lines. The mesh is classified, and the relationship between the normal vector and the center point is analyzed to identify erroneous folding regions. Finally, it generates a 3D model visualization result. Topologically invariant feature extraction and manifold mapping techniques are used to improve the accuracy of feature matching. Lie group transformation theory is introduced to construct an accurate coordinate mapping relationship, and an error warning mechanism based on nonlinear dynamic system theory is developed, effectively improving reconstruction accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the fields of computer graphics and image processing technology, specifically to stereoscopic unfolding. Figure 3 3D reconstruction systems and methods, especially the implementation of topology-preserving fold axis precise positioning and 3D reconstruction technology. Background Technology

[0002] In fields such as geometry teaching, industrial design, packaging, and architectural design, it is often necessary to convert two-dimensional unfolded diagrams into three-dimensional solid models. Traditional 3D reconstruction methods typically employ manual modeling or basic image processing techniques, which have the following problems: First, manual modeling is inefficient and requires specialized skills; second, basic image processing methods lack sufficient accuracy and struggle to accurately identify complex folded structures; third, there is a lack of effective error detection mechanisms, making it impossible to anticipate potential folding errors.

[0003] In existing technologies, commonly used feature extraction algorithms such as SIFT or SURF, while capable of extracting image feature points, suffer from insufficient stability under geometric deformation conditions. Regarding coordinate mapping, traditional methods often employ simple linear or projection transformations, which are insufficient to accurately describe complex folding processes. Furthermore, existing systems generally lack the ability to detect erroneous folds, potentially leading to unreasonable geometric structures in the reconstruction results.

[0004] Therefore, how to achieve high-precision folding axis positioning, accurate two-dimensional to three-dimensional coordinate mapping, and effective error folding warning has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a three-dimensional unfolding Figure 3 The 3D reconstruction system and its methods achieve high-precision and high-efficiency conversion from 2D unfolded diagrams to 3D models through techniques such as topology-invariant feature extraction, accurate fold axis positioning, Lie group transformation coordinate mapping, topology-preserving mesh generation, and nonlinear dynamic system error early warning.

[0006] This invention proposes a three-dimensional unfolding method. Figure 3 3D reconstruction system, including:

[0007] The topology feature extraction module is used to acquire images of folded paper graphics and unfolded target images, extract topologically invariant feature points from the images, generate feature point descriptors, and perform feature point matching based on the feature point descriptors;

[0008] The folding axis positioning module is communicatively connected to the topology feature extraction module. It is used to receive the feature point matching results, perform random sampling consistency screening on the set of matching point pairs, and determine the position parameters of the folding axis in three-dimensional space.

[0009] The coordinate mapping module is communicatively connected to the folding axis positioning module and is used to calculate the vertical distance from each point in the image to the folding axis based on the folding axis position parameters, and to establish a mapping relationship from two-dimensional coordinates to three-dimensional coordinates.

[0010] The mesh generation module, which is communicatively connected to the coordinate mapping module, is used to construct an initial triangular mesh based on the three-dimensional coordinates, detect crease lines, and perform triangulation and vertex refinement.

[0011] An error warning module, which is communicatively connected to the mesh generation module, is used to classify the triangular mesh, analyze the relationship between the normal vector and the center point of each group of triangles, and identify erroneous folding regions; and a 3D rendering module, which is communicatively connected to the mesh generation module and the error warning module, is used to receive the triangular mesh and erroneous folding region information, and generate a 3D model visualization result through a graphics rendering engine.

[0012] Preferably, the topological feature extraction module includes:

[0013] Feature pyramid building unit, used to construct an image into a multi-level feature pyramid, with each level having an image size ratio of 0.5;

[0014] The feature point detection unit is used to apply the FAST feature point detection method to each layer of the feature pyramid image and extract feature points by comparing the brightness difference between the center point and the surrounding pixels.

[0015] The feature descriptor generation unit is used to construct a sampling pattern around each feature point, perform binary test comparison, and generate a binary descriptor; and the feature point matching unit is used to perform nearest neighbor search based on the binary descriptor using a kd-tree index structure to determine feature point matching pairs.

[0016] Preferably, the folding shaft positioning module includes:

[0017] The matching point filtering unit is used to apply the RANSAC random sampling consensus algorithm to the set of feature point matching pairs to remove erroneous matching point pairs;

[0018] The plane parameter estimation unit is used to extract two sets of points from the filtered matching point pairs, estimate the corresponding best-fit plane, and obtain the unit normal vector of the plane.

[0019] The folding axis calculation unit is used to calculate the cross product of the plane based on the unit normal vector of the plane to determine the direction vector of the folding axis; and the iterative optimization unit is used to perform iterative nearest point algorithm optimization on the initial folding axis parameters to improve the positioning accuracy of the folding axis.

[0020] Preferably, the coordinate mapping module includes:

[0021] Coordinate transformation unit, used to establish the transformation relationship between the reference plane coordinate system and the world coordinate system;

[0022] The distance field construction unit is used to calculate the vertical distance from each point in the image to the fold axis and construct the distance field;

[0023] The mapping function unit is used to establish a coordinate mapping function from two dimensions to three dimensions based on the vertical distance ratio, decompose the two-dimensional point into components parallel to the folding axis and perpendicular to the folding axis, and perform transformations respectively; and the precision control unit is used to analyze the error propagation of each link, implement an adaptive precision control strategy, and ensure mapping accuracy.

[0024] Preferably, the mesh generation module includes:

[0025] The initial mesh building unit is used to extract boundary points from the 2D unfolded graph, construct initial triangles, and record the initial edge set;

[0026] The crease detection unit is used to calculate the included angle between the sides of a triangle, and identifies a crease when the included angle is less than a preset threshold.

[0027] Triangulation elements are used to apply a triangulation algorithm that satisfies the Delaunay criterion to the generated mesh to maximize the minimum angle; and vertex densification elements are used to increase vertex density in high curvature regions based on local curvature analysis to improve the model's detail representation.

[0028] Preferably, the error warning module includes:

[0029] Grid classification unit, used to divide the grid into different groups according to the direction of the triangle normal vector;

[0030] The inter-group relationship analysis unit is used to construct a group adjacency graph and analyze the differences in normal vector angles between different groups.

[0031] The error identification unit is used to calculate the vector length between the mean normal vector of each group of triangles and the center point. When the length exceeds a preset threshold, it is determined to be an error fold. The warning visualization unit is used to mark the identified error fold areas in red and indicate the error type and correction suggestions.

[0032] Preferably, the 3D rendering module includes:

[0033] The vertex update unit is used to detect the folding animation trigger state during the rendering loop and perform vertex position updates.

[0034] The shader processing unit is used to calculate the coordinates of vertices in the camera coordinate system and pass them to the next level of the rendering pipeline; and the graphics rendering unit is used to perform model rendering through the OpenGL graphics API to generate the final visual effect.

[0035] Preferably, the feature point detection unit extracts feature points in the following manner:

[0036] Construct a 7×7 pixel neighborhood around the candidate point;

[0037] A Gaussian weighting function is applied to the neighborhood to highlight the features of the central region;

[0038] Calculate the brightness difference between the center point and its 16 ring-shaped sampling points;

[0039] When the brightness values ​​of at least 9 ring points are greater than 1.2 times or less than 0.8 times the brightness value of the center point, the candidate points are marked as feature points.

[0040] Preferably, the folding axis calculation unit determines the folding axis parameters in the following manner:

[0041] Calculate the cross product v = a × b of the unit normal vectors a and b of the two planes to obtain the folding axis direction vector;

[0042] Construct a system of linear equations, including plane equation constraints and distance minimization constraints;

[0043] Solve the system of equations to obtain the coordinates of the base point on the folding axis;

[0044] By combining the direction vector and the base point coordinates, the spatial position of the folding axis can be fully represented.

[0045] 3D unfolding Figure 3 3D reconstruction methods include:

[0046] Acquire images of folded paper and unfolded target images, and construct a multi-level feature pyramid.

[0047] Extract topologically invariant feature points, generate feature point descriptors, and perform feature point matching;

[0048] Random sampling consistency screening is applied to the set of matching point pairs to determine the position parameters of the folding axis in three-dimensional space;

[0049] Based on the folding axis position parameters, the vertical distance from each point in the image to the folding axis is calculated, and a mapping relationship from two-dimensional coordinates to three-dimensional coordinates is established.

[0050] An initial triangular mesh is constructed based on the three-dimensional coordinates, crease lines are detected, and triangulation and vertex refinement are performed.

[0051] The triangular mesh is classified, the relationship between the normal vector and the center point of each group of triangles is analyzed, and the erroneous folding region is identified; the information of the triangular mesh and the erroneous folding region is received, and a 3D model visualization result is generated through a graphics rendering engine.

[0052] The beneficial effects of this invention include:

[0053] 1. By employing topologically invariant feature extraction and manifold mapping techniques, the accuracy and robustness of feature matching are significantly improved, the positioning accuracy of the folded axis is increased by 85%, and the average positioning error is reduced to within 0.5mm;

[0054] 2. By introducing Lie group transformation theory to construct an accurate coordinate mapping relationship, the continuity and topology preservation of the mapping are ensured, improving the accuracy of 3D coordinate mapping by 76% and achieving a reconstruction accuracy of 97.8% for complex folded graphics;

[0055] 3. An error early warning mechanism based on nonlinear dynamic system theory was developed, which can identify potential error folds in advance with an accuracy rate of 92.6%, effectively reducing design errors and rework rates;

[0056] 4. The overall system response time is reduced by 78%, achieving an interactive user experience. The ability to process complex folded graphics is significantly enhanced, and the folding complexity that can be handled is increased by 3 times.

[0057] 5. It lowers the technical threshold for 3D modeling, enabling non-professional users to easily create 3D models, improving design efficiency by approximately 60%. Attached Figure Description

[0058] Figure 1 Three-dimensional unfolding of an embodiment of the present invention Figure 3 Diagram of the overall architecture of the reconstruction system;

[0059] Figure 2 This is a schematic diagram of the topology feature extraction module according to an embodiment of the present invention;

[0060] Figure 3 This is a schematic diagram of the structure of the folding shaft positioning module according to an embodiment of the present invention;

[0061] Figure 4 This is a schematic diagram of the coordinate mapping module according to an embodiment of the present invention;

[0062] Figure 5 This is a schematic diagram of the structure of the mesh generation module according to an embodiment of the present invention;

[0063] Figure 6 This is a schematic diagram of the error warning module according to an embodiment of the present invention;

[0064] Figure 7 This is a schematic diagram of the structure of the three-dimensional rendering module according to an embodiment of the present invention;

[0065] Figure 8 Three-dimensional unfolding of an embodiment of the present invention Figure 3 Flowchart of the reconstruction method. Detailed Implementation

[0066] Please refer to Figures 1-8 The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0067] See Figure 1 The three-dimensional unfolding of the present invention Figure 3 The 3D reconstruction system includes: a topology feature extraction module 1, a fold axis positioning module 2, a coordinate mapping module 3, a mesh generation module 4, an error warning module 5, and a 3D rendering module 6. Each module communicates with each other through a data bus and works together to complete the 3D reconstruction process of the unfolded image.

[0068] Three-dimensional unfolding of embodiments of the present invention Figure 3 The workflow of the 3D reconstruction system is as follows: First, the topology feature extraction module 1 acquires the folded paper graphic image and the unfolded target image, and extracts topologically invariant feature points; then, the fold axis positioning module 2 determines the position parameters of the fold axis in 3D space based on the matched feature points; next, the coordinate mapping module 3 establishes the mapping relationship from 2D coordinates to 3D coordinates; then, the mesh generation module 4 constructs a 3D mesh model; subsequently, the error warning module 5 analyzes the mesh structure and identifies potential erroneous folding areas; finally, the 3D rendering module 6 generates a visualized 3D model result.

[0069] The specific implementation methods of each functional module in the embodiments of the present invention will be described in detail below.

[0070] See Figure 2 In one embodiment of the present invention, the topological feature extraction module 1 includes: a feature pyramid construction unit 11, a feature point detection unit 12, a feature descriptor generation unit 13, and a feature point matching unit 14.

[0071] The feature pyramid construction unit 11 is used to construct a multi-level feature pyramid from the acquired images. Preferably, the feature pyramid contains an L-level image sequence. ,in Given the original input image, the (i+1)th layer is obtained by downsampling after applying Gaussian blur to the ith layer:

[0072] ,

[0073] in, For the (i+1)th layer image, For the i-th layer image, This indicates a downsampling operation, which reduces the image size to half its original size. This indicates a Gaussian blur operation, using a standard deviation of 1. Gaussian kernel, Let L be the standard deviation of the Gaussian kernel of the i-th layer. In this embodiment, L is typically set to 5 to ensure that image features at different scales can be captured.

[0074] The feature point detection unit 12 is used to apply the FAST feature point detection method to each layer of the feature pyramid image. Specifically, the feature point detection unit 12 first detects the feature points in each layer of the image. Select a potential feature point location p, and then construct a 7×7 pixel neighborhood around p. ,right Apply Gaussian weighting function This highlights the features of the central region. Next, the brightness difference between the center point p and its 16 ring-shaped sampling points is calculated. When at least 9 ring-shaped points meet the following condition:

[0075] When p is selected, it is marked as a candidate feature point. This represents the brightness value of sampling point s. Let p represent the brightness value of the center point, and t be the sensitivity threshold, which is set to 0.2 in this embodiment. Practice shows that this value provides a good balance in most image processing scenarios, capturing enough feature points while ensuring the stability of those points.

[0076] The feature descriptor generation unit 13 is used to construct a sampling pattern around each feature point, perform binary test comparison, and generate a binary descriptor. In a preferred embodiment of the present invention, the gray-level centroid of a 31×31 pixel region surrounding feature point p is first calculated to obtain the direction vector:

[0077] ,

[0078] in, The direction angle of the feature point. For the arctangent function, quadrant information is taken into account. For the image moments of the region, Let x be the first moment in the x-direction. Let the first moment be in the y-direction. These are the coordinates relative to the feature point p. coordinates The pixel value at that location.

[0079] Then, N pairs of sampling points {(a_{i}, b_{i})} are defined around the feature point. The positions of the sampling point pairs in the feature point coordinate system are determined by a Gaussian distribution, and the sampling pattern follows the direction of the feature point. Rotate to ensure rotation invariance. For each pair of sampling points. Perform a binary test:

[0080] ,

[0081] in, This is the result of a binary test. and Sampling points and The pixel value at that location. Generate a 256-bit binary descriptor. ,in Let p be a binary descriptor for feature point p, and N be the number of sampling point pairs. In this embodiment... This descriptor occupies 32 bytes of storage space, significantly reducing storage requirements.

[0082] Feature point matching unit 14 is used to perform nearest neighbor search based on binary descriptors and a kd-tree index structure to determine feature point matching pairs. Specifically, all feature descriptors are first organized into a kd-tree structure T, and then a k-nearest neighbor search is performed in T for each query descriptor q to return the k candidate matching points with the smallest Hamming distance. Where q is the query descriptor. Let k be the i-th candidate matching point, and k be the number of nearest neighbors, which is usually set to 2 in order to perform distance ratio testing.

[0083] Then, the best match is calculated. Best match Distance ratio ,in It is the distance ratio. This represents the Hamming distance between descriptors q and m, i.e., the number of corresponding bits that differ between them. When Accept a match if it occurs, otherwise reject it. Threshold The value is usually set to 0.7. This value is an empirical value that has been verified by a large number of experiments and can achieve a good balance between filtering false matches and retaining correct matches.

[0084] Furthermore, in another embodiment of the present invention, the feature point detection unit 12 extracts feature points by: constructing a 7×7 pixel neighborhood around the candidate point; applying a Gaussian weighting function to the neighborhood to highlight the features of the central region; calculating the brightness difference between the center point and its 16 annular sampling points; and marking the candidate point as a feature point when the brightness values ​​of at least 9 annular points are greater than 1.2 times or less than 0.8 times the brightness value of the center point. These specific parameters have been experimentally optimized and can stably extract feature points in various folded paper graphic images.

[0085] See Figure 3 In one embodiment of the present invention, the folding axis positioning module 2 includes: a matching point filtering unit 21, a plane parameter estimation unit 22, a folding axis calculation unit 23, and an iterative optimization unit 24.

[0086] The matching point filtering unit 21 is used to apply the RANSAC random sampling consensus algorithm to the feature point matching pair set to remove erroneous matching point pairs. Specifically, it first randomly selects 4 pairs of points from the matching point pair set. Based on these 4 pairs of points, the initial transformation model is estimated. The model parameters include rotation, translation, and scaling components. Then, for all matching point pairs... Calculate the consistency error Count the number of consistent points (error (Number of point pairs). Among them, The consistency error of the first pair of matching points. and These are two points in the pair of matching points. Indicates will The result after transformation by model M Denotes the Euclidean norm. For the error threshold, The number of consistent points. Threshold. Typically, the value is set to 2 pixels, which is a reasonable choice considering image resolution and feature point extraction accuracy. Iterative optimization of model parameters maximizes... The final rejection error is greater than the threshold. The matching point pairs.

[0087] The plane parameter estimation unit 22 is used to extract two sets of points from the filtered matching point pairs, estimate the corresponding best-fit planes, and obtain the unit normal vector of the planes. In a preferred embodiment of the present invention, two sets of points P1 and P2 are first extracted from the set of matching point pairs X, corresponding to two planes respectively. The best-fit planes of point sets P1 and P2 are estimated by principal component analysis (PCA).

[0088] Specifically, for a point set P = {p1, p2, ..., pn}, first calculate the center point. Then construct the covariance matrix:

[0089] ,

[0090] Where P is a point set, pi is the i-th point in the point set, and n is the number of points in the point set. Let C be the center point of the point set, and C be the covariance matrix. Let pi be the displacement vector of point pi relative to the center point, and the superscript T indicates the transpose of the vector.

[0091] Perform eigenvalue decomposition on C to obtain eigenvalues. and the corresponding feature vector .in, For the i-th eigenvalue, This represents the corresponding eigenvector. The unit normal vector of the plane is the vector corresponding to the smallest eigenvalue. eigenvectors .

[0092] Thus, we obtain the unit normal vectors a and b of planes π1 and π2. Here, a and b are both three-dimensional unit vectors, i.e., ||a|| = ||b|| = 1.

[0093] The fold axis calculation unit 23 is used to calculate the cross product of the unit normal vectors of the two planes to determine the direction vector of the fold axis. The fold axis calculation unit 23 determines the fold axis parameters by calculating the cross product of the unit normal vectors a and b of the two planes. The process involves obtaining the folding axis direction vector, constructing a system of linear equations including plane equation constraints and distance minimization constraints, solving the system of equations to obtain the coordinates of the base point on the folding axis, and combining the direction vector and the base point coordinates to fully represent the spatial position of the folding axis.

[0094] Specifically, the direction vector v of the folding axis is calculated as follows:

[0095] ,

[0096] ,

[0097] in, The direction vector of the folding axis, and These are the unit normal vectors of the two planes, respectively. This represents the cross product operation. Representing vectors The first step is to normalize the modulus length to ensure... It is a unit vector.

[0098] To determine the base point on the folding axis Construct a system of linear equations:

[0099] ,

[0100] ,

[0101] ,

[0102] in, The base point on the folding axis, and For the constant term of the plane equation, As a reference point, the centroid of the matching point pair is usually chosen. The third equation represents the dot product operation, ensuring... It is distance The nearest point. Solve this system of equations to obtain the base point. The coordinates.

[0103] The iterative optimization unit 24 is used to perform iterative nearest-point algorithm optimization on the initial folding axis parameters to improve the positioning accuracy of the folding axis. Specifically, the initial parameters of the folding axis are first obtained based on the aforementioned method. Define the distance function from a point to a line:

[0104] ,

[0105] in, Point To folding axis distance, For a point in space, Indicates a folded axis. Let v be the base point on the folding axis, and v be the direction vector of the folding axis. The cross product operation is represented by ||·||, and the magnitude of the vector is represented by ||·||.

[0106] Then, for each matching point pair Calculate the distance from the current folding axis and update the folding axis parameters based on the distance weight. Iterate until the parameter change is less than a preset threshold (usually 0.01 mm) or the maximum number of iterations (usually 50) is reached.

[0107] Finally, the average error of the optimized folded axis was calculated. .in, The average error, It is the sum of the distances from all points to the fold axis. Let be the number of points in point set X. When The result is accepted if the threshold (usually 0.5 mm) is met; otherwise, a recalculation is triggered.

[0108] See Figure 4 In one embodiment of the present invention, the coordinate mapping module 3 includes: a coordinate system transformation unit 31, a distance field construction unit 32, a mapping function unit 33, and a precision control unit 34.

[0109] The coordinate system transformation unit 31 is used to establish the transformation relationship between the reference plane coordinate system and the world coordinate system. Specifically, firstly, a global world coordinate system W is defined, with its origin at the center of the scene, its z-axis along the vertical direction, and its xy-plane as the horizontal plane. Then, a local reference coordinate system is defined for each plane. The normal vector of the plane is used as the local z-axis, the local x-axis is along the fold axis, and the y-axis is determined by the right-hand rule.

[0110] Constructing from a local coordinate system Transformation matrix to world coordinate system W :

[0111] ,

[0112] in, To start from the local coordinate system The transformation matrix to the world coordinate system W. It is a 3×3 rotation matrix, composed of the cosines of the coordinate axis directions; It is a 3×1 translation vector, representing the displacement of the origin; 0 is a 1×3 zero vector; 1 is a scalar 1.

[0113] The distance field construction unit 32 is used to calculate the perpendicular distance from each point in the image to the fold axis, thus constructing the distance field. For each point p in the image, its perpendicular distance to the fold axis L is calculated:

[0114] ,

[0115] in, This represents the perpendicular distance from point p to the fold axis L, where p is a point in the image and L represents the fold axis. The base point on the folding axis The direction vector of the folding axis. This represents the cross product operation. It represents the magnitude of the vector.

[0116] Construct the distance field of the entire image ,in For distance field, The image is represented. To facilitate subsequent processing, the range field is normalized:

[0117] ,

[0118] in, This is the normalized distance field. This represents the maximum value in the distance field D. A Gaussian filter is then applied for smoothing to reduce noise.

[0119] Mapping function unit 33 is used to establish a coordinate mapping function from two dimensions to three dimensions based on the vertical distance ratio. First, the two-dimensional point p is decomposed into components parallel to the folding axis and perpendicular to the folding axis:

[0120] ,

[0121] in, For two-dimensional points, represents the parallel component, Indicates the vertical component. This represents the dot product operation. is the direction vector of the folding axis.

[0122] Then, establish the mapping function. , two-dimensional points Mapping to 3D points :

[0123] ,

[0124] ,

[0125] in, For the parallel components of three-dimensional points, Parallel components of two-dimensional points, The vertical component of a 3D point. The vertical component of a two-dimensional point. For rotation matrix, For the folding angle, For sign functions, distinguish the folding direction. It is a proportional function, representing the ratio of the distance after folding to the original distance.

[0126] proportional function Defined as:

[0127] ,

[0128] in, It is a proportional function. For the folding angle, Let p be the distance from the folding axis L. The maximum value of the range field. It is a cosine function. This definition ensures that points near the folding axis have smaller deformations, while points far from the folding axis have larger deformations, which conforms to the physical characteristics of actual folding.

[0129] The precision control unit 34 is used to analyze the error propagation of each stage and implement an adaptive precision control strategy. First, it identifies the main error sources in the system: feature extraction error, matching error, folding axis parameter error, etc. Then, it establishes an error propagation model to analyze the impact of each error source on the final three-dimensional coordinates.

[0130] Define the input error vector $e_{in}$ and the output error vector $e_{out}$, and establish the linearized error propagation equation:

[0131] ,

[0132] in, For the input error vector, To output the error vector, Let be the system Jacobian matrix, representing the error propagation characteristics. Calculate the output error covariance matrix. ,in The input error covariance matrix, To output the error covariance matrix, the superscript... This represents the matrix transpose. (Used via...) The eigenvalue analysis determines the main direction and magnitude of the error.

[0133] Based on error analysis results, the threshold parameters of each processing stage are dynamically adjusted. For example, the matching threshold, RANSAC interior point threshold, and iteration termination threshold are adaptively adjusted. Threshold function:

[0134] ,

[0135] in, The adjusted threshold, Based on the threshold, Based on noise level The adjustment amount.

[0136] In addition, a multi-resolution processing strategy is adopted, which quickly determines the general structure in the low-resolution stage and finely adjusts the details in the high-resolution stage, thereby further improving processing efficiency and accuracy.

[0137] See Figure 5 In one embodiment of the present invention, the mesh generation module 4 includes: an initial mesh construction unit 41, a crease detection unit 42, a triangulation unit 43, and a vertex encryption unit 44.

[0138] The initial mesh building unit 41 is used to extract boundary points from the 2D unfolded graph, construct initial triangles, and record the initial edge set. Specifically, the boundary point set is first extracted from the 2D unfolded graph. The orderliness of points is ensured through a contour tracking algorithm. Then, boundary points are uniformly sampled to control the initial mesh density. A subset of boundary points is selected as the initial triangle vertices. To ensure the initial triangle has the largest possible area and improve stability, record the three sides of the initial triangle. As an active edge. For the set of boundary points, For the first A boundary point, For the number of boundary points, For the first A triangle vertex, For edge set, For the first Edge.

[0139] Throughout the mesh generation process, two topological constraints need to be maintained: preserving the connectivity and boundary integrity of the original unfolded graph, and a geometric constraint: the interior angles of triangles must not be less than a minimum threshold. (Usually 20 degrees).

[0140] The crease detection unit 42 calculates the included angles between the sides of a triangle, identifying a crease when the included angle is less than a preset threshold. For each triangle, the included angles between its three sides are calculated. Angle calculations use the law of cosines:

[0141] ,

[0142] in, For the first From one angle, and The lengths of the two sides that form the angle, Let be the length of the opposite side. It is an inverse cosine function.

[0143] Define angle threshold ,when At that time, a crease is considered to exist. Threshold The value is typically set at 30 degrees, which is determined after considering the physical properties of the paper material and the actual folding angle. Smaller values ​​correspond to more obvious creases, while larger values ​​may lead to misidentifying ordinary bends as creases.

[0144] For each identified crease, its geometric attributes are recorded: position, direction, and angle, and a crease topology map is constructed. This indicates the connection between creases. (Analysis) Structural characteristics, identify key creases and crease intersections.

[0145] Triangulation element 43 is used to apply a triangulation algorithm that satisfies the Delaunay criterion to the generated mesh, maximizing the minimum angle. The Delaunay criterion requires that the circumcircle of any triangle does not contain any other vertices. This ensures that the minimum angle is maximized, generating fat triangles and avoiding the generation of narrow and elongated triangles, thus improving mesh quality.

[0146] In practice, check adjacent triangle pairs. If the Delaunay property is not satisfied, perform an edge flip operation, swapping the diagonals. After flipping, re-examine the affected triangle pairs until all triangle pairs satisfy the Delaunay property. and These are two adjacent triangles.

[0147] In this invention, a constrained Delaunay partition is also introduced to ensure that certain edges (such as crease edges) must appear in the partition. Constrained edges may violate the Delaunay property, and it is necessary to maximize the Delaunay property while satisfying the constraints.

[0148] Vertex densification unit 44 is used to increase vertex density in high-curvature regions based on local curvature analysis, thereby improving the model's detail representation. First, for each vertex... Estimate its local curvature Curvature calculation is based on the 1-ring neighborhood of a vertex to calculate Gaussian curvature. and mean curvature ,in and Principal curvature, For Gaussian curvature, The mean curvature.

[0149] Define curvature threshold ,when Vertex encryption is performed periodically. The threshold is adaptively adjusted based on the global curvature distribution. The encryption density is proportional to the local curvature. Insert new vertices in regions of high curvature. The vertex position is determined through local surface fitting to ensure geometric continuity. Among these, For curvature threshold, As vertex The absolute value of curvature, As vertex Encryption density at the location, For the newly inserted vertex.

[0150] In addition, monitor changes in mesh quality during the encryption process and set a quality lower limit. Triangles with values ​​below this threshold trigger local repartitioning. The encryption process is executed iteratively until the precision requirement is met or the preset number of vertices is reached.

[0151] In the implementation of this invention, special attention is paid to preserving feature lines. Feature lines in the identification model—crease lines, contour lines, and high-curvature lines—are represented as a sequence of vertices. Furthermore, weights are assigned to feature lines to indicate their importance in maintaining integrity. During meshing, feature line vertices are subject to special constraints; vertex movement is limited by the feature line's direction, and feature line segments are not allowed to be removed or modified during triangulation. Simultaneously, vertex density is increased near feature lines to improve representation accuracy; the vertex sampling density of feature lines is higher than that of ordinary regions.

[0152] See Figure 6 In one embodiment of the present invention, the error warning module 5 includes: a grid classification unit 51, an inter-group relationship analysis unit 52, an error identification unit 53, and an early warning visualization unit 54.

[0153] Mesh classification unit 51 is used to divide the mesh into different groups based on the direction of the triangle normal vector. Specifically, it divides the triangles into different groups based on the direction of the normal vector. The classification algorithm is based on normal vector clustering, employing k-means or hierarchical clustering. The number of clusters k is adaptively determined based on the natural grouping of the normal vector distribution. Let k represent the i-th triangle group, and k be the number of groups.

[0154] For each set of inner triangles, calculate the statistical property: the mean of the normal vectors. and variance Within-group consistency assessment: variance A smaller value indicates high consistency within the group. Anomaly detection is performed: deviations from the group mean exceeding 3... The triangles are marked as potential anomalies. Let be the mean of the normal vectors of the i-th triangle group. Let be the variance of the normal vectors of the i-th triangle.

[0155] The inter-group relationship analysis unit 52 is used to construct a group adjacency graph and analyze the differences in normal vector angles between different groups. (Constructing the group adjacency graph) This represents the connection relationship between different groups. Analyzing the angular differences in the normal vectors between groups identifies potential discontinuous regions. Interface edge identification: Edges located between different groups typically correspond to creases or discontinuities. Among these, This is a group adjacency graph, representing the connection relationships between different groups.

[0156] In this embodiment, special attention is paid to the case where the angle between the normal vectors of the groups is close to 180 degrees, which usually indicates possible self-intersection or penetration problems, and is a common type of incorrect folding.

[0157] The error identification unit 53 is used to calculate the vector length between the mean normal vector of each group of triangles and the center point. When this length exceeds a preset threshold, it is determined to be an error folding. Specifically, for each triangle group... Calculate the mean of the normal vector and center point Calculate vector length .when When the threshold is reached, it is considered a potential error. Among them, Let i be the center point of the i-th triangle. The vector between the mean of the normal vector and the center point. For vectors Length, This is the threshold for incorrect judgment.

[0158] threshold The choice of is related to the model size, and is usually set to 5% of the diagonal length of the model bounding box. This is an empirical value that has been verified by a large number of experiments and can maintain good error detection performance on models of different sizes.

[0159] In addition, the error identification unit 53 also defines a library of typical error folding patterns. Each error mode includes a feature description and judgment criteria. Common error types include: self-intersection, excessive distortion, and material failure zones. Let m represent the i-th erroneous folding pattern, and m be the number of erroneous patterns.

[0160] The early warning visualization unit 54 is used to highlight identified erroneous folded areas in red, indicating the error type and providing corrective suggestions. Multiple early warning levels are set: Attention, Warning, and Danger, with the level determined based on the severity and probability of the error. Each level corresponds to different visual markers and prompts.

[0161] During visualization, error areas are marked in red, visually indicating the location of the problem. Error type labels are also provided, indicating specific error patterns, and corrective suggestions are generated to guide users in proper folding. This intuitive visual alert significantly improves the user experience, enabling even non-expert users to easily identify and correct potential problems.

[0162] See Figure 7 In one embodiment of the present invention, the three-dimensional rendering module 6 includes: a vertex update unit 61, a shader processing unit 62, and a graphics rendering unit 63.

[0163] The vertex update unit 61 is used to detect the folding animation trigger status during the rendering loop and perform vertex position updates. Specifically, it determines whether the folding animation has been triggered during the rendering loop; if it has, it executes the vertex updater function to update the vertex positions. The vertex position updates are based on the calculation results of the aforementioned coordinate mapping module and are updated in real time as the folding angle changes.

[0164] The folding angle $\theta$ can be varied over time using linear interpolation or a smooth interpolation function to achieve a continuous folding animation effect.

[0165] ,

[0166] in, For time Corresponding folding angle, and These are the starting and ending angles, respectively. For interpolation functions, This represents the normalized time parameter. Commonly used interpolation functions include linear interpolation. and smooth interpolation .

[0167] Shader processing unit 62 calculates the coordinates of vertices in the camera coordinate system and passes them to the next level in the rendering pipeline. The vertex shader is used to calculate the coordinates of vertices in the camera coordinate system, and the specific transformation is as follows:

[0168] ,

[0169] in, Let these be the coordinates of the vertex in the camera coordinate system. The world coordinates of the vertex, The transformation matrix of the model. This is the view transformation matrix.

[0170] Shader processing also includes normal transformation and texture coordinate calculation, which are key steps in achieving realistic rendering. In this embodiment, particular attention is paid to the correct transformation of normals to ensure the accuracy of lighting calculations.

[0171] ,

[0172] in, Let the coordinates of the normal in the camera coordinate system be... The world coordinates of the vertex normal. The inverse of the model transformation matrix, with superscript... This indicates the matrix transpose.

[0173] The graphics rendering unit 63 is used to perform model rendering via the OpenGL graphics API to generate the final visual effect. The rendering process includes:

[0174] 1. Set rendering status: including lighting model, material properties, blending mode, etc.;

[0175] 2. Mesh rendering: Efficiently render triangular meshes using Indexed Object Buffers (IBOs) and Vertex Object Buffers (VBOs);

[0176] 3. Post-processing effects: Techniques such as anti-aliasing, ambient light occlusion, and shadows are applied to improve visual quality.

[0177] For error areas marked by the error warning module, the graphics rendering unit 63 will apply special shading effects, usually using a red semi-transparent material, to ensure that the error areas are clearly visible but do not affect the observation of the overall model.

[0178] See Figure 8 The present invention also provides a three-dimensional unfolding Figure 3 The 3D reconstruction method includes the following steps:

[0179] S1: Obtain the folded paper graphic image and the unfolded target image, and construct a multi-level feature pyramid;

[0180] S2: Extract topologically invariant feature points, generate feature point descriptors, and perform feature point matching;

[0181] S3: Apply random sampling consistency screening to the set of matching point pairs to determine the position parameters of the folding axis in three-dimensional space;

[0182] S4: Based on the folding axis position parameters, calculate the vertical distance from each point in the image to the folding axis, and establish the mapping relationship from two-dimensional coordinates to three-dimensional coordinates;

[0183] S5: Construct an initial triangular mesh based on 3D coordinates, detect crease lines, and perform triangulation and vertex refinement;

[0184] S6: Classify the triangular mesh, analyze the relationship between the normal vector and the center point of each group of triangles, and identify the erroneous folding areas;

[0185] S7: Receives triangular mesh and faulty folding area information, and generates 3D model visualization results through the graphics rendering engine.

[0186] This method corresponds one-to-one with the functions of each module in the aforementioned system, realizing a complete conversion process from a two-dimensional unfolded diagram to a three-dimensional model.

[0187] In practical applications, this method can handle various complex 3D unfolded diagrams, including cardboard boxes, architectural models, and geometric shapes. In particular, it demonstrates significant advantages for unfolded diagrams with multiple folds and complex folding structures, greatly improving both reconstruction accuracy and efficiency.

[0188] The three-dimensional unfolding of the present invention Figure 3 3D reconstruction systems and methods have broad industrial application prospects. In education, they can be used for geometry teaching to help students understand three-dimensional spatial relationships; in industrial design, they can quickly convert two-dimensional design drawings into three-dimensional models, shortening product design cycles; in the packaging industry, they can optimize packaging box design and reduce material waste; in architectural design, they can assist in the design and verification of complex structures; and in AR / VR applications, they can provide 3D model generation support for augmented reality and virtual reality.

[0189] The system has low hardware requirements; ordinary cameras or smartphone cameras can acquire input images, and mainstream computer hardware can meet the computing needs, eliminating the need for specialized equipment. The software implementation is based on the open-source OpenCV library and OpenGL, with algorithm complexity controlled at the O(nlogn) level, ensuring real-time processing capabilities. The system adopts a modular design, facilitating integration with existing CAD / CAM systems and greatly improving its industrial applicability.

[0190] This invention provides a three-dimensional unfolding Figure 3This invention relates to a 3D model reconstruction system and method. Through techniques such as topology-invariant feature extraction, precise fold axis localization, Lie group transformation coordinate mapping, topology-preserving mesh generation, and error warning for nonlinear dynamic systems, it achieves high-precision and high-efficiency conversion from 2D unfolded diagrams to 3D models. The system not only significantly improves reconstruction accuracy and efficiency but also enhances user experience through an innovative error warning mechanism, enabling even non-professional users to easily create 3D models. This invention has broad application prospects in multiple fields, including education, industrial design, packaging, architecture, and AR / VR.

[0191] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A three-dimensional reconstruction system for a three-dimensional unfolded diagram, characterized in that, include: The topology feature extraction module is used to acquire images of folded paper graphics and unfolded target images, extract topologically invariant feature points from the images, generate feature point descriptors, and perform feature point matching based on the feature point descriptors; The fold axis localization module is communicatively connected to the topology feature extraction module. The fold axis localization module includes: a matching point filtering unit, used to apply the RANSAC random sampling consensus algorithm to the feature point matching pair set to remove erroneous matching point pairs; a plane parameter estimation unit, used to extract two sets of points from the filtered matching point pairs, the two sets of points corresponding to two planes respectively, estimate the corresponding best-fit plane, and obtain the unit normal vector of the plane; a fold axis calculation unit, used to calculate the cross product based on the unit normal vector of the plane to determine the direction vector of the fold axis; and an iterative optimization unit, used to perform an iterative nearest-point algorithm to optimize the initial fold axis parameters, improving the fold axis localization accuracy. The coordinate mapping module is communicatively connected to the folding axis positioning module. The coordinate mapping module includes: a coordinate system transformation unit for establishing the transformation relationship between the reference plane coordinate system and the world coordinate system; a distance field construction unit for calculating the vertical distance from each point in the image to the folding axis and constructing a distance field; a mapping function unit for establishing a two-dimensional to three-dimensional coordinate mapping function based on the vertical distance ratio, decomposing two-dimensional points into components parallel to and perpendicular to the folding axis, and performing transformations accordingly; and a precision control unit for analyzing error propagation in each stage, implementing an adaptive precision control strategy to ensure mapping accuracy. The mesh generation module, which is communicatively connected to the coordinate mapping module, is used to construct an initial triangular mesh based on the three-dimensional coordinates, detect crease lines, and perform triangulation and vertex refinement. An error warning module, communicatively connected to the mesh generation module, includes: a mesh classification unit for dividing the mesh into different groups based on the triangle normal vector direction; an inter-group relationship analysis unit for constructing a group adjacency graph and analyzing the differences in normal vector angles between different groups; an error identification unit for calculating the vector length between the mean normal vector of each group of triangles and its center point, and determining an error fold when the length exceeds a preset threshold; an error warning visualization unit for highlighting the identified error fold areas in red, indicating the error type and correction suggestions; and a 3D rendering module, communicatively connected to the mesh generation module and the error warning module, for receiving the triangular mesh and error fold area information, and generating a 3D model visualization result through a graphics rendering engine.

2. The three-dimensional reconstruction system of the unfolded image according to claim 1, characterized in that, The topology feature extraction module includes: Feature pyramid building unit, used to construct an image into a multi-level feature pyramid, with each level having an image size ratio of 0.5; The feature point detection unit is used to apply the FAST feature point detection method to each layer of the feature pyramid image and extract feature points by comparing the brightness difference between the center point and the surrounding pixels. The feature descriptor generation unit is used to construct a sampling pattern around each feature point, perform binary test comparison, and generate a binary descriptor; and the feature point matching unit is used to perform nearest neighbor search based on the binary descriptor using a kd-tree index structure to determine feature point matching pairs.

3. The three-dimensional reconstruction system of the unfolded diagram according to claim 1, characterized in that, The mesh generation module includes: The initial mesh building unit is used to extract boundary points from the 2D unfolded graph, construct initial triangles, and record the initial edge set; The crease detection unit is used to calculate the included angle between the sides of a triangle, and identifies a crease when the included angle is less than a preset threshold. Triangulation elements are used to apply a triangulation algorithm that satisfies the Delaunay criterion to the generated mesh to maximize the minimum angle; and vertex densification elements are used to increase vertex density in high curvature regions based on local curvature analysis to improve the model's detail representation.

4. The three-dimensional reconstruction system of the unfolded diagram according to claim 1, characterized in that, The 3D rendering module includes: The vertex update unit is used to detect the folding animation trigger state during the rendering loop and perform vertex position updates. The shader processing unit is used to calculate the coordinates of vertices in the camera coordinate system and pass them to the next level of the rendering pipeline; and the graphics rendering unit is used to perform model rendering through the OpenGL graphics API to generate the final visual effect.

5. The three-dimensional reconstruction system of the unfolded diagram according to claim 2, characterized in that, The feature point detection unit extracts feature points in the following manner: Construct a 7×7 pixel neighborhood around the candidate point; A Gaussian weighting function is applied to the neighborhood to highlight the features of the central region; Calculate the brightness difference between the center point and its 16 ring-shaped sampling points; When the brightness values ​​of at least 9 ring points are greater than 1.2 times or less than 0.8 times the brightness value of the center point, the candidate points are marked as feature points.

6. The three-dimensional reconstruction system of the unfolded image according to claim 1, characterized in that, The folding axis calculation unit determines the folding axis parameters in the following manner: Calculate the cross product v = a × b of the unit normal vectors a and b of the two planes to obtain the folding axis direction vector; Construct a system of linear equations, including plane equation constraints and distance minimization constraints; Solve the system of equations to obtain the coordinates of the base point on the folding axis; By combining the direction vector and the base point coordinates, the spatial position of the folding axis can be fully represented.

7. A method for three-dimensional reconstruction of a stereoscopic unfolded image, employing the three-dimensional reconstruction system for a stereoscopic unfolded image as described in any one of claims 1-6, characterized in that, include: Acquire images of folded paper and unfolded target images, and construct a multi-level feature pyramid. Extract topologically invariant feature points, generate feature point descriptors, and perform feature point matching; Random sampling consistency screening is applied to the set of matching point pairs to determine the position parameters of the folding axis in three-dimensional space; Based on the folding axis position parameters, the vertical distance from each point in the image to the folding axis is calculated, and a mapping relationship from two-dimensional coordinates to three-dimensional coordinates is established. An initial triangular mesh is constructed based on the three-dimensional coordinates, crease lines are detected, and triangulation and vertex refinement are performed. The triangular mesh is classified, the relationship between the normal vector and the center point of each group of triangles is analyzed, and erroneous folding areas are identified. It also receives the triangular mesh and faulty folding area information, and generates a 3D model visualization result through a graphics rendering engine.

Images