A high-precision three-dimensional grid fusion method based on optimal energy field

Abstract

The present invention discloses a high-precision three-dimensional mesh fusion method based on an optimal energy field, comprising the following steps: projecting a three-dimensional mesh model onto a ground plane, taking the accumulated height values ​​falling within the same pixel as the energy value, and generating an energy field map; using the energy field map, finding a path with the minimum energy within the overlapping area of ​​the two models as the boundary line between the two models; keeping the three-dimensional mesh models within the directions of each model stationary, removing the three-dimensional point clouds and triangular faces outside the boundary; extracting the point clouds corresponding to the triangular faces that fall exactly on the boundary line; performing a restricted De Novo meshing on the point clouds, and then placing the newly generated triangular faces into the model with the faces at the boundary line removed, and performing hole filling and defect removal operations on the final model to obtain the final fusion model. The present invention uses the energy field map to find the optimal boundary line between only a small number of overlapping three-dimensional mesh model blocks, thereby ensuring the accuracy of the generated model.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional reconstruction of remote sensing photography, and in particular relates to a high-precision three-dimensional grid fusion method based on an optimal energy field. Background Art

[0002] When using remote sensing imagery to reconstruct terrain in 3D, the large area required and the limitations of existing equipment make large-scale 3D reconstruction impossible. Therefore, the image is often reconstructed in blocks. However, the resulting 3D models lack any topological relationships, making them difficult to view and process as 3D meshes. Furthermore, to reduce duplicate 3D reconstruction areas, only small overlapping areas are often retained between 3D mesh blocks, making 3D mesh fusion between these blocks extremely difficult.

[0003] How to smooth the transition area between two models to be spliced ​​is the core issue in the study of three-dimensional mesh model fusion algorithms. In recent years, some algorithms have basically been studied around this issue. This article divides the algorithms into two categories, namely three-dimensional mesh model splicing methods and three-dimensional mesh model stitching methods.

[0004] (1) Three-dimensional mesh model splicing method

[0005] The three-dimensional mesh model splicing method mainly includes the following steps.

[0006] Determine the splicing direction: First, you need to determine the splicing direction of the 3D mesh model, that is, determine which faces need to be spliced.

[0007] Align mesh models: Align the mesh models to be spliced ​​to ensure that their coordinate systems are consistent.

[0008] Merge mesh models: Merge aligned mesh models, that is, integrate their vertices, faces, textures and other information to form a complete 3D model.

[0009] Processing overlapping parts: If there are overlapping parts in the spliced ​​mesh models, the overlapping parts need to be processed, such as deleting overlapping faces or adjusting vertex positions.

[0010] Optimize stitching results: Finally, optimize the stitched 3D model, such as removing redundant vertices and merging duplicate faces, to reduce the size of the model and improve performance.

[0011] (2) Three-dimensional mesh model stitching method

[0012] Different from mesh stitching methods, 3D mesh model stitching methods are usually used to stitch two or more independent mesh models into a whole model. There are mainly the following methods for stitching 3D mesh models:

[0013] Point cloud registration: Convert two or more mesh models into point cloud data and perform point cloud registration to find the correspondence between the two point clouds, thereby stitching the mesh models together. Point cloud registration is usually implemented using the ICP (Iterative Closest Point) algorithm or other registration algorithms.

[0014] Surface reconstruction: Reconstructing the surfaces of two or more mesh models and performing surface registration to seamlessly connect the surfaces of the two models. Surface reconstruction techniques include point cloud-based surface reconstruction and surface fitting-based surface reconstruction.

[0015] Topological connection: By creating connecting surfaces or connecting lines, the boundaries or surfaces of two mesh models are connected to achieve model stitching. This method can ensure the topological structure of the connection is complete and stable.

[0016] Texture mapping: After stitching two mesh models together, use texture mapping technology to seamlessly connect the textures of the two models, making the transition at the stitching more natural and smooth.

[0017] Voxel fusion: Convert two mesh models into voxel data and perform voxel fusion to achieve model stitching. Voxel fusion can achieve seamless connection of models by fusing two voxel mesh data.

[0018] It can be found by analyzing the topological connection method in the stitching of three-dimensional mesh models.

[0019] (1) Poor stability

[0020] When the model is more complex, its boundaries may be uneven or even missing. Direct topological connection at the boundaries may result in many deformed, narrow triangles. It may even be impossible to find suitable connection points to topologically connect the two sides to form new triangles, resulting in failure of 3D mesh model fusion.

[0021] (2) Low accuracy

[0022] Since it directly performs topological connections at the boundaries without considering the shape and characteristics of the actual model at the boundaries, it is easy to cause too many incorrect connections and incorrect triangle generation, which reduces the quality of the model triangles at the connections and easily leads to many difficult-to-remove triangle defects, such as self-intersection, face inversion and other defects.

[0023] To address the poor stability and low precision of direct topological connections at boundaries, this paper proposes a high-precision 3D mesh fusion method based on an optimal energy field. This method first generates an energy map. Using this energy field map, it selects an optimal boundary line that avoids objects as much as possible to prevent topological connection failures. A constrained De Noël meshing method is then used to triangulate the point cloud at the boundary, ensuring that the newly generated triangles maintain a good shape. Summary of the Invention

[0024] This method is similar to the third method for suturing 3D mesh models: topological connection. However, when the 3D mesh model is too complex, directly topologically connecting the joints can result in excessive deformed, narrow triangles in the fused model, or even topological connection failure. This method addresses this situation, specifically 3D mesh blocks with only a small amount of overlap and no topological relationship, and invents a method for merging meshes quickly while maintaining the accuracy of the original 3D mesh model.

[0025] The present invention first uses the height information of the 3D grid model to generate an energy field map with the ground plane as the reference plane. Then, the overlapping area of ​​the models in the energy field map is found, and a path with the minimum energy is found in the overlapping area as the fusion boundary line of the two models to be fused (the boundary line will avoid places with high objects as much as possible and select flat areas, such as Figure 2 As shown in the figure, the triangular faces falling on the fusion boundary line are extracted from the model to be fused, while the triangular faces within the respective directions of the fusion boundary line remain unchanged. Finally, the constrained De Novo inner meshing method is used to construct triangular faces on the 3D point cloud at the boundary line (the triangle edges within the boundary line are set as fixed edges, and the three vertices in the triangular face cannot come from the same model at the same time). The newly constructed triangular faces are placed into the model to be fused, thus generating a new fused model with a topological structure.

[0026] To achieve the above objectives, the present application discloses a high-precision three-dimensional grid fusion method based on an optimal energy field, comprising the following steps:

[0027] S1: Generate energy field map

[0028] Taking the ground plane as the reference, the 3D grid model is projected onto the ground plane, and the accumulated values ​​of the heights falling within the same pixel are used as the energy value to generate the energy field map.

[0029] S2 searches for the minimum energy boundary line

[0030] Using the energy field diagram, a path with the minimum energy is found in the overlapping area of ​​the two models as the boundary line of the two models;

[0031] S3 extracts vertices at the boundary

[0032] After finding the optimal boundary line of the 3D mesh model, the 3D mesh model in each model direction is kept stationary, and the 3D point clouds and triangular faces outside the boundary are removed; the point clouds corresponding to the triangular faces that fall exactly on the boundary line are extracted;

[0033] S4: Limited De Novo triangular face

[0034] The point cloud at the extracted boundary is used to keep the triangle edges whose two vertices fall inside the boundary line unchanged. At the same time, the three vertices of the new triangle face cannot come from the same model. The point cloud is subjected to constrained DeNovo meshing. The newly generated triangle face is then placed in the model with the boundary line face removed. The final model is then subjected to hole filling and defect removal operations to obtain the final fusion model.

[0035] Preferably, in the energy field map, the larger the pixel value, the more it represents that there is an object above the height threshold; subsequently, by finding a path with the minimum energy in the energy field map, the path will bypass the object as much as possible, so that the fusion boundary line falls on an object that is below the height threshold and flat.

[0036] Preferably, the method for finding the boundary line includes: searching downward through the top pixels side by side, and then finding the minimum energy value as the path for the next step each time, until the bottom pixel position is found, to several paths with different energy intensities; obtaining the path with the minimum energy by sorting, projecting the path onto the three-dimensional grid model, and obtaining the boundary line of the three-dimensional grid model to be fused.

[0037] Preferably, when extracting vertices at the boundary, if both vertices of a triangle edge fall inside the boundary line, the edge is recorded and subsequently set as a fixed edge for topological connection.

[0038] The beneficial effects of the present invention are as follows:

[0039] The present invention uses the generated energy field map to find an optimal boundary line between only a small number of overlapping three-dimensional grid model blocks. This boundary line will bypass objects as much as possible and fall on smooth areas such as roads, ensuring the accuracy of the final fusion model. At the same time, it increases the correctness of the topological connection at the boundary, thereby improving the stability of the model fusion. At the same time, the restricted DeNone method is used to construct triangular faces at the boundary, so that the triangular faces at the boundary will not have defects such as narrowness and self-intersection, and the triangular faces on both sides of the boundary remain basically unchanged. This will basically leave the accuracy of the final fusion model unchanged, ensuring the high accuracy of the fusion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A framework diagram of the present invention;

[0041] Figure 2 Schematic diagram of the minimum energy boundary line;

[0042] Figure 3 Original plan view from above;

[0043] Figure 4 The generated energy field map;

[0044] Figure 5 Schematic diagram of minimum energy path finding;

[0045] Figure 6 Optimal mesh model boundary line results;

[0046] Figure 7 Schematic diagram of the boundary line defining the edge;

[0047] Figure 8 3D mesh model fusion generation results. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention fall within the scope of protection of the present invention.

[0049] refer to Figure 1 The high-precision three-dimensional grid fusion method based on the optimal energy field proposed in this invention mainly includes the following four steps.

[0050] S1: Generate energy field map

[0051] Taking the ground plane as the reference, the 3D grid model is projected onto the ground plane, and the accumulated value of the height within the same pixel is used as the energy value, and a graph such as Figure 4 The energy field diagram shown, Figure 3 This is a top-down view of the original model. In this energy field map, larger pixel values ​​indicate the presence of taller objects. The subsequent process involves finding a path with the lowest energy within the energy field map. This path will bypass objects as much as possible, ensuring that the fusion boundary falls on low, flat areas, such as roads.

[0052] Projecting a 3D mesh model usually refers to converting the 3D model into a 2D image or a visual representation on a 2D plane. This process can be divided into several steps, as follows:

[0053] 1. Model import: First, import the 3D mesh model. This usually involves reading model files such as .obj, .stl, .fbx, etc., and loading the model's vertices, faces, and texture information into memory.

[0054] 2. Coordinate system establishment and model positioning: Establish a spatial rectangular coordinate system with the geometric centroid of the 3D mesh model as the origin O. Adjust the long axis of the model to coincide with the Z axis of the coordinate system to simplify subsequent projection calculations.

[0055] 3. Set the reference surface and projection point: Set a reference surface parallel to the XOY plane outside the 3D model. This will serve as the receiving plane for the projection. Determine the location of the projection point, which is usually the position of the observer or the camera.

[0056] 4. Projection Type Selection: Decide whether to use perspective or orthographic projection. Perspective projection makes distant objects appear smaller, mimicking the natural visual effect of the human eye; orthographic projection, on the other hand, does not change the size of objects and is suitable for engineering drawings and CAD designs.

[0057] 5. Coordinate transformation: Through matrix transformation, the coordinates of the 3D model are converted from the model coordinate system to the observation coordinate system or projection coordinate system. This step may include rotation, translation and scaling operations.

[0058] 6. Projection calculation: For perspective projection, use the perspective projection matrix to calculate the position of a 3D point on a 2D plane. For orthographic projection, ignore the Z coordinate of the 3D point and retain the X and Y coordinates as the 2D coordinates of the projected point.

[0059] 7. Depth buffer or culling: In order to correctly handle occlusion relationships, you may need to use a depth buffer (Z-buffer) or a culling algorithm, such as the painter's algorithm, to determine which surfaces are visible.

[0060] 8. Texture mapping and lighting calculations: If necessary, texture mapping and lighting calculations can be applied to the projected model to increase realism.

[0061] 9. Rendering output: Finally, the projected two-dimensional image is rendered to the screen or saved as an image file.

[0062] S2 searches for the minimum energy boundary line

[0063] By using the energy field map generated in step 1, a path with the minimum energy can be found within the overlapping area of ​​the two models as the boundary line of the two models. The search method is as follows: Figure 5 As shown in the figure, by searching downward from the top pixels side by side, each time finding the path with the minimum energy as the next step until the bottom pixel is found, several paths with different energy intensities are obtained. Finally, the path with the minimum energy is obtained and projected onto the 3D mesh model to obtain the boundary line of the 3D mesh model to be fused.

[0064] S3 extracts vertices at the boundary

[0065] After finding the optimal boundary line of the 3D mesh model, keep the 3D mesh model in the respective model directions unchanged and remove the 3D point cloud and triangular faces outside the boundary. Extract the point cloud corresponding to the triangular face that falls exactly on the boundary line. At the same time, if both vertices of the triangle edge fall inside the boundary line, record the edge and set it as a fixed edge for topological connection later. The result of removing the point cloud and triangular faces outside the boundary line is as follows: Figure 6 shown.

[0066] S4: Limited De Novo triangular face

[0067] Using the point cloud extracted at the boundary, the triangle edges whose two vertices fall inside the boundary line remain unchanged (such as Figure 7 At the same time, it is restricted that the three vertices of the new triangular face cannot come from the same model. The point cloud is subjected to a restricted DeNovo mesh. The newly generated triangular face is then placed into the model with the face at the boundary line removed. The final model is then subjected to hole filling and defect removal operations to obtain the final fusion model.

[0068] Constrained Delaunay Triangulation (CDT) is a triangulation technique commonly used in fields such as computer graphics, computational geometry, and finite element analysis. It is a variant of Delaunay Triangulation and is used to handle triangulation problems with constrained boundaries. Introduction to Delaunay Triangulation Delaunay Triangulation is a point set-based triangulation with the following properties: 1. Hollow circle property: The circumcircle of any triangle does not contain any other input points. 2. Maximum and minimum angles: Among all possible triangulations, the minimum angle of Delaunay Triangulation is the largest, which usually means a more uniform triangle distribution, which is beneficial for numerical stability and visualization.

[0069] The model generated by the three-dimensional grid fusion of the present invention is as follows: Figure 7 shown.

[0070] The beneficial effects of the present invention are as follows:

[0071] The present invention uses the generated energy field map to find an optimal boundary line between only a small number of overlapping three-dimensional grid model blocks. This boundary line will bypass objects as much as possible and fall on smooth areas such as roads, ensuring the accuracy of the final fusion model. At the same time, it increases the correctness of the topological connection at the boundary, thereby improving the stability of the model fusion. At the same time, the restricted DeNone method is used to construct triangular faces at the boundary, so that the triangular faces at the boundary will not have defects such as narrowness and self-intersection, and the triangular faces on both sides of the boundary remain basically unchanged. This will basically leave the accuracy of the final fusion model unchanged, ensuring the high accuracy of the fusion model.

[0072] As used herein, the word "preferred" is intended to serve as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word "preferred" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the naturally inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples.

[0073] Moreover, although the present disclosure has been shown and described with respect to one or implementation, those skilled in the art will think of equivalent variations and modifications based on reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if structurally different from the disclosed structure that performs the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that can be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0074] The functional units in the embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or multiple or more units may be integrated into a single module. The aforementioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc. The aforementioned devices or systems may execute the storage method in the corresponding method embodiment.

[0075] In summary, the above embodiment is one implementation method of the present invention, but the implementation method of the present invention is not limited to the described embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A high-precision three-dimensional grid fusion method based on optimal energy field, characterized in that: The following steps are involved: S1: Generate energy field map Taking the ground plane as the reference, the 3D grid model is projected onto the ground plane, and the accumulated values ​​of the heights falling within the same pixel are used as the energy value to generate the energy field map. S2 searches for the minimum energy boundary line Using the energy field diagram, a path with the minimum energy is found in the overlapping area of ​​the two models as the boundary line of the two models; S3 extracts vertices at the boundary After finding the optimal boundary line of the 3D mesh model, the 3D mesh model in each model direction is kept stationary, and the 3D point clouds and triangular faces outside the boundary are removed; the point clouds corresponding to the triangular faces that fall exactly on the boundary line are extracted; S4: Limited De Novo triangular face The point cloud at the extracted boundary is used to keep the triangle edges whose two vertices fall inside the boundary line unchanged. At the same time, the three vertices of the new triangle face cannot come from the same model. The point cloud is subjected to constrained DeNovo meshing. The newly generated triangle face is then placed in the model with the boundary line face removed. The final model is then subjected to hole filling and defect removal operations to obtain the final fusion model.

2. The high-precision three-dimensional grid fusion method based on optimal energy field according to claim 1 is characterized in that: In the energy field map, the larger the pixel value, the more likely it is that there is an object above the height threshold. Subsequently, a path with the minimum energy is found in the energy field map. This path will bypass the object as much as possible, so that the fusion boundary line falls on a flat object below the height threshold.

3. The high-precision three-dimensional grid fusion method based on optimal energy field according to claim 1 is characterized in that: The method for finding the boundary line includes: searching downward through the top pixels side by side, and then finding the minimum energy value as the path for the next step each time, until the bottom pixel position is found, to several paths with different energy intensities; obtaining the path with the minimum energy by sorting, and projecting the path onto the three-dimensional grid model to obtain the boundary line of the three-dimensional grid model to be fused.

4. The high-precision three-dimensional grid fusion method based on optimal energy field according to claim 1 is characterized in that: When extracting vertices at the boundary, if both vertices of a triangle edge fall inside the boundary line, the edge is recorded and subsequently set as a fixed edge for topological connection.

Images