Tetrahedron surface labeling Mesh construction method and system combined with UAV segmentation image

Abstract

The present invention provides a tetrahedron surface marking Mesh construction method and system combined with drone segmentation images, including first obtaining the original drone oblique image, performing aerial triangulation on the image to obtain the camera internal and external orientation elements; densely reconstructing the drone image processed by aerial triangulation to obtain dense point cloud and visual information of the point cloud; segmenting the original drone oblique image to obtain segmented blocks; constructing a tetrahedron by point-by-point insertion on the basis of the dense point cloud; projecting the triangular faces in the constructed tetrahedron onto the image, screening out the triangular faces falling in the segmented area; comparing the depth value of the triangular face with the depth map, and marking the triangular faces with consistent depth as the surface local mesh; calculating the weight of the tetrahedron through the line, adding the weight in the s-t map, and then solving the final continuous surface. The model constructed by the present invention is closer to the shape of the real three-dimensional object, has low implementation complexity, fast speed, and is easier to be applied in large-scale three-dimensional modeling scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional modeling and virtual reality processing, and in particular to a tetrahedron surface marking Mesh construction technical solution combined with unmanned aerial vehicle segmentation images. Background Art

[0002] Since point clouds can capture a wide range of 3D data and are applicable to large scenes and buildings, constructing 3D Mesh networks based on point cloud data is currently the most commonly used method for constructing mesh surfaces. There are three main methods for constructing Mesh surfaces: mesh surfaces, parametric surfaces, and implicit surfaces.

[0003] Mesh surface is to build multiple polygonal models in the scene, and these polygonal models need to include all the points of the point cloud. This technology is represented by the Delaunay triangulation method. First, the point cloud in the scene is Delaunay triangulated to construct a Delaunay tetrahedron. According to the set conditions, the triangles that meet the requirements are selected to complete the surface reconstruction of the point cloud. Delaunay triangulation network construction technology is suitable for point clouds without noise and dense enough. In this case, a better network construction effect can be obtained. However, when there is noise in the point cloud and the model to be constructed is more complex, it lacks stability and flexibility, and the cost is higher.

[0004] The Mesh surface reconstruction method of parametric surface can use the entire point cloud as input for global surface fitting, or divide the point cloud data into blocks for partial surface fitting. Classic parametric surface algorithms include B-spline surface and Bezier surface. This method has the problem of balancing global fitting and local details. When global surface fitting is performed on the entire point cloud, the accuracy of local details may be sacrificed. On the contrary, although local surface fitting by block can capture details, it may cause discontinuity between blocks, affecting the overall smoothness and consistency.

[0005] Implicit surface reconstruction is to construct an implicit function in three-dimensional space to represent the location of the object surface. The Poisson implicit surface reconstruction method is the most commonly used implicit surface method. The Poisson equation is a partial differential equation that can be used to calculate the gradient field of point cloud data to derive the surface. However, the accuracy of this method is very sensitive to the noise and outliers of the point cloud, which can cause holes and loss of details in the reconstructed surface. At the same time, the computational efficiency is greatly reduced when processing dense point clouds. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a tetrahedron surface labeling mesh construction method combined with drone segmentation images. Through this method, watertight mesh construction of complex geometric structure objects and large-scale objects can be realized efficiently and with high quality, so as to reduce the space cost and time cost of the surface reconstruction process.

[0007] In order to achieve the above object, the present invention proposes a tetrahedron surface marking Mesh construction method combined with drone segmentation image, comprising the following steps:

[0008] First, the original UAV oblique image is obtained, and the image is processed by aerial triangulation to obtain the internal and external orientation elements of the camera; then, the UAV image processed by aerial triangulation is densely reconstructed to obtain dense point clouds and visual information of point clouds;

[0009] Segment the original UAV oblique image to obtain segmented blocks;

[0010] Based on the dense point cloud, the tetrahedron is constructed by point-by-point insertion;

[0011] The triangular faces in the constructed tetrahedron are projected onto the image, and the triangular faces falling within the segmented area are screened out; the depth values ​​of the triangular faces are compared with the depth map, and the triangular faces with consistent depth are marked as local meshes on the surface;

[0012] Based on the tetrahedron, the source point s and the sink point t are introduced to obtain the corresponding ST graph, the weights of the tetrahedron passing rays are calculated, the weights in the ST graph are added, and then the final continuous surface is solved.

[0013] Moreover, the implementation method of segmenting the original drone oblique image is, based on the preliminary segmentation using the segmentation model segment—anything, an improvement is proposed for the situation in which surfaces of the same object in different directions are segmented into one category in the segmentation result, a normal vector map is introduced, and the segmentation image result is further optimized by calculating the change of the normal vector gradient.

[0014] Furthermore, when the triangular faces of the constructed tetrahedron are projected onto the image, the implementation is as follows,

[0015] Assume that the internal reference element has focal length f x 、f y , like the principal point offset u0, v0, constitute the camera intrinsic parameter matrix K, which can realize the conversion between the pixel coordinate system and the camera coordinate system.

[0016]

[0017]

[0018] Where u and v are pixel coordinates, XC , Y C , Z C is the camera coordinate system, d is the depth value;

[0019] Assume that the external parameter elements constitute the camera external parameter matrix R, t, R is the rotation matrix, t is the translation matrix, which can realize the conversion between the camera coordinate system and the world coordinate system.

[0020]

[0021] Among them, X, Y, and Z are the world coordinate systems;

[0022] The final conversion formula between the pixel coordinate system and the world coordinate system is as follows:

[0023]

[0024] The three vertices of the triangular face in the tetrahedron are projected onto the two-dimensional image using the above conversion formula, and compared with the extracted segmented blocks to determine whether the triangular face falls within a certain segmented block, thereby preliminarily screening out potential triangular faces as surfaces.

[0025] Moreover, the depth value of the triangle surface is compared with the depth map, which is implemented as follows:

[0026] After calculating the depth values ​​of the triangle vertices in the three-dimensional space using the conversion formula between the world coordinate system and the pixel coordinate system, the depth comparison analysis is performed by extracting the depth values ​​at the two-dimensional pixel coordinate positions corresponding to the three vertices of the triangle in the depth map;

[0027] When performing depth comparison, first, the depth values ​​calculated from the vertices of the triangle surface are compared with the depth values ​​of the corresponding positions in the depth map; second, the depth values ​​calculated by interpolation inside the triangle surface are compared with the depth values ​​of the corresponding positions in the depth map; when these two sets of depth values ​​show a high degree of consistency, the triangle surface is judged to be the actual surface of the object and marked accordingly;

[0028] All the triangular faces of the tetrahedron are traversed so that partial surface triangulation can be marked and identified.

[0029] Moreover, the weights of the tetrahedron passing rays are calculated and the weights in the st graph are added. The implementation method is as follows:

[0030] Suppose a 3D point of a dense point cloud in the scene and a photographic center are used to restore a ray. This ray will pass through several triangular faces in the constructed Delaunay tetrahedrons, some of which are pre-marked surfaces and some are unmarked triangular faces. When a ray passes through a pre-marked surface, the tetrahedron passed by the ray segment from the camera to the marked triangular face is marked as s, and the tetrahedron passed by the ray segment from the marked triangular face to the 3D point in the point cloud is marked as t. When assigning weights to the pre-marked surfaces, a minimum value is assigned to these pre-marked surfaces. When the ray does not pass through the pre-marked surface, the graph cut method is continued to be used to weight the tetrahedrons and triangular faces.

[0031] On the other hand, the present invention also provides a tetrahedron surface labeling Mesh construction system combined with drone segmentation images, which is used to implement the tetrahedron surface labeling Mesh construction method combined with drone segmentation images as described above.

[0032] Furthermore, the following modules are included,

[0033] The first module is used to first obtain the original UAV oblique image, perform aerial triangulation on the image to obtain the camera's internal and external orientation elements; then densely reconstruct the drone image processed by aerial triangulation to obtain dense point cloud and visual information of the point cloud;

[0034] The second module is used to segment the original UAV oblique image to obtain segmented blocks;

[0035] The third module is used to construct tetrahedrons based on dense point clouds by point-by-point insertion;

[0036] The fourth module is used to project the triangular faces in the constructed tetrahedron onto the image, filter out the triangular faces that fall within the segmented area, compare the depth values ​​of the triangular faces with the depth map, and mark the triangular faces with consistent depth as the local mesh of the surface;

[0037] The fifth module is used to introduce source point s and sink point t based on the tetrahedron, obtain the corresponding ST graph, calculate the weight of the tetrahedron passing line, add the weight in the ST graph, and then solve the final continuous surface.

[0038] Alternatively, it includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned tetrahedral surface marking Mesh construction method combined with drone segmentation image.

[0039] Alternatively, it includes a readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for constructing a tetrahedral surface marker Mesh in combination with drone segmented images as described above is implemented.

[0040] The present invention is a three-dimensional triangulated network construction scheme based on Delaunay tetrahedron and graph cut. The tetrahedral triangular faces falling inside the segmented block are preliminarily marked using the prior information of the segmented image to obtain a potential surface set. The triangular faces in the set are then subjected to secondary depth judgment. The triangular faces whose depth values ​​are consistent with the depth of the depth map are marked as surfaces to obtain partial surface triangular faces. Finally, the remaining surfaces are cut out using a penetration line to assign triangular face weights and a maximum flow-minimum cut algorithm to achieve the goal of building a watertight mesh network.

[0041] The differences and advantages of the present invention and the prior art are:

[0042] 1. The present invention constructs a watertight mesh model based on the Delaunay meshing-graph cut method, so that the mesh model can be closer to the shape of the real three-dimensional object. When there is noise in the input point cloud data, the existing meshing method lacks the necessary stability. This instability is mainly manifested in that the constructed Mesh model may introduce interference caused by point cloud noise, resulting in a deviation between the final model and the shape of the actual three-dimensional object.

[0043] 2. The present invention proposes a method for constructing a mesh network with the ability to preliminarily mark the surface of a model. The advantage of this method is that it can pre-mark the surface of the model during the process of constructing the mesh network. This significant feature helps to reduce the amount of data required to process the surface in the subsequent graph cutting method, thereby effectively reducing the computing cost and improving the efficiency of the entire mesh construction process. In contrast, the prior art usually adopts an indiscriminate approach when constructing a mesh network, performing weighted calculations on the triangular faces of all tetrahedrons. Since this method does not pre-mark the surface, the amount of data involved is greatly increased.

[0044] 3. The computational complexity of the technical solution of the present invention is low, the computational speed is fast, and it is easier to apply to large-scale 3D modeling scenarios. Traditional Mesh model construction methods often involve complex calculation processes, especially when dealing with large-scale scenes. This high complexity leads to a significant increase in model construction time. This time delay limits the practical applicability of traditional methods in large-scale 3D modeling applications.

[0045] The solution of the present invention is simple and convenient to implement, has strong practicality, solves the problems of low practicality and inconvenience in actual application existing in related technologies, can improve user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0047] Figure 2 FIG. 4 is a light transmission line diagram according to an embodiment of the present invention.

[0048] Figure 3 It is a comparison chart of the application effects of the technical solution of the embodiment of the present invention and the existing technical solution. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0050] The present invention is based on the graph cut method and regards the surface construction problem as an optimization labeling problem. First, the original UAV oblique image is obtained, and the image is processed by aerial triangulation to obtain the camera's internal and external orientation elements; the drone image processed by aerial triangulation is densely reconstructed to obtain a dense point cloud and the visual information of the point cloud; the original UAV image is segmented to obtain segmented blocks; a tetrahedron is constructed for the dense point cloud using a point-by-point insertion algorithm, and the desired triangulated network is contained in the tetrahedron; the triangular faces of the tetrahedron are projected onto the image to determine whether the triangular face falls within a segmented block. Each triangular face in the mesh network constructed by the present invention cannot cross a segmented block; for the triangular faces that meet the segmented block conditions, their depth values ​​are compared with the depth map, and the depth values ​​of the triangular faces are compared with the depth map. Figure 1 If the triangle is marked as a surface, we call the marked triangle mesh a local mesh. Since the marked triangles cannot form a continuous triangle mesh, we need to calculate the weights of the tetrahedron rays and add the weights in the ST graph; the minimum cut-maximum flow algorithm is used to solve the final surface.

[0051] See also Figure 1 , a tetrahedron surface marking Mesh construction method combined with drone segmentation images proposed in an embodiment of the present invention includes the following steps:

[0052] Step 1. Obtain dense point cloud data: First, obtain the original UAV oblique image, perform aerial triangulation on the image to obtain the camera's internal and external orientation elements; then perform dense reconstruction on the drone image processed by aerial triangulation to obtain dense point cloud and visual information of the point cloud;

[0053] The specific implementation method preferably recommended in the embodiment is as follows:

[0054] 1.1 Aerial triangulation. After obtaining the image of an object taken by a drone, the drone image is processed by aerial triangulation, including feature extraction, feature matching, incremental reconstruction and bundle adjustment, to obtain the image pose, feature points on the image and three-dimensional sparse points.

[0055] 1.2 Dense reconstruction. The present invention preferably recommends using a semi-global matching algorithm (SGM) to generate a disparity map for dense reconstruction. The grayscale difference between the left and right images is calculated pixel by pixel as the stereo matching cost. By accumulating the disparity cost of the current pixel and the neighboring pixels, the global optimal matching result is obtained to generate a disparity map. Finally, the disparity map can be converted into a depth map according to the baseline length between the two images to obtain a dense point cloud.

[0056] Step 2. Obtain image segmentation results: Use segmentation technology to segment the original UAV oblique image to obtain several segmented blocks;

[0057] The present invention further proposes that the segmentation method used is: based on the segmentation technology segment—anything, the present invention proposes an improvement on the situation where surfaces of the same object in different directions are segmented into one category in the segmentation results, introduces a normal vector map, and further optimizes the segmented image results by calculating the changes in the normal vector gradient.

[0058] In the embodiment, the drone image is firstly segmented using the segment-anything segmentation model in the prior art. In order to obtain the segmented blocks applicable to the present invention, the embodiment proposes an optimization scheme for the situation where the facades of the same object in different directions are mistakenly classified into one category: by introducing the object normal vector image, the present invention can accurately obtain the normal vector value of each pixel position in the image. In the same facade area of ​​an object, the observed normal vector values ​​are relatively consistent, showing only slight changes. However, at the different facade corner areas of the object, the normal vector values ​​are significantly different. The present invention utilizes this feature to determine the surface direction change by calculating the difference between the normal vectors between adjacent pixels, that is, the normal vector gradient. In the area where the normal vector gradient value reaches the maximum, it usually marks the segmentation boundary of different facade directions. The present invention extracts the two-dimensional pixel coordinate values ​​of these key areas and merges them with the boundary pixel values ​​of the original segmented blocks, thereby optimizing the segmented blocks.

[0059] Step 3. Construct Delaunay tetrahedron: Based on the dense point cloud obtained in step 1, a point-by-point insertion algorithm is used to construct a tetrahedron;

[0060] The purpose of constructing a Delaunay tetrahedron is to preliminarily construct the surface shape of an object. The dense point cloud obtained through dense reconstruction is constructed using a point-by-point insertion algorithm. The vertices and directed edges of the dual graph (i.e., the Voronoi graph) of the Delaunay tetrahedron correspond to the vertices and directed edges of the st graph. The vertices of the st graph represent tetrahedrons, and the directed edges represent triangular faces with directions. This step can be implemented using existing technology, and the present invention will not go into details.

[0061] Step 4. Mark the local surface: Project the triangular faces in the constructed tetrahedron onto the image, and filter out the triangular faces that fall within the segmented area; compare the depth values ​​of the triangular faces with the depth map, and mark the triangular faces with consistent depth as the local mesh of the surface;

[0062] The present invention proposes to combine the segmentation image results to preliminarily mark the triangular faces in the tetrahedron. The triangular faces of the tetrahedron in the three-dimensional coordinates can be projected onto the two-dimensional image using the camera's internal and external orientation elements, and the segmentation area boundary pixel values ​​of the segmented image can be extracted to determine whether the triangular face falls into a certain segmentation area.

[0063] The present invention further proposes to use a depth judgment method to mark the triangulated mesh of the surface of a three-dimensional object. The depth value information of the three vertices of the triangular face is compared with the depth map information of the projection image of the triangular face. If the depth is consistent, the triangular face is marked as a surface. The present invention partially marks the tetrahedral triangular faces constructed in the three-dimensional scene, reduces the input and calculation amount of the subsequent graph cutting algorithm, cuts out the surface of the object with a more efficient method, reduces the data burden, and improves the efficiency of Mesh construction.

[0064] The specific implementation method preferably recommended in the embodiment is as follows:

[0065] 4.1 Projection of triangulated surface to segmented image In the embodiment, based on the internal and external reference elements of the image after aerial triangulation, the internal and external reference elements can be converted into a transformation matrix from a three-dimensional coordinate system to a two-dimensional image coordinate system. The internal reference elements include the focal length f x 、f y , like the principal point offsets u0 and v0, constitute the camera intrinsic parameter matrix K, which can realize the conversion between the pixel coordinate system and the camera coordinate system.

[0066]

[0067]

[0068] Where u and v are pixel coordinates, X C , Y C , Z C is the camera coordinate system, and d is the depth value.

[0069] The external parameter elements constitute the camera external parameter matrix R, t, R is the rotation matrix, t is the translation matrix, which can realize the conversion between the camera coordinate system and the world coordinate system.

[0070]

[0071] Among them, X, Y, and Z are the world coordinate systems.

[0072] The final conversion formula between the pixel coordinate system and the world coordinate system is as follows

[0073]

[0074] The embodiment uses the above conversion formula to project the three vertices of the triangular face in the tetrahedron onto the two-dimensional image, and compares it with the extracted segmented blocks to determine whether the triangular face falls within a certain segmented block, thereby preliminarily screening out potential triangular faces as surfaces.

[0075] 4.2 Determination of depth value of triangular face. The embodiment first uses the conversion formula between the world coordinate system and the pixel coordinate system to calculate the depth value of the triangular face vertices in the three-dimensional space. Subsequently, the present invention performs depth comparison analysis by extracting the depth value at the two-dimensional pixel coordinate position corresponding to the three vertices of the triangular face in the depth map. When performing depth comparison, the embodiment considers two key factors: one is to compare the depth value calculated by the triangular face vertices with the depth value of the corresponding position in the depth map; the other is to compare the depth value calculated by interpolation inside the triangular face with the depth value of the corresponding position in the depth map. When both sets of depth values ​​show a high degree of consistency, the present invention determines that the triangular face is the actual surface of the object and marks it accordingly. Furthermore, the method traverses the triangular faces of all tetrahedrons, so that partial surface triangulation can be marked and identified. This step improves the accuracy of three-dimensional reconstruction and modeling, and is particularly suitable for depth analysis and surface recognition of complex scenes.

[0076] Step 5. Cut the surface using the graph cut method: The Voronoi dual graph of the Delaunay tetrahedron is represented as a directed graph G = (V, E), where V represents a set of vertices, each vertex corresponds to a tetrahedron; E represents a set of edges, representing the triangular faces between the tetrahedrons. In addition, two special points are introduced, the source point s (indicating the "starting point" or "source") and the sink point t (indicating the "end point" or "target"). The vertices of the tetrahedron are connected to the source point s or sink point t of the directed graph, so the vertices of the directed graph are divided into two categories, S and T, where s∈S, t∈T, making the directed graph an st graph. Calculate the weights for the tetrahedron penetration lines and add the weights in the st graph; use the minimum cut-maximum flow algorithm to solve the final continuous surface.

[0077] The specific implementation method preferably recommended in the embodiment is as follows:

[0078] 5.1 Assign weights to triangles by passing rays. Figure 2, a 3D point of a dense point cloud in the scene and a photographic center (camera) can restore a ray of light, which will pass through several triangular faces in the constructed Delaunay tetrahedrons, including surfaces pre-marked in the embodiment and unmarked triangular faces. When a ray of light passes through a pre-marked surface, the tetrahedron passed by the ray segment from the camera to the marked triangular face is marked as s, and the tetrahedron passed by the ray segment from the marked triangular face to the 3D point in the point cloud is marked as t. When assigning weights to the pre-marked surfaces, the embodiment assigns a minimum value, i.e., the value "1", to these pre-marked surfaces based on the weight formula for triangular faces using the traditional method. When the ray of light does not pass through a pre-marked surface, the traditional graph cut method continues to be used to weight the tetrahedrons and triangular faces. This weight allocation strategy optimizes the ray tracing process, improves computational efficiency, and ensures the accuracy of the processing results.

[0079] 5.2 Maximum flow-minimum cut cutting surface. The minimum cut problem can also be regarded as a binary labeling problem of vertices. If the labels of two tetrahedrons in the neighborhood of a triangle are different, then this triangle is the reconstructed surface. You can find a path with the minimum cost value to get the final surface of the object.

[0080] In order to facilitate understanding of the technical effects of the present invention, the following specific examples of applying the embodiments of the present invention are provided:

[0081] Compared with the prior art, the embodiments of the present invention show significant improvement effects, such as Figure 3 As shown. This embodiment optimizes the segmentation results and can effectively process the surfaces of the same object in different directions, dividing it into two independent blocks, thereby achieving a more refined segmentation. In addition, this embodiment also optimizes the Mesh network construction results, especially in the building facade area, the generated mesh network model is flatter and the triangular mesh is more regular.

[0082] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0083] In some possible embodiments, a tetrahedron surface labeling Mesh construction system combining drone segmentation images is provided, comprising the following modules:

[0084] The first module is used to first obtain the original UAV oblique image, perform aerial triangulation on the image to obtain the camera's internal and external orientation elements; then densely reconstruct the drone image processed by aerial triangulation to obtain dense point cloud and visual information of the point cloud;

[0085] The second module is used to segment the original UAV oblique image to obtain segmented blocks;

[0086] The third module is used to construct tetrahedrons based on dense point clouds by point-by-point insertion;

[0087] The fourth module is used to project the triangular faces in the constructed tetrahedron onto the image, filter out the triangular faces that fall within the segmented area, compare the depth values ​​of the triangular faces with the depth map, and mark the triangular faces with consistent depth as the local mesh of the surface;

[0088] The fifth module is used to introduce source point s and sink point t based on the tetrahedron, obtain the corresponding ST graph, calculate the weight of the tetrahedron passing line, add the weight in the ST graph, and then solve the final continuous surface.

[0089] In some possible embodiments, a tetrahedral surface labeling Mesh construction system combined with drone segmentation images is provided, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a tetrahedral surface labeling Mesh construction method combined with drone segmentation images as described above.

[0090] In some possible embodiments, a tetrahedral surface labeling Mesh construction system combined with drone segmentation images is provided, including a readable storage medium, on which a computer program is stored. When the computer program is executed, a tetrahedral surface labeling Mesh construction method combined with drone segmentation images as described above is implemented.

[0091] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A tetrahedron surface labeling Mesh construction method combined with drone segmentation images, characterized by: The method includes the following steps: firstly, obtaining the original UAV oblique image, performing aerial triangulation on the image to obtain the internal and external orientation elements of the camera; then, densely reconstructing the UAV image processed by aerial triangulation to obtain a dense point cloud and visual information of the point cloud; Segment the original UAV oblique image to obtain segmented blocks; Based on the dense point cloud, the tetrahedron is constructed by point-by-point insertion; The triangular faces in the constructed tetrahedron are projected onto the image, and the triangular faces falling within the segmented area are screened out; the depth values ​​of the triangular faces are compared with the depth map, and the triangular faces with consistent depth are marked as local meshes on the surface; Based on the tetrahedron, the source point s and the sink point t are introduced to obtain the corresponding ST graph, the weights of the tetrahedron passing rays are calculated, the weights in the ST graph are added, and then the final continuous surface is solved.

2. The method for constructing a tetrahedron surface labeling mesh in combination with drone segmentation images according to claim 1, characterized in that: The implementation method of segmenting the original drone oblique image is to propose an improvement based on the preliminary segmentation using the segmentation model segment-anything, aiming at the situation where the surfaces of the same object in different directions are segmented into one category in the segmentation result, introduce a normal vector map, and further optimize the segmented image result by calculating the change of the normal vector gradient.

3. The method for constructing a tetrahedron surface labeling mesh in combination with drone segmentation images according to claim 1, characterized in that: When the triangular faces of the constructed tetrahedron are projected onto the image, the implementation is as follows: Assume that the internal reference element has focal length f x 、f y , like the principal point offset u0, v0, constitute the camera intrinsic parameter matrix K, which can realize the conversion between the pixel coordinate system and the camera coordinate system. Where u and v are pixel coordinates, X C , Y C , Z C is the camera coordinate system, d is the depth value; Assume that the external parameter elements constitute the camera external parameter matrix R, t, R is the rotation matrix, t is the translation matrix, which can realize the conversion between the camera coordinate system and the world coordinate system. Among them, X, Y, and Z are the world coordinate systems; The final conversion formula between the pixel coordinate system and the world coordinate system is as follows: The three vertices of the triangular face in the tetrahedron are projected onto the two-dimensional image using the above conversion formula, and compared with the extracted segmented blocks to determine whether the triangular face falls within a certain segmented block, thereby preliminarily screening out potential triangular faces as surfaces.

4. The method for constructing a tetrahedron surface labeling mesh in combination with drone segmentation images according to claim 1, characterized in that: The depth value of the triangle surface is compared with the depth map. The implementation method is as follows: After calculating the depth values ​​of the triangle vertices in the three-dimensional space using the conversion formula between the world coordinate system and the pixel coordinate system, the depth comparison analysis is performed by extracting the depth values ​​at the two-dimensional pixel coordinate positions corresponding to the three vertices of the triangle in the depth map; When performing depth comparison, first, the depth values ​​calculated from the vertices of the triangle surface are compared with the depth values ​​of the corresponding positions in the depth map; second, the depth values ​​calculated by interpolation inside the triangle surface are compared with the depth values ​​of the corresponding positions in the depth map; when these two sets of depth values ​​show a high degree of consistency, the triangle surface is judged to be the actual surface of the object and marked accordingly; All the triangular faces of the tetrahedron are traversed so that partial surface triangulation can be marked and identified.

5. The method for constructing a tetrahedral surface marker mesh in combination with drone segmentation images according to claim 1, 2, 3 or 4, characterized in that: The weights of the tetrahedron passing rays are calculated and the weights are added to the st graph. The implementation method is as follows: Suppose a 3D point of a dense point cloud in the scene and a photographic center are used to restore a ray. This ray will pass through several triangular faces in the constructed Delaunay tetrahedrons, some of which are pre-marked surfaces and some are unmarked triangular faces. When a ray passes through a pre-marked surface, the tetrahedron passed by the ray segment from the camera to the marked triangular face is marked as s, and the tetrahedron passed by the ray segment from the marked triangular face to the 3D point in the point cloud is marked as t. When assigning weights to the pre-marked surfaces, a minimum value is assigned to these pre-marked surfaces. When the ray does not pass through the pre-marked surface, the graph cut method is continued to be used to weight the tetrahedrons and triangular faces.

6. A tetrahedron surface marking Mesh construction system combined with drone segmentation images, characterized by: Used to implement a tetrahedron surface marking Mesh construction method combined with drone segmentation images as described in any one of claims 1 to 5, Includes the following modules, The first module is used to first obtain the original UAV oblique image, perform aerial triangulation on the image to obtain the camera's internal and external orientation elements; then densely reconstruct the drone image processed by aerial triangulation to obtain dense point cloud and visual information of the point cloud; The second module is used to segment the original UAV oblique image to obtain segmented blocks; The third module is used to construct tetrahedrons based on dense point clouds by point-by-point insertion; The fourth module is used to project the triangular faces in the constructed tetrahedron onto the image, and filter out the triangular faces that fall within the segmented area; compare the depth values ​​of the triangular faces with the depth map, and mark the triangular faces with consistent depth as the surface local mesh localmesh; The fifth module is used to introduce source point s and sink point t based on the tetrahedron, obtain the corresponding ST graph, calculate the weight of the tetrahedron passing line, add the weight in the ST graph, and then solve the final continuous surface.

7. A computer device, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a tetrahedral surface marking Mesh construction method combined with drone segmentation images as described in any one of claims 1-5.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed, it implements the tetrahedral surface marking Mesh construction method combined with drone segmentation imagery as described in any one of claims 1 to 5.

Images