Polyhedron Optimization Representation Method for Sparse Point Clouds for Photogrammetry Planning

Abstract

The present invention discloses a polyhedron optimization representation method for sparse point cloud for close-up photography planning. It divides the point cloud obtained by close-up photography into blocks, fits planes to the block point clouds and clusters them to complete the segmentation of the point cloud, and finally optimizes the geometric structure information of the point cloud in the form of an outer polyhedron. The present invention solves the problem that the existing close-up photography trajectory planning method is difficult to adapt to various scenes; it has the advantages of being widely applicable to various scenes, requiring no manual intervention, saving time, and being able to achieve coplanar and connected point cloud segmentation.

Description

Technical Field

[0001] The present invention relates to the field of photogrammetry and remote sensing, and in particular to a sparse point cloud polyhedron optimization representation method for close-to-photographic trajectory planning, wherein point cloud segmentation based on point cloud block clustering is a key technology of the present invention. Background Art

[0002] In recent years, with the rapid development of the information age, the social production has a growing demand for refined digital information products, and real-scene 3D models have gradually become a new basic surveying and mapping product. Due to its advantages such as flexible data collection, high image resolution, and fine models, drone close-up photogrammetry has become a new photogrammetry method for making ultra-fine real-scene 3D models and performing precise measurements;

[0003] The core idea of ​​close-up photogrammetry is "from nothing to something" and "from coarse to fine". First, low-resolution images of the target are obtained through regular UAV flights to generate a rough model. Then, guided by the rough model, the trajectory and posture of the UAV's automatic fine data collection are planned. In recent years, scene-aware path planning methods have been studied in depth. This method is divided into an exploration phase and an exploitation phase (Explore-Then-Exploit, ETE), that is, using existing scene prior information such as sparse point clouds to efficiently generate the necessary information for trajectory planning. However, this type of method often lacks clear constraints on the regularity of shooting points, resulting in scattered and irregular shooting points, forcing the UAV to frequently adjust its flight movements, affecting data collection efficiency and energy utilization. In contrast, close-up photography roughly represents the scene through a combination of multiple planes in the coarse flight stage, and constructs a strip-type photography network in the fine flight stage to ensure that the UAV takes most of the shots along a straight trajectory, reducing pause points, saving time and energy;

[0004] Targets that are close to photogrammetry include unconventional ground such as landslides, dams, and high slopes, as well as artificial target objects such as building facades, tall ancient buildings, and landmark buildings. There are two main traditional methods for dealing with these targets: (1) For artificial targets such as buildings, the traditional method is to abstract them into simple basic geometric elements such as facades and columns and combinations of multiple basic units through manual interpretation. (2) For natural environment and other landforms, traditional methods mainly use normal vector clustering or region growing methods for expansion, set critical thresholds to segment the landforms, and use each segmented surface as a basic unit for trajectory planning. However, at the current stage, with the increasing requirements for automated and intelligent operations, people are no longer satisfied with this form of operation that requires frequent manual intervention, and are looking for a rough geometric proxy processing paradigm that can be widely used in various scenarios. However, there are two main problems with traditional methods: (1) For ancient buildings or natural landforms with complex shapes, their scenes cannot be simply described using a single regular shape, and a large amount of manual intervention is required through the combination of basic units. (2) Calculating dense normal vectors from the initial point cloud is a very time-consuming process, and normal vector clustering cannot solve the problem of segmenting point clouds that are coplanar but not connected, resulting in topological ambiguity in the model (it is usually believed that two unconnected planar units are two separate instances in space). As a result, it is impossible to correctly restore its geometric shape from a topological perspective using an outer polyhedron, increasing the risk of collision between the track points and the target scene during close-up photography by the drone.

[0005] Therefore, it is necessary to develop a sparse point cloud polyhedron optimization representation method for close photography planning that can be widely used in various scenarios, does not require manual intervention, saves time, and can achieve coplanar and connected point cloud segmentation. Summary of the invention

[0006] The purpose of the present invention is to provide a sparse point cloud polyhedron optimization representation method for close-up photography planning. The method is based on a sparse three-dimensional point cloud polyhedron optimization representation method. The method can simplify the surface structure of complex targets and retain their intrinsic geometric shape to the greatest extent. On this basis, a photography network that can be used for close-up photography fine data collection can be efficiently planned; the problem that the existing close-up photography trajectory planning method is difficult to adapt to various scenarios is solved.

[0007] In order to achieve the above-mentioned purpose, the technical scheme of the present invention is: a polyhedron optimization representation method for sparse point clouds close to photography planning, which divides the point clouds close to photography into blocks, completes the segmentation of the point clouds by fitting planes and clustering the block point clouds, and finally optimizes the geometric structure information of the point clouds in the form of an outer polyhedron. The present invention first uses conventional ground and aerial photogrammetry to obtain the initial image of the target object, and reconstructs the initial rough point cloud of the target scene by motion recovery structure method, and constructs the skeleton graph of the point cloud based on morphological and topological features; then, the point cloud is divided into blocks based on the octree, and the voxel plane of each sub-block point cloud is fitted, and the normal vector orientation and plane position are corrected by querying the external orientation elements of the film recorded by the photography control field or the feature points of the point cloud skeleton graph, and the voxel planes of the blocks are merged into segmentation planes by graph clustering; finally, the polyhedron combination is converted into a 0-1 integer linear programming problem by using the segmentation plane parameters and the original point cloud, and the polyhedron outer packaging model suitable for close trajectory planning is solved. The method of the present invention innovatively segments the point cloud by block clustering, avoiding the process of constructing a mesh required for traditional triangle normal vector region growing segmentation, and effectively solves the segmentation problem of coplanar but disconnected point clouds. By constructing an outsourced model, the collision risk of drones in close-up photography is reduced.

[0008] In the above technical scheme, the present invention uses the initial sparse point cloud of the target scene as input data, divides the point cloud into blocks and then clusters it, segments the surface feature structure of the point cloud, and combines the segmentation planes into a polyhedron (i.e., a polyhedron geometric proxy model) using a binary linear programming method. The polyhedron simplifies the surface structure of the complex target and retains its intrinsic geometric shape to the greatest extent. On this basis, a photographic network for close-to-photographic fine data collection is efficiently planned, and then a fine three-dimensional model of the complex target (such as a three-dimensional image) can be constructed through three-dimensional reconstruction. Figure 3 As shown, Figure 3 The method of the present invention can be widely used in various scenes, does not require manual intervention, saves time, can achieve coplanar and connected point cloud segmentation, and can effectively avoid the collision between the track point and the target scene;

[0009] The specific method includes the following steps:

[0010] Step 1: First, the point cloud data (i.e., input data) is expressed in voxel blocks using an octree, and a plane is fitted based on the voxels and its adjacency table is established;

[0011] Step 2: Then use the camera position of the rough flight image or the feature points of the point cloud skeleton to correct the plane normal vector orientation, and establish the voxel connection matrix based on the adjacency relationship between voxels and the similarity of the normal vectors of adjacent voxels;

[0012] Step 3: Finally, graph clustering is performed on the connection matrix, and a new plane is fitted using all points belonging to the same cluster to form a split plane. After outsourcing correction, the split plane can be combined into a polyhedron through binary integer linear programming (i.e., 0-1 integer linear programming).

[0013] In the above technical solution, in step one, the sparse point cloud of the three-dimensional scene obtained by the rough flight of the UAV is divided into octree voxel blocks, and the voxel coordinates calculated by the octree child node index are used to establish a voxel adjacency table for storing the adjacency relationship between voxels; the nearest neighbor distances between adjacent voxel points are traversed two by two, and the adjacency relationship whose nearest neighbor distance is less than the minimum resolution of the point cloud is modified; the voxel point cloud whose merged points are less than the set threshold is set, and the adjacency table is updated.

[0014] In the above technical solution, in step 1, after voxelization and block division by octree, a voxel coordinate system (0 v , X v , Y v , Z v ), the voxel coordinate system is defined as follows:

[0015] (1) Let the depth of the tree be k, and the side length of the cube where the root node is located be L, then (2 k ) 3 The voxel space of individual pixels, take The smallest voxel is the origin (0, 0, 0);

[0016] (2) The coordinates of each node can be determined by its index in the octree; the index i can be represented as a triple (i x ,i y ,i z ), where i x ,i y ,i z are the components of i on the x, y, and z axes respectively;

[0017] (3) Given an index i, its corresponding coordinates (x, y, z) can be calculated using the following formula:

[0018]

[0019] Where: i x ,i y ,i z is the component extracted from index i and can be obtained by bitwise operation:

[0020] i x =imod2 k

[0021] i y=(i÷2 k )mod2 k (2)

[0022] i z =(i÷(2 k ) 2 )mod2 k .

[0023] In the above technical solution, in step 1, after the octree is established, the intrinsic topological relationship between the point clouds is restored by repairing the erroneous adjacency situation that is inconsistent with the adjacency relationship between the point clouds in the voxel space and the actual space. The specific implementation is as follows:

[0024] Let P be the original 3D point cloud set, T be the octree structure, V be the voxel set in T; for any two adjacent voxels v in V i , v j , their nearest neighbor distance is defined as:

[0025]

[0026] If d min (v i , v j )>∈, where ∈ is the minimum resolution of the point cloud, then v i and v j The two nodes are considered non-adjacent and the relationship is deleted from the adjacency table.

[0027] In the above technical solution, in step 2, the method of establishing the inter-voxel connection matrix includes the following steps:

[0028] Step 21: Fit a plane to the point cloud located in each voxel using the singular value decomposition (SVD) least squares method;

[0029] Step 22: Look up the voxel adjacency table, correct the normal vector direction and calculate the normal vector similarity of adjacent voxels, and establish the voxel connection matrix.

[0030] In the above technical solution, in step 22, the specific methods for correcting the normal vector orientation include the following two methods:

[0031] ① Use the photography station position to make corrections. The specific method is as follows:

[0032] Calculate and count the reverse direction of all points in a single voxel to its camera point - I i , perform a Gaussian weighted average based on the center of gravity to determine the main direction of the voxel point cloud, that is,

[0033]

[0034] Where: p i is the position of a point in a voxel point cloud, c is the center of gravity of the voxel, σ 2 is the variance of the Gaussian distribution; the main direction of the voxel point cloud is used to correct the plane normal vector direction n: N is the total number of points in the voxel point cloud, and i is a certain point;

[0035] n′=sign(n·I)·n (5)

[0036] Where sign(x) represents the sign function, which takes the value of +1 when x ≥ 0 and -1 otherwise; l is the main direction of the point cloud in the single voxel, and n′ is the direction of the normal vector after correction;

[0037] ② Use the feature points of the point cloud skeleton image for correction. The specific method is:

[0038] First, calculate the point cloud skeleton feature points, and then calculate the direction l from the voxel centroid to the nearest skeleton point v , correct the direction of the normal vector of the voxel square using the same method as in formula (5).

[0039] In the above technical solution, in step 22, the similarity between normal vectors is calculated as follows:

[0040]

[0041] Among them, n′ a , n′ b is the direction of the normal vector of the voxel plane of two adjacent voxels after correction, and σ is the kernel parameter of the Gaussian function.

[0042] In the above technical solution, in step three, the method for obtaining a polyhedron includes the following steps:

[0043] Step S31: clustering the voxel connection matrix, merging the point clouds of each cluster and refitting a new plane, correcting the direction of the plane normal vector, and performing an outward translation process on the plane to obtain a segmentation plane;

[0044] Step S32: Intersect the segmentation planes to obtain candidate points, lines, and surfaces, define energy functions and constraints, select a subset from the candidate surface set, and ensure that the selected surfaces form a manifold and watertight polygonal surface.

[0045] In the above technical solution, in step S31, outliers in a cluster are screened out by the RANSAC method and refitted into a new plane.

[0046] In the above technical solution, in step S32, the energy term and hard constraint of the objective function are defined as follows:

[0047] ①Energy term, including data fitting energy E f and point coverage energy E c :

[0048]

[0049] Where: |P| represents the number of points in the point cloud, x i It can be understood as a binary variable, representing the candidate face f i Is it selected, when x i 1 means selection, when x i 0 means not selected, support(f i ) is the confidence level considering the local neighborhood of the point cloud;

[0050]

[0051] Among them: area(M) represents the surface area of ​​the final model M; atea(f i ) represents the candidate face f i of surface area, and Indicates that the point is on the candidate surface f i The α-shape covers the area on the

[0052] ②Hard constraints are defined as follows:

[0053]

[0054] The meaning of formula (9) is that for each edge e i , the number of connected faces must be 2 (indicating two adjacent faces) or 0 (indicating a boundary face), x i Must be binary, that is, each x i It is 0 (indicating that the corresponding candidate face is not selected) or 1 (indicating that it is selected).

[0055] The present invention has the following advantages:

[0056] (1) A new polyhedron optimization representation method based on sparse point clouds is proposed, which can adapt to various irregular natural scenes with complex shapes and regular artificial objects, and provides an efficient geometric proxy model for UAV close-to-photography trajectory planning;

[0057] (2) The present invention does not need to calculate dense normal vectors, but adopts the idea of ​​point cloud segmentation and re-clustering, gradually extracts and fuses the surface features of the point cloud in the form of a plane, and makes the point cloud coplanar and connected through graph clustering, thereby simplifying the surface structure of the complex target and retaining its intrinsic geometric shape to the greatest extent; it solves the problem that the conventional normal vector clustering algorithm is prone to produce coplanar but unconnected point cloud segmentation; it solves the problem that the existing traditional method uses the method of calculating dense normal vectors from the initial point cloud, which is time-consuming, and the normal vector clustering algorithm is prone to produce coplanar but unconnected point cloud segmentation, resulting in topological ambiguity of the model (it is generally believed that two unconnected plane units are two separate instance individuals in space), thereby resulting in the inability to correctly restore its geometric shape from the topology using an outer polyhedron;

[0058] (3) The three-dimensional point cloud obtained by sparse reconstruction is optimized and calculated into a rough model represented by a polyhedron (simplifying the surface structure of the complex target and retaining its intrinsic geometric shape to the greatest extent), and then the trajectory and attitude planning are performed for all the faces of the polyhedron one by one, so as to drive the UAV to perform close-up photography to obtain high-definition images, so that the close-up photography trajectory planning method can adapt to various scenes, improve versatility, and avoid collision between the track point and the target scene; solve the problems that the existing close-up photography trajectory planning method is difficult to adapt to various scenes, has poor versatility (cannot be widely used in various complex scenes, such as ancient buildings or natural landforms with complex shapes), cannot avoid collision between the track point and the target scene, requires a lot of manual intervention, cannot achieve coplanar and connected segmentation of point clouds, and consumes a lot of time;

[0059] (4) The present invention corrects and outsources the model in units of faces, which can effectively avoid collisions between track points and target scenes;

[0060] (5) The method of the present invention does not require separate preprocessing for specific targets. All scenarios share a set of processing paradigms, i.e., first block division and then clustering, which makes the present invention highly versatile and can be widely used in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is the overall technical flow chart of the present invention;

[0062] Figure 2 is a schematic diagram of a voxel coordinate system of the present invention;

[0063] Figure 3 It is a schematic diagram of the data processing process of the present invention;

[0064] Figure 4 A schematic diagram of a sparse point cloud model of a teaching experiment building in an embodiment of the present invention;

[0065] Figure 5 This is an octree block result diagram of the teaching experiment building in an embodiment of the present invention;

[0066] Figure 6 It is a thermal diagram of the contact matrix in an embodiment of the present invention;

[0067] Figure 7 This is a Kamada-Kawai layout result diagram of Louvain clustering in an embodiment of the present invention;

[0068] Figure 8 This is a polyhedron model diagram of a teaching laboratory building optimized and calculated according to an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following is a detailed description of the implementation of the present invention in conjunction with the accompanying drawings, but they do not constitute a limitation of the present invention and are only given as examples. At the same time, the advantages of the present invention are made clearer and easier to understand through the description.

[0070] Referring to the attached drawings, it can be seen that: a sparse point cloud polyhedron optimization representation method for close-to-photographic trajectory planning, the method uses the sparse point cloud data of the target scene as input data, first uses an octree to voxelize and divide the point cloud data into blocks, and fits the plane according to the voxels and establishes its adjacency table; then uses the camera station position of the rough flight image or the point cloud skeleton feature points to correct the plane normal vector orientation, and establishes a voxel connection matrix based on the adjacency relationship between voxels and the similarity of adjacent voxel normal vectors; finally, the connection matrix is ​​graph clustered, and all points belonging to the same cluster are used to fit a new plane to form a segmentation plane. After the segmentation plane is outsourced and corrected, it can be combined into a polyhedron through binary integer linear programming (i.e., 0-1 integer linear programming). The overall technical process of the present invention is as follows. Figure 1 As shown, the core process includes the following steps:

[0071] Step 1: Perform octree voxelization and block division on the sparse point cloud of the three-dimensional scene obtained by rough flight, and use the voxel coordinates calculated by the tree's child node index to establish the adjacency table between each voxel; traverse the nearest neighbor distance between adjacent voxel points, and modify the adjacency relationship whose nearest neighbor distance is less than the minimum resolution of the point cloud. Set the voxel point cloud with a number of merged points less than a certain threshold, and update the adjacency table;

[0072] Step 2: Use the singular value decomposition (SVD) least squares method to fit the plane for each voxelized sub-block point cloud;

[0073] Step 3, search the adjacency table, correct and calculate the normal vector similarity of adjacent voxels, and establish the voxel connection matrix;

[0074] Step 4: perform graph clustering on the contact matrix, merge the point clouds of each cluster and refit a new plane, correct the plane normal vector according to the method in step 3, and perform outsourcing translation processing on the plane to obtain the segmentation plane;

[0075] Step 5, intersect the segmentation planes to obtain candidate points, lines, and surfaces, define energy functions and constraints, select a subset from the candidate surface set, and ensure that the selected surfaces form a manifold and watertight polygonal surface.

[0076] In step 1, the voxel resolution for dividing the scene sparse point cloud into blocks can be set according to the size of the local geometric structure of the actual target. For example, if the floor height of a building is 3m, it can be approximately considered that the 3m resolution can express the local geometric features of the surface point cloud of a building. Voxels share one vertex, that is, 26 adjacency, and the voxel coordinate system (O) is established with the smallest voxel as the unit. v , X v , Y v , Z v )(like Figure 2 As shown), the voxel coordinate system is defined as follows:

[0077] (1) Let the depth of the tree be k, and the side length of the cube where the root node is located be L, then (2 k ) 3 The voxel space of individual pixels, take The smallest voxel is the origin (0, 0, 0);

[0078] (2) The coordinates of each node can be determined by its index in the octree; the index i can be represented as a triple (i x ,i y ,i z ), where i x ,i y ,i z are the components of i on the x, y, and z axes respectively;

[0079] (3) Given an index i, its corresponding coordinates (x, y, z) can be calculated using the following formula:

[0080]

[0081] where i x ,i y ,i z is the component extracted from index i, which can be obtained by bitwise operation;

[0082] i x =imod2 k

[0083] i y =(i÷2 k )mod2 k (2)

[0084] i z =(i÷(2 k ) 2 )mod2 k

[0085] The adjacency relationship is calculated based on the voxel coordinates. The update of the adjacency relationship is described as follows:

[0086] Let P be the original 3D point cloud set, T be the octree structure, and V be the voxel set in T. For any two adjacent voxels v in V i , v j , their nearest neighbor distance is defined as:

[0087]

[0088] If d min (v i , v j )>∈, where ∈ is the minimum resolution of the point cloud, then v i and v j The points are considered non-adjacent and the relationship is deleted from the adjacency table. This step will be used to repair the erroneous adjacency judgment that the point clouds are adjacent in the voxel space but not in the actual space, and restore the intrinsic topological relationship between the point clouds;

[0089] Assume n merge is the threshold of the number of merged points. If |vx|<n merge , then consider v i The voxels closest to their centroids are merged and the adjacency table is updated. The above merging process is iterative until a certain stopping condition is met; for example, the number of points of all voxels is greater than or equal to n. merge When , the adjacency table no longer changes; the present invention believes that too fragmented point clouds cannot express the local geometric features of the target scene, so appropriate merging is required to discard overly detailed geometric structures.

[0090] In step 2, the least squares plane fitting is performed on the block point clouds obtained in step 1. The specific process is as follows:

[0091] In three-dimensional space, a plane can generally be expressed by the following equation:

[0092] Ax+By+Cz+D=0 (10)

[0093] Where A, B, C are plane normal vectors, D is the plane offset, and the objective function can be expressed as:

[0094]

[0095] Find the first-order partial derivative and get the error equation:

[0096]

[0097] make and Then the normal equation is in the form of:

[0098] M T MA=M T D=Db (13)

[0099] Where A is a vector In short, written in matrix form:

[0100]

[0101] Perform singular value decomposition on the M matrix (error equation coefficient matrix) obtained by least squares, and get A = U∑V T , where U and V are orthogonal matrices, ∑ is a diagonal matrix;

[0102] When the equation is overdetermined, the least squares solution can be obtained through SVD decomposition, which is expressed as:

[0103] x=V∑+U T D (15)

[0104] Among them, ∑ + is the pseudo-inverse of ∑, when D=0, x represents the homogeneous solution of the equation.

[0105] In step 3, for the point cloud obtained from the sparse reconstruction method, the plane normal vector is corrected using the image exterior orientation elements as a reference, and the reverse direction of the photographic direction of all points in a single voxel is counted. i , perform a Gaussian weighted average based on the center of gravity to determine the main direction of the voxel point cloud, that is,

[0106]

[0107] where p i is the position of the point, c is the center of gravity of the voxel, σ 2 is the variance of the Gaussian distribution; the main direction of the voxel point cloud is used to correct the plane normal vector direction n:

[0108] n'=sign(n·l)·n (5)

[0109] Where sign(x) represents the sign function, which takes the value of +1 when x ≥ 0 and -1 otherwise;

[0110] In addition, for regular and closed buildings, the feature points of the point cloud skeleton map are used to correct the normal vector of the voxel plane. The specific method is as follows:

[0111] (1) After downsampling the point cloud, local principal component analysis (PCA) and Delaunay triangulation are performed to construct a local neighborhood T;

[0112] (2) For each point p0 in the point cloud S, calculate its surface normal And the mean curvature H(p0); the mean curvature can be calculated from the principal curvatures k1, k2:

[0113] (3) Define the curvature normal flow operator L for point cloud shrinkage; then, use the implicit integration scheme to iteratively update the point cloud position until the volume approaches zero, thus achieving point cloud shrinkage: Perform spherical sampling on the shrunk point cloud, establish a mapping M between the sampling point set S and the original point cloud, and iteratively mark all shrunk points;

[0114] (4) Define the cost function Γ(e ij ), iteratively collapse the edges in the skeletal mesh until there are no triangles left:

[0115] Γ(e ij )=γ a Γ a (e ij )+γ b Γ b (e ij ) (17)

[0116] (5) Connect the broken skeletons and finally adjust the positions of the skeleton points to center them on the centroid of the local point cloud area:

[0117]

[0118] (6) Calculate the direction l from the voxel center of gravity to the nearest skeleton point v , correct the direction of the normal vector of the voxel square using the same method as in formula (5).

[0119] In step 3, the similarity between voxels is calculated as follows:

[0120]

[0121] Among them, n′ a , n′ b is the direction of the normal vector of the voxel plane of two adjacent voxels after correction, σ is the kernel parameter of the Gaussian function;

[0122] In the above formula, n′ a , n′ b are the corrected normal vectors of the adjacent voxel pairs queried from the adjacency table, query the adjacency table established in step 1, and calculate the similarity to obtain the connection matrix between the voxels:

[0123]

[0124] In step 4, the graph clustering method used gives priority to modularity optimization algorithms such as Louvain; it is a hierarchical clustering method that discovers the community structure in the graph by continuously optimizing the modularity. Its main advantages are that it can handle large-scale graphs, can calculate large-scale sparse matrices, does not rely on any artificial parameters, and can discover communities of different sizes; after clustering, the clustering result is first corrected by the Random Sample Consensus (RANSAC) algorithm, and then the plane is refitted to the clustered point cloud according to the method described in steps 2 and 3, and the normal vector is corrected; all points on the positive side of the normal vector belonging to the cluster are counted and calculated, and the distance d from each point p = (x, y, z) to the plane is calculated. p :

[0125]

[0126] Calculate the translation vector t along the corrected normal vector n′:

[0127]

[0128] Get the new plane equation constant term and the new plane equation:

[0129] d'=d-at a -bt b -ct c

[0130] a'x+b'y+c'z+d'=0 (22)

[0131] In step 5, the energy terms include the degree of fit and the degree of coverage, which are defined as follows:

[0132] (1) Data fitting energy E f : Evaluate the quality of the fit between the selected face and the point cloud, taking into account the confidence level;

[0133]

[0134] Where |P| represents the number of points in the point cloud, x i It can be understood as a binary variable, representing the candidate face f i Is it selected (x iThe value 1 indicates selection, and the value 0 indicates unselection), support(f i ) Consider the confidence of the local neighborhood of the point cloud;

[0135] (2) Point coverage energy E c : Process missing data caused by occlusion to ensure that the unsupported area of ​​the final model (i.e. the area not covered by the point) is as small as possible; at the same time, in order to measure the coverage area of ​​a surface, all points are first projected onto the candidate surface f i And construct the mesh (α-shape) in the form of a triangulated network;

[0136]

[0137] Among them, area(M) represents the surface area of ​​the final model M, area(f i ) represents the candidate face f i of surface area, and Indicates that the point is on the surface f i The α-shape covers the area on the

[0138] By minimizing the weighted sum of all energy terms, the optimal set of faces is selected under the hard constraint of ensuring that the final model is manifold and has no watertightness; the final polyhedron optimization objective function is:

[0139]

[0140] To ensure that the final model is manifold and closed, add the following hard constraints:

[0141]

[0142] This means that for each edge e i , the number of connected faces must be 2 (indicating two adjacent faces) or 0 (indicating a boundary face), x i Must be binary, that is, each x i is 0 (indicating that the corresponding candidate face is not selected) or 1 (indicating that it is selected); thus, the above polyhedron optimization problem is converted into a 0-1 integer linear programming problem, which can be solved by linear solvers such as Gurobi and SCIP.

[0143] Embodiment: The present invention is now described in detail by taking the present invention being tried in a university's information department and performing polyhedron optimization representation of its teaching laboratory building based on sparse point clouds as an embodiment. The present invention can also be used as a guide for applying the present invention to other irregular natural scenes and regular artificial objects with complex shapes and performing polyhedron optimization representation of natural scenes or artificial objects based on sparse point clouds.

[0144] In this example, a teaching experiment building of the Department of Information Science of a certain university was selected as the experimental target. A DJI Phantom 4 RTK drone was used to roughly fly the target at an altitude of 120m. 262 images were taken with a resolution of about 3.29cm. The image was sparsely reconstructed to obtain a rough point cloud of the target object as the input data of this example (e.g. Figure 4 The main process includes the following steps:

[0145] Step 1: Preprocess the sparse point cloud and use statistical filtering and other methods to filter out outliers. The voxel resolution is set to 10m, which is the height of a three-story building. The point cloud is voxelized by establishing an octree. The results are as follows: Figure 5 As shown in the figure, using octree for block division can easily find the adjacency relationship between voxels according to the index, and then establish an adjacency table and delete the adjacency relationship between spatially discontinuous voxels. Set the threshold n for point cloud merging merge The number of voxel point cloud fragments is 300, and the voxel point cloud fragments are merged and new adjacency relationships are established to obtain 56 sub-block point cloud data;

[0146] Step 2: Perform SVD plane fitting on the 56 point clouds obtained in step 1 to obtain 56 plane equations expressed in general form:

[0147] A i x+B i y+C i z+D i =0|i∈{0,1,2,...,55}

[0148] Step 3: Use the mean curvature flow algorithm to calculate the point cloud skeleton map, obtain the skeleton feature points, calculate the direction of each voxel's center of gravity to the nearest skeleton feature point, correct the directions of the 56 plane normal vectors to ensure that their directions are toward the outside of the teaching laboratory building; set the Gaussian kernel parameter size for similarity calculation to 0.15, calculate the normal vector similarity according to formula (19) and the voxel adjacency table, and construct the following: Figure 6 The connection matrix shown in the figure not only reflects the adjacency relationship between voxels, but also can intuitively reflect the topological relationship of the object surface;

[0149] Step 4: Use Louvain modularity clustering algorithm to cluster the contact matrix and obtain 8 clusters, such as Figure 7 As shown. For each cluster point cloud, RANSAC is used for screening, with the distance threshold set to 1 and the upper limit of the number of iterations set to 1000. The voxel point clouds with similar normal vectors in the cluster but more than 50% outliers in space are screened out separately and divided into a new cluster, and finally 9 clusters are obtained;

[0150] For all points in each cluster, the SVD plane fitting method is used to fit a new plane, that is, the split plane:

[0151] A j x+B j y+C j z+D j =0|j∈{0,1,2,...,8}

[0152] According to the method in step 3, the normal vector direction of each segmentation plane is corrected and translated along the normal vector direction. The translation distance is the maximum distance d from the outer point cloud of each cluster to the plane. p max; get the corrected splitting plane equation:

[0153] A′ j x+B′ j y+C' j z+D′ j =0|j∈{0,1,2,...,8}

[0154] Step 5: Intersect the obtained segmentation planes with each other, and use the coordinate range of the original point cloud to pre-screen all planes that can form a polyhedron and their intersection points and intersection lines as candidates. Combined with the original point cloud information, calculate the two energy terms corresponding to each candidate face, and set the two weights of the optimization objective function to:

[0155] λ f =0.45,λ c =0.55

[0156] After adding the constraints shown in formula (9) to the objective function, the Gurobi linear solver is used to solve the problem. The obtained optimal plane combination and its corner point data are saved as an obj format file. The rendering result of the vector model of the teaching laboratory building is as follows: Figure 8 As shown, the polyhedron simplifies the surface structure of the complex target and retains its intrinsic geometric shape to the greatest extent, avoiding the collision between the track point and the target scene.

[0157] The present invention mainly solves the problem that traditional close-up photography trajectory planning methods are difficult to adapt to various complex scenes, and proposes a method for optimizing the representation of sparse point clouds as polyhedrons. This method can simplify the surface structure of complex targets and retain their intrinsic geometric shapes to the greatest extent, and on this basis efficiently plan a photography network that can be used for close-up photography to collect fine data. It solves the ambiguity problem of segmentation of coplanar but unconnected point clouds caused by ordinary normal vector clustering. The extraction of point cloud surface features is completed based on plane fitting through the idea of ​​first dividing blocks and then clustering, and the correction of the point cloud's outer vector model from the point cloud skeleton map or the external orientation elements of the photography is the key technology of the polyhedron optimization representation described in the present invention.

[0158] The specific embodiments described in the present invention are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the attached claims.

[0159] Other parts not described belong to the prior art.

Claims

1. A polyhedron optimization representation method for sparse point clouds close to photogrammetry planning, characterized by: The point cloud close to the rough photography is divided into blocks, and the point cloud is segmented by fitting planes and clustering the block point clouds. Finally, the geometric structure information of the point cloud is optimized in the form of an outer polyhedron. The specific method includes the following steps: taking the sparse point cloud data of the target scene as input data, Step 1: First, use octree to voxelize the point cloud data and express it in blocks, and establish an adjacency table for each voxel fitting plane; Step 2: Then use the camera position of the rough flight image or the feature points of the point cloud skeleton to correct the plane normal vector orientation, and establish the voxel connection matrix based on the adjacency relationship between voxels and the similarity of the normal vectors of adjacent voxels; The method for establishing a voxel-to-voxel connection matrix comprises the following steps: Step 21: Fit a plane to the point cloud located within each voxel using the singular value decomposition least squares method; Step 22: Look up the voxel adjacency table, correct the normal vector direction and calculate the normal vector similarity of adjacent voxels to establish a voxel connection matrix; Step 3: Finally, the connection matrix is ​​subjected to graph clustering, and a new plane is fitted using all points belonging to the same cluster to form a segmentation plane. After the segmentation plane is corrected by outsourcing, it is combined into a polyhedron through binary integer linear programming. The method for obtaining the polyhedron includes the following steps: Step S31: clustering the voxel connection matrix, merging the point clouds of each cluster and refitting a new plane, correcting the direction of the plane normal vector, and performing an outward translation process on the plane to obtain a segmentation plane; Step S32: Intersect the segmentation planes to obtain candidate points, lines, and surfaces, define energy functions and constraints, select a subset from the candidate surface set, and ensure that the selected surfaces form a manifold and watertight polygonal surface.

2. The method for optimizing polyhedron representation of sparse point clouds for close-to-photographic planning according to claim 1, characterized in that: In step 1, the sparse point cloud of the three-dimensional scene obtained by the rough flight of the UAV is divided into blocks by octree voxelization, and the voxel coordinates calculated by the octree child node index are used to establish a voxel adjacency table for storing the adjacency relationship between voxels; Traverse the nearest neighbor distances between adjacent voxel points two by two, and modify the adjacency relationship whose nearest neighbor distance is less than the minimum resolution of the point cloud; Set the voxel point cloud with a point count less than the set threshold to merge and update the adjacency table.

3. The method for optimizing polyhedron representation of sparse point clouds for close-to-photographic planning according to claim 2, characterized in that: In step 1, after voxelization and block division by octree, a voxel coordinate system (O v , X v , Y v , Z v ), the voxel coordinate system is defined as follows: (1) Let the depth of the tree be k, and the side length of the cube where the root node is located be L, then (2 k ) 3 The voxel space of individual pixels, take The smallest voxel is the origin (0, 0, 0); (2) The coordinates of each node are determined by its index in the octree; index i is represented as a triple (i x ,i y ,i z ), where i x ,i y ,i z are the components of i on the x, y, and z axes respectively; (3) Given an index i, its corresponding coordinates (x, y, z) are calculated using the following formula: Where: i x ,i y ,i z is the component extracted from index i, obtained by bitwise operation: i x =imod2 k i y =(i÷2 k )mod2 k (2) i z =(i÷(2 k ) 2 )mod2 k 。 4. The method for optimizing polyhedron representation of sparse point clouds for close-to-photographic planning according to claim 3, characterized in that: In step 1, after the octree is established, the intrinsic topological relationship between point clouds is restored by repairing the erroneous adjacency between the point cloud in voxel space and the actual space. The specific implementation is as follows: let P be the original 3D point cloud set, T be the octree structure, V be the voxel set in T; for any two adjacent voxels v in V i , v j , their nearest neighbor distance is defined as: If d min (v i , v j )>∈, where ∈ is the minimum resolution of the point cloud, then v i and v j The two nodes are considered non-adjacent and the relationship is deleted from the adjacency table.

5. The method for optimizing polyhedron representation of sparse point clouds for close-to-photographic planning according to claim 4, characterized in that: In step 22, the specific methods for correcting the normal vector orientation include the following two methods: ① Use the photography station position to make corrections. The specific method is as follows: Calculate and count the reverse direction of all points in a single voxel to its camera point - l i , perform a Gaussian weighted average based on the center of gravity to determine the main direction of the voxel point cloud, that is, Where: p i is the position of a point in a voxel point cloud, c is the center of gravity of the voxel, σ 2 is the variance of the Gaussian distribution; the main direction of the voxel point cloud is used to correct the plane normal vector direction n: N is the total number of points in the voxel point cloud, and i is a certain point; n′=sign(n·l)·n (5); Where sign(x) represents the sign function, which takes the value of +1 when x ≥ 0 and -1 otherwise; I is the main direction of the point cloud in the single voxel, and n′ is the direction of the normal vector after correction; ② Use the feature points of the point cloud skeleton image for correction. The specific method is: First, calculate the point cloud skeleton feature points, and then calculate the direction l from the voxel centroid to the nearest skeleton point v , correct the direction of the normal vector of the voxel square using the same method as in formula (5).

6. The method for optimizing polyhedron representation of sparse point clouds for close-to-photographic planning according to claim 5, characterized in that: In step 22, the similarity between normal vectors is calculated as follows: Among them, n′ a , n′ b is the direction of the normal vector of the voxel plane of two adjacent voxels after correction, and σ is the kernel parameter of the Gaussian function.

7. The method for optimizing polyhedron representation of sparse point clouds for close-to-photographic planning according to claim 6, characterized in that: In step S31, outliers in a cluster are screened out by the RANSAC method and refitted into a new plane; In step S32, the energy term and hard constraints of the objective function are defined as follows: ①Energy term, including data fitting energy E f and point coverage energy E c : Where: |P| represents the number of points in the point cloud, x i Understood as a binary variable, representing the candidate face f i Whether it is selected, when x i 1 means selection, when x i 0 means not selected, support(f i ) is the confidence level considering the local neighborhood of the point cloud; Among them: area(M) represents the surface area of ​​the final model M; area(f i ) represents the candidate face f i of surface area, and Indicates that a point on a voxel point cloud is on the candidate face f i The α-shape covers the area on the ②Hard constraints are defined as follows: Make Among them, for each edge e i , the number of connected faces x j is 2 or 0; x i is binary, that is, each x i Is 0 or 1.

Images