A UAV LiDAR flight zone adjustment method integrating semantic features

Abstract

The present invention relates to the field of UAV laser radar data processing, and discloses a UAV laser radar flight zone adjustment method integrating semantic features, which sequentially uses multi-level non-rigid registration oriented to semantic objects and multi-flight point cloud flight zone adjustment taking into account non-rigid deformation. By diagonalizing the coefficient matrix of the overall error equation of different flight zones of different flights, the complexity of solving unknown parameters using nonlinear optimization algorithms is effectively reduced, thereby improving the efficiency of solving unknown parameters.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) laser radar data processing, and in particular to an unmanned aerial vehicle (UAV) laser radar flight track adjustment method integrating semantic features. Background Art

[0002] Adjustment refers to the difference between the observed value and the true value caused by the accuracy of the instrument, human factors and external conditions. In order to eliminate the difference, it is generally required that the number of observations is greater than the number of observations needed to solve the unknown quantity. In redundant observations, there will definitely be contradictions between the observation results. Adjustment is the process of eliminating these contradictions to obtain the most reliable results of the observation and evaluate the accuracy of the measurement results.

[0003] A flight strip refers to a sequence of overlapping images obtained by aerial photography along a certain direction. It now also refers to a sequence of overlapping point cloud information data obtained by collecting 3D point cloud information data including lidar along a certain direction. Flight strip adjustment refers to an adjustment method based on the flight strip model as the basic unit in order to eliminate errors between adjacent flight strips.

[0004] UAV laser radar (LiDAR) is an active measurement method that can quickly obtain target three-dimensional point cloud information data. Like vehicle-mounted laser radar, UAV laser radar can measure the geometric information and laser echo intensity information of the road surface and road environment in real time during the platform movement, as well as the position, attitude, speed and other trajectory information of the carrier obtained by the GNSS / IMU combined system (POS). Compared with vehicle-mounted laser radar, UAV laser radar has the advantages of all-weather, high precision and low cost. Therefore, in recent years, UAV laser radar has been widely used in forestry, cities, transportation, water conservancy and other fields.

[0005] However, the installation process of the UAV laser radar cannot ensure that the installation angle accuracy of the laser radar meets the requirements and is completely consistent; and because of the accuracy problem of the POS system itself, non-rigid deformation of the flight strip will be formed, resulting in distortion on a flight strip. That is, in a GNSS weak positioning environment, the state error of the mobile measurement carrier will accumulate over time, resulting in platform position and attitude (referred to as "pose") errors, the accuracy levels in different areas are inconsistent and non-rigid transformations occur over time, and the point accuracy will decrease. In the process of generating point clouds, there will be varying degrees of accuracy loss, so the scanned image needs to be processed by an adjustment algorithm to reduce the accuracy loss.

[0006] In order to achieve complete overlap of point clouds on a flight strip and reduce the acquisition error caused by the POS system, the traditional approach is to use a flight strip as a rigid registration unit for registration. However, such an operation requires a large amount of data to be registered, and it is difficult to ensure the accuracy of the registration due to the differences in the settings of the actual same-name points. Summary of the invention

[0007] The present invention aims to provide a UAV laser radar flight zone adjustment method integrating semantic features, so as to achieve high-precision registration of the flight zone under the premise of limited data calculation amount and reduce the non-rigid deformation caused by the POS system.

[0008] In order to achieve the above object, the present invention adopts the following technical scheme:

[0009] A method for adjusting the flight path of UAV laser radar by integrating semantic features includes the following steps:

[0010] Step 1: Multi-level non-rigid registration for semantic objects: The flight path point cloud that has been pre-processed is divided into multiple segmented point clouds that can be regarded as rigid registration units; by separating the segmented point clouds into object point clouds of different categories, the matching relationship between objects is established by using the similarity of semantic features, and the generalized corresponding points of the same name are established between semantic objects of the same name by using the similarity of point primitive features;

[0011] Step 2: Multi-flight point cloud flight strip adjustment taking into account non-rigid deformation: establish the pose error observation equation for each flight and each flight strip point cloud in the measured area, and use the cubic spline function to approximate the trajectory pose error in time state, and establish a UAV lidar point cloud time-varying model that conforms to the characteristics of multiple flights;

[0012] Step three, by diagonalizing the coefficient matrix of the overall error equation of different flight strips for different sorties, the final flight strip adjustment is calculated through the UAV lidar point cloud time-varying model.

[0013] The principles and advantages of this solution are:

[0014] In the previous scheme, in order to make the point clouds on different flight strips completely overlap and reduce the collection error caused by the POS system, one flight strip was used as a rigid registration unit for registration, resulting in a large amount of registration data, and it was difficult to ensure the accuracy of the registration due to the difference in the setting of the actual same-name points. This scheme does not use one flight strip as a rigid registration unit, but adopts a semantic registration strategy from the object level to the primitive level. Through multiple flights of drones to collect point cloud data, multiple objects with the same name are defined semantically, without the need for actual same-name points. This not only solves the problem of data calculation, but also effectively guarantees accuracy.

[0015] In practical applications, this solution is not only applicable to complex and changeable mountainous scenarios, but also to other scenarios such as strip-shaped power corridors. This solution has strong versatility and portability.

[0016] In this field, when it comes to improving the accuracy of three-dimensional point clouds, people often unilaterally emphasize the accuracy of the drone laser radar itself, or unilaterally emphasize the accuracy of the POS system itself. In short, they over-consider the impact of the equipment itself on accuracy.

[0017] At present, there is too much pursuit of high precision brought by the equipment itself, mainly in the processing of point cloud data, especially in the processing of flight strip adjustment. This field usually stays in a single data-driven or model-driven method. In order to further improve the accuracy, the existing data processing method needs to set too many parameters to be estimated and too many calculation processes. Through complex and cumbersome calculation content, more accurate data processing results can be obtained. However, this leads to a very large amount of data calculation and low computing efficiency. There is no sufficient room for improvement.

[0018] After research, it was found that the previous solutions, whether focusing on the accuracy of the equipment itself or in the processing of complex data, ignored the restoration of the real scene, especially the actual non-rigid registration effect on site, and the calculation data was separated from the real scene, resulting in poor accuracy and robustness, and a large difference between the registration results and the real scene. This solution abandons such inertial thinking, does not blindly pursue unilateral improvement of the accuracy of a certain link, and fully considers the complexity of the on-site environment. In the face of the point cloud data that has been collected, the first thing is to choose an object-oriented approach, perform multi-level non-rigid registration on non-rigid object data that may produce large errors, establish generalized corresponding points of the same name, and use a relatively small amount of calculation to complete the classification of the flight strip point cloud data, and complete the data processing on the basis of restoring the real scene as much as possible; then, the error observation equation is established through the flight strip point cloud collected by multiple flights, and the time state approximation of the trajectory posture error of the flight strip point cloud data collected by multiple flights is used to use the cubic spline function, and the UAV laser radar point cloud time-varying model that meets the characteristics of multiple flights is established. This scheme is verified by the flight strip point cloud data collected by multiple UAV lidars. Instead of taking a whole flight strip as a rigid registration unit, each flight strip is divided into segmented point clouds, and each segmented point cloud set is divided into different levels of objects. Finally, the final flight strip adjustment is calculated using the UAV lidar point cloud time-varying model to correct the position and posture of the POS system, thereby minimizing the deviation in the correction process and making the correction result more accurate.

[0019] This solution effectively solves the problem of being unable to obtain high-quality point cloud three-dimensional coordinates under the premise of limited data calculation in the current complex mountainous environment and strip power corridor environment, solves the problem of poor non-rigid registration effect caused by the existence of various non-rigid ground objects in urban scenes, solves the problem of poor accuracy of UAV lidar point cloud acquisition due to weak GNSS positioning in urban scenes, and solves the current problem of poor reality restoration of urban scenes.

[0020] Preferably, as an improvement, in step one, the flight strip point cloud that has undergone point cloud preprocessing is obtained by scanning multiple unmanned aerial vehicle lidars to form a multi-strip initial point cloud and then processed according to a preprocessing strategy.

[0021] Effect: The preprocessing strategy can effectively screen the initial point clouds of multiple flight strips to obtain the flight strip point cloud, which can minimize the erroneous data collected in the initial point cloud data due to the acquisition equipment itself.

[0022] Preferably, as an improvement, in step one, the object point cloud includes a linear object point cloud, a planar object point cloud and a columnar object point cloud.

[0023] Effect: This solution creatively divides the object point cloud into three different types, which can identify and distinguish most of the point cloud data. It can select the types of data that are more important for the true restoration of the environment through different types of object point cloud data for key processing. It can improve the accuracy of data processing as much as possible under the premise of limited data computing power and improve the true restoration of the scene.

[0024] Preferably, in step one, the matching relationship between objects is established by utilizing the similarity of semantic features, namely, associating and matching single objects in each category set of linear object point cloud, planar object point cloud and columnar object point cloud with single objects in the category set and other category sets.

[0025] Preferably, as an improvement, in step one, establishing a matching relationship between objects using the similarity of semantic features includes sequentially performing point cloud monomer separation, feature descriptor calculation, and feature similarity matching.

[0026] Effect: Through the above steps, the matching relationship between objects can be quickly established.

[0027] Preferably, as an improvement, in step one, establishing generalized corresponding points of the same name using point primitive feature similarity between semantic objects of the same name includes sequentially performing point primitive matching, constructing a combined distance error equation, and solving conversion parameters.

[0028] Effect: This solution abandons the current common method of registering actual homonymous points and eliminates the need to set up actual homonymous points. Compared with the existing technology, this solution requires very little data and the calculation steps are relatively simpler. However, due to the establishment of generalized homonymous corresponding points, it is possible to achieve accurate semantic registration of point cloud data at each object level based on the classification of object point clouds.

[0029] Preferably, as an improvement, the preprocessing strategy is to perform ground filtering, point cloud spatiotemporal division, and conjugate point cloud detection in sequence.

[0030] Effect: The above preprocessing strategy can filter out the deviation data in the collected original data as much as possible without affecting the accuracy of subsequent calculations.

[0031] Preferably, as an improvement, the point cloud is divided into time and space in accordance with a heading rule, a time rule, and a speed rule in sequence.

[0032] Effect: The above method can be used to divide the point cloud into time and space, which can quickly extract valid data and filter out deviation data, providing a prerequisite for the accuracy of subsequent calculations.

[0033] Preferably, as an improvement, before step three, an error equation is constructed, and in step three, the coefficient equation is diagonalized and then solved by the LM algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of Embodiment 1 of the present invention.

[0035] Figure 2 This is a diagram showing the effect of dividing various ground object point clouds in the segmented point cloud in the first embodiment of the present invention.

[0036] Figure 3 This is the main category point cloud binary image of embodiment 1.

[0037] Figure 4 This is a schematic diagram of a connected domain labeling method for a binary graph according to Embodiment 1.

[0038] Figure 5 This is a diagram showing the effect of separating various types of objects in the first embodiment of the present invention.

[0039] Figure 6 This is a schematic diagram for comparing objects with different names but with different geometric dimension features.

[0040] Figure 7 Generate a schematic diagram for reference ground height.

[0041] Figure 8 Construct a schematic diagram for the semantic object solid voxels.

[0042] Fig. 9 The figure is a diagram of the pixel area feature histogram construction process. The left frame is the construction process diagram, and the right frame is the formed histogram.

[0043] Fig.10 Schematic diagram of the search space for matching various objects in the first embodiment of the present invention.

[0044] Fig.11 This is a comparison diagram of the separation effect of separating the segmented point cloud monomers into object point clouds in Example 1 of the present invention.

[0045] Fig.12 Schematic diagram of primitive-level matching of segmented point clouds in Embodiment 1 of the present invention.

[0046] Fig.13 Schematic diagram of corresponding points with the same name of objects of different types in the first embodiment of the present invention.

[0047] Fig.14 It is a schematic diagram of the non-rigid registration result in the first embodiment of the present invention.

[0048] Fig.15 The figure is a schematic diagram of the segmentation of the flight strip in the first embodiment of the present invention.

[0049] Fig.16 This is the error distribution diagram before and after the flight strip adjustment in Example 1 of the present invention.

[0050] Fig.17 Schematic diagram of the cubic spline function in Example 1. DETAILED DESCRIPTION

[0051] The following is further described in detail through specific implementation methods:

[0052] Embodiment 1

[0053] The embodiment is basically as shown in the attached Figure 1 As shown in the figure: the UAV laser radar flight zone adjustment method integrating semantic features in this embodiment includes the following contents:

[0054] S1, multi-level non-rigid registration for semantic objects:

[0055] S11 uses multiple UAV laser radar scans to form multiple flight strips of initial point clouds, and obtains flight strip point clouds after processing according to the preprocessing strategy; the flight strip point clouds are filtered out of ground points to obtain non-ground point clouds and ground point clouds; the flight strip point clouds that have been preprocessed, that is, the non-ground point clouds, are divided into multiple segmented point clouds that can be regarded as rigid body registration units;

[0056] Among them, the initial point cloud data of multiple flight strips includes the collected point cloud data and the original POS data.

[0057] The preprocessing strategy is to sequentially perform ground filtering, point cloud time-space division and conjugate point cloud detection. The point cloud time-space division is to sequentially classify the point cloud data and trajectory state data according to heading rules, time rules and speed rules.

[0058] Specifically, in order to achieve effective matching of regional data, the point cloud is pre-spatiotemporally divided along the trajectory into several short-time and spatial domain "scanning blocks". Within the local block, the non-rigid transformation model can be approximated by a low-order polynomial transformation model.

[0059] The spatiotemporal division of point cloud is divided according to the heading rule, time rule and speed rule. From the starting moment, the heading rule, time rule and speed rule are compared in turn. Starting from the first rule, when different rules are met, the point cloud should be divided into new blocks, and when the conditions of the rules are consistent, they are divided into the same block. The time rule is to divide the block if it exceeds the given time; the heading rule is to divide the block if it exceeds the given steering angle, which is used to handle the sharp turn moment; the speed rule is that if the speed is less than the given minimum speed, it is considered that the platform is close to stationary and needs to be divided into blocks. The length of the point cloud block should not be too small, otherwise there will be few matching feature objects inside the block; the length of the point cloud block should not be too large, otherwise the registration accuracy will be affected due to non-rigid transformation. Through the above method to divide the point cloud into space and time, it is possible to quickly extract effective data, filter out deviation data, and provide a prerequisite for the accuracy of subsequent calculations. The above preprocessing strategy can filter out deviation data in the collected original data as much as possible without affecting the accuracy of subsequent calculations. Through the preprocessing strategy, the flight strip point cloud can be obtained after effective screening from the initial point cloud of multiple flight strips, which can minimize the erroneous data collected in the initial point cloud data due to the collection device itself.

[0060] S12 divides the segmented point cloud into object point clouds of different categories;

[0061] like Figure 2 As shown, the obtained non-ground point cloud is divided into different types of object point clouds. The object point clouds in this embodiment include linear object point clouds, planar object point clouds and columnar object point clouds.

[0062] The point cloud of real scenes is very complex. The point cloud obtained by the UAV LiDAR system contains objects of different scales. Extracting objects of different scales from the complex scene point cloud is the basis for object-level registration. To achieve fast object extraction, the segmented point cloud must first be classified.

[0063] Ground points play the role of connecting different objects in segmented point clouds. In order to classify different types of objects, cloth filtering (Zhang et al., 2016) is first used to separate ground point clouds from non-ground point clouds. Ground point clouds of different scales in non-ground point clouds can be mainly divided into linear, planar, and cylindrical ground objects according to their types. In addition to the above three types of ground point clouds in non-ground point clouds, the remaining points can be regarded as redundant points in the object-level point cloud registration process.

[0064] By setting the linearity threshold of the point cloud (L th ) and the flatness threshold (P th) can greatly divide linear points and planar points. The linearity and planarity of the point cloud can be calculated through the covariance matrix. The calculation of the covariance matrix requires the use of the neighborhood information of the point. Directly performing neighborhood search on a point cloud without topological structure is extremely inefficient. Using the KD tree to construct the point cloud topological relationship and perform K-neighborhood search can quickly calculate the covariance matrix of each point in the segmented point cloud, thereby obtaining the eigenvalues ​​and eigenvectors. The eigenvalues ​​are arranged in order of size as λ1>λ2>λ3, and the corresponding eigenvectors are v1, v2, and v3. According to the eigenvalues ​​and eigenvectors, the linearity (L λ ) and flatness (P λ ), the calculation formula is as follows:

[0065]

[0066]

[0067] The greater the linearity, the greater the possibility that the point and its neighboring points form a linear structure. Similarly, the greater the flatness, the greater the possibility that the point and its neighboring points form a planar structure. Since some linear features are inside the surface features, it is also necessary to use the existence of normal vectors to constrain. Independent linear feature points should not have normal vectors. According to experience, the linearity and planarity thresholds are set in the range of [0.8, 0.9]. In order to ensure that the number of feature points extracted later is sufficient, this paper uniformly lowers the threshold appropriately and sets L th =0.7, P th =0.7.

[0068] Among the remaining points except linear and surface points, columnar points have a higher degree of aggregation and their curvature values ​​are smaller than those of redundant points. th ) and the curvature value threshold (C th ) constraint can extract columnar feature points, and the discreteness (S λ ) and curvature value (C λ ) is calculated as:

[0069]

[0070]

[0071] The setting range of the discreteness threshold is usually [0.05, 0.1], and the setting range of using the curvature threshold to constrain scattered points is usually [0.3, 0.8]. Similarly, to ensure that the number of extracted ground object points is sufficient, S is uniformly set th =0.15, C th = 0.7. The remaining redundant points can be directly removed during the object-level registration process. In the segmented point cloud, the division effect of different types of point clouds is as follows: Figure 2 shown.

[0072] This solution creatively divides object point clouds into three different types, which can identify and distinguish most of the point cloud data. It can select the types of data that are more important for the true restoration of the environment through different types of object point cloud data for key processing, and can improve the accuracy of data processing as much as possible under the premise of limited data computing power, thereby improving the true restoration of the scene.

[0073] By using object-level point cloud registration, point cloud monomer separation, feature descriptor calculation, and feature similarity matching are performed in sequence. Through the above steps, the matching relationship between objects can be quickly established.

[0074] like Figure 3 , Figure 4 , Figure 5 As shown, point cloud monomers are separated, and point clouds of various types of objects are separated separately.

[0075] Each type of point cloud after classification is only a general overall type of point cloud, and the basic unit of object-level point cloud registration is the object point cloud in the segmented point cloud. To obtain the object point cloud, it is also necessary to separate the same type of point cloud into monomers. The monomer separation method of the same type of point cloud currently mainly uses the Euclidean clustering algorithm, the K-means clustering algorithm, and the DBSCAN density clustering algorithm. Clustering-based algorithms usually need to use the K-neighborhood information of the point cloud, which will undoubtedly cause a huge amount of calculation for the flight path point cloud. In two-dimensional image processing, there are many mature methods for dividing different areas. Therefore, this paper will two-dimensionalize the segmented point cloud after category division, and achieve the purpose of efficient monomer separation of the same type of ground object point cloud through the two-dimensional image algorithm.

[0076] As an important algorithm for binary image analysis, the connected region labeling method has extremely high robustness. Therefore, the segmented point cloud after category division can be converted into a binary image for connected region labeling method, so as to achieve the purpose of monomer separation. The so-called binary image (BI) means that there are only two grayscale levels in the image, that is, the grayscale value of any pixel in the image can only be 0 or 255, representing black and white respectively. Determining the size of the binary image is the first task in converting the point cloud into a binary image. Since the UAV LiDAR flight path point cloud extends roughly in the Y-axis direction in the spatial rectangular coordinate system, by calculating the maximum and minimum horizontal and vertical coordinates (x max , x mi n;y max ,y mi n) Get the length l and width w of the binary image:

[0077]

[0078] For the side length l of a single grid in a binary graph cell, set l cell The smaller the value, the higher the subsequent processing accuracy, but it will also lead to greater memory consumption. In real scenes, different objects usually exist at a certain distance, so l is set here. cell =0.5m to ensure that the same object is not divided into multiple objects. After the two-dimensional grid is established, the point cloud points are positioned on the grid. The row and column numbers of the grid where the point is located can be calculated based on the point coordinates. The calculation formula is:

[0079]

[0080] Row and Column are the row and column numbers of a point in the binary image; floor() means rounding down; x and y are the horizontal and vertical coordinates of the point cloud. If there is a point in the positioning grid, its grayscale value is defined as 255; if there is no point, the grayscale value is 0. The binary images of linear, planar, and columnar point clouds in the segmented point cloud are as follows Figure 3 shown.

[0081] The connected domain labeling method of binary graph is as follows Figure 4 As shown in the figure, it can be divided into four-connected labeling method and eight-connected labeling method (He et al., 2017). The main difference lies in the different search directions near the query point. The former is 4 directions, and the latter is 8 directions. The eight-connected labeling method has higher accuracy than the four-connected labeling method, but it also has higher complexity. In order to ensure the accuracy of separation, this paper adopts the eight-connected labeling method.

[0082] After completing the separation of different object point clouds of the same type in the two-dimensional image, the point clouds are back-projected back to the three-dimensional point cloud, and the monomer separation of the same type of ground object point clouds is completed. The monomer separation effect of the three types of objects in the segmented point cloud is as follows: Figure 5 shown.

[0083] S13 uses the similarity of semantic features to establish a matching relationship between objects;

[0084] The single object in each category set of the linear object point cloud, the planar object point cloud and the cylindrical object point cloud is associated and matched with the single objects in the category set and other category sets.

[0085] Each single object has its own characteristics that distinguish it from similar objects. The degree of distinction of object feature description directly determines the accuracy of the subsequent search for similarity between objects. In order to describe the semantic feature information of the object as detailed as possible, object-oriented geometric features and shape features are selected, and the semantic feature descriptor of the object is constructed using these two types of features.

[0086] (1) Geometric features: Point clouds regarded as objects usually have rich geometric features that are different from scattered points. At the same time, different individuals of the same object also have highly distinguishable geometric features.

[0087] For geometric dimensions, Figure 6 As shown in the figure, the left and right figures are not objects of the same name. There may be obvious size differences between objects of the same type, such as the lengths of linear objects may be obviously different; the aspect ratios of planar objects may be obviously different; the height differences of columnar objects may be obviously different, etc. Therefore, we can first establish the size features of the object point cloud, and calculate the length L, width W, and height H of the minimum directed bounding box of the object point cloud as the feature values ​​of the three dimensions to form the size feature vector (SD):

[0088] SD=(L,W,H)

[0089] If two objects of the same type in a conjugate point cloud are point clouds with the same name, they should also have the same height feature. An object point cloud is essentially a point set, and its height feature is difficult to measure. Here, the maximum value of the Z axis in the object point cloud point set is uniformly calculated. max As its height value, and because the ground height fluctuates, it also needs to be represented by an accurate value. Figure 7 As shown in the figure, the ground points of the segmented point cloud are projected onto the XOY plane to generate a 3m×3m two-dimensional grid map, and the minimum height value in all grids is taken as the reference ground height value Z ground , then the height feature (ΔZ) can be expressed as:

[0090] ΔZ=Z max -Z ground

[0091] (2) Shape features: The shape feature description of the object point cloud is based on voxels and multi-view methods.

[0092] The distribution of the object point cloud in space indirectly reflects its shape characteristics. To measure its spatial distribution, we can use voxels as the smallest unit for proportional statistics. Figure 8 As shown in the figure, the minimum bounding box of the object point cloud is first divided into cubes with a side length of d and a height of h, and the voxelization of its minimum bounding box is completed. The corresponding voxels are filled in sequence according to the point coordinates of the object point cloud, and the voxel unit with the object point cloud point is used as a solid voxel.

[0093] The ratio of the number of solid voxels to the total number of voxels is considered as the solid voxel ratio feature (p). The solid voxel ratio feature reflects the point distribution of the object point cloud in the local space to a certain extent, and can play a strong role in distinguishing point clouds of similar objects. The calculation formula of the solid voxel ratio feature is:

[0094]

[0095] v s and ve Respectively represent the number of solid and hollow voxels in the minimum bounding box space of the object point cloud. The size of the voxel directly determines the accuracy of the feature description. The larger the side length d and height h are set, the lower the accuracy of searching for objects with the same name in the object point cloud using this feature. In order to take into account the search efficiency of this feature, d = 10cm and h = 15cm are uniformly set.

[0096] Describing the shape features of an object in two-dimensional space is an efficient way, and its efficiency in the field of two-dimensional object recognition can reach real-time level (Lin et al., 2016). Projecting three-dimensional objects and then describing shape features is a dimensionality reduction processing idea. The specific idea is to generate a histogram that can describe the outline of the object point cloud by projecting in a certain direction. Based on this, the accuracy can be further improved by using three-view projection to generate three groups of histograms. First, the directed bounding box technology is used to generate the main direction of the minimum bounding box of the object point cloud, and the object point cloud is projected along the main direction to generate a two-dimensional image; then the minimum circumscribed moment of the projected image is established, and the image is divided into eight equal parts along one of the two directions. Each equally divided rectangle is evenly divided into small grids of equal length. The value in the grid is regarded as a unit area (s cell =1), and then calculate the pixel area in each equally divided rectangle to form Fig. 9 The pixel area feature histogram shown in Figure 1 is used to describe the shape features of the object point cloud. The specific expression is:

[0097] S [1]h =(s1,s2...s8)

[0098] Among them, S [1]h is the vector expression of the pixel area feature histogram projected along the main direction; s i is the pixel area value of each equally divided rectangle. Similarly, the semantic object can be projected into two sub-directions to complete the construction of the three-view pixel area feature histogram. Then the feature (S h ) is:

[0099] S h =[S [1]h S [2]h S [3]h ]

[0100] Through the above description, a multi-dimensional semantic feature descriptor (F) can be formed, thereby achieving the purpose of more comprehensive description of the object point cloud. The semantic feature descriptor expression is:

[0101] F=(SD,ΔZ,p,S h )

[0102] S14 establishes generalized corresponding points of the same name by using the similarity of point primitive features between semantic objects of the same name, obtains a point cloud of the semantic objects of the same name, and synchronously performs primitive-level point cloud registration on the point cloud of the semantic objects of the same name and the ground point cloud;

[0103] The point primitive matching, combined distance error equation construction and conversion parameter solution are carried out in sequence.

[0104] like Fig.10 As shown, when performing point primitive matching, different types of point clouds of objects to be registered are matched with pre-stored reference segmented point clouds of corresponding types, which can effectively reduce the search and matching time and the amount of calculation.

[0105] The object point cloud that is assigned with a semantic feature descriptor label is called a semantic object, and the process of finding matching semantic objects in a conjugate point cloud is called homonymous semantic object detection. To achieve this process, the similarity of semantic objects needs to be evaluated. The key to evaluating the similarity of semantic objects lies in the selection of distance measurement methods for various types of spaces. Existing distance measurement methods for feature space mainly use Euclidean distance, Chebyshev distance, cosine similarity, etc. This paper uses Euclidean distance as the distance measurement standard for semantic feature descriptors in feature space, and regards a pair of semantic objects with the smallest Euclidean distance of feature descriptors in the conjugate point cloud as homonymous semantic objects.

[0106] Euclidean distance is an easy-to-understand distance measurement method, which is derived from the distance formula between two points in Euclidean space. For two n-dimensional vectors A(a1, a2...a n ) and B(b1,b2...b n ), the calculation formula can be expanded to:

[0107]

[0108] Among them, dist(A,B) is the Euclidean distance between A and B. In practice, the specific detection steps are as follows:

[0109] (1) In the conjugate point cloud, one of the segmented point clouds is selected as the reference segmented point cloud and the other is selected as the segmented point cloud to be registered.

[0110] (2) In the segmented point cloud to be registered, the Euclidean distance between the semantic object and the same semantic object in the reference segmented point cloud is calculated. The process is as follows: Fig.10 For example, when registering linear semantic objects in segmented point clouds, only the Euclidean distance with the linear semantic object feature descriptor in the reference point cloud is calculated, so that the search space is greatly compressed and the computational efficiency is improved to a certain extent.

[0111] (3) A pair of semantic objects with the smallest sub-Euclidean distance between semantic feature descriptions is selected and determined as semantic objects with the same name. All conjugate point clouds between different flight strips are traversed in sequence to complete the object-level point cloud registration of different flight strips.

[0112] like Fig.12 As shown, from left to right, there are two different segmented point clouds through primitive-level matching. By using primitive-level matching and comparison of different types of object point clouds in each segmented point cloud, not only can the matching be more accurate, but also meaningless matching and comparison between different types can be avoided, reducing the amount of comparison calculations.

[0113] like Fig.13 As shown, point clouds of different types of objects establish corresponding points of the same name for different types of objects. Compared with the substantial corresponding points of the same name established in the prior art, the generalized corresponding points of the same name in this solution are established according to the object category, which is not only faster but also ensures a certain degree of accuracy.

[0114] This scheme abandons the current common method of registering actual homonymous points and eliminates the need to set up actual homonymous points. Compared with the existing technology, this scheme requires very small amounts of data and the calculation steps are relatively simpler. However, due to the establishment of generalized homonymous corresponding points, it is possible to achieve accurate semantic registration of point cloud data at each object level based on the classification of object point clouds.

[0115] like Fig.14 As shown, the accuracy of non-rigid registration using the method of this embodiment is significantly improved.

[0116] S2, taking into account the overall optimization of the point cloud with non-rigid deformation, calculates the flight zone adjustment of multiple point clouds:

[0117] Establish the pose error observation equation for each flight and each flight path point cloud in the test area; use the cubic spline function to approximate the time state of the trajectory pose error, and establish a time-varying model of the UAV lidar point cloud that conforms to the characteristics of multiple flights;

[0118] The multi-strip initial point cloud including the collected point cloud data and the original POS data is estimated for the same-name correspondence, and non-rigid registration and same-name corresponding point sampling are performed in sequence to obtain a high-precision same-name corresponding point cloud. The high-precision same-name corresponding point cloud and the original POS data are substituted into the existing initial time-varying model, and the error equation is constructed through the existing posture error observation equation and posture error time approximation.

[0119] S3, by diagonalizing the coefficient matrix of the overall error equation of different flight strips for different sorties, the final flight strip adjustment is calculated through the UAV lidar point cloud time-varying model.

[0120] After the error equation is established, nonlinear optimization is performed to solve it. After the coefficient equation is diagonalized, it is substituted into the error equation and solved by the LM algorithm. When correcting the error model, the error model is corrected after comparing the solution obtained by experimental data calculation with the actual flight zone adjustment. This process is repeated to obtain the final error model.

[0121] When the actual point cloud data is substituted into the final error equation, the final flight zone adjustment is obtained.

[0122] like Fig.11 As shown in the figure, the upper part from left to right is the effect diagram of the actual data after monomer separation, and the lower part from left to right is the effect diagram of the monomer separation of the experimental data object point cloud during the model construction process. It can be clearly seen from the figure that the effect in the actual application is almost the same as that in the experimental stage. This scheme can complete the accurate separation and identification of monomers under the premise of limiting the amount of calculation, and provide a prerequisite for the subsequent accurate calculation of the flight strip adjustment.

[0123] like Fig.15 As shown in the figure, after the actual data point cloud is collected, Fig.15 After segmentation as shown, the final flight zone adjustment is calculated and optimized according to the above method. The flight zone adjustment errors before and after optimization by the method of this embodiment are compared, and the following is obtained: Fig.16 The error distribution comparison diagram is shown in Figure 2. Fig.16 It can be seen intuitively that after optimization by this method, the error is effectively reduced and the accuracy is effectively improved.

[0124] Among them, when evaluating the accuracy of the flight strip adjustment before and after optimization, it is necessary to strictly evaluate the non-rigid deformation correction effect of all areas within the survey area. It is necessary to take the one with the smaller flight strip number among the overlapping flight strips as the reference flight strip, and use time as the horizontal coordinate and the distance between each corresponding point with the same name on the other flight strip and the corresponding point with the same name on the reference flight strip as the vertical coordinate to establish a statistical graph of the error change over time. Here, the time is based on the time of the corresponding point with the same name on the other flight strip. The trend of this statistical graph can express the error distribution of different areas, and then judge the correction effect of non-rigid deformation.

[0125] In this embodiment, the columnar objects include columnar objects of various different attributes, such as lighting street lamps, traffic signs, and traffic lights. In complex road scenes, objects often adhere to each other, forming a mixed adhesion object of multiple objects. It is proposed to use Euclidean clustering segmentation, combined with morphological feature adhesion object detection and minimum cut-based adhesion object segmentation to complete the object segmentation. Lighting street lamps, traffic signs, traffic lights, and roadside trees are all columnar objects. Compared with objects such as pedestrians, guardrails, and vehicles, columnar objects have obvious height characteristics; at the same time, compared with objects such as buildings, columnar objects have smaller projection length characteristics. Based on these morphological information, a height filter and a length filter can be used to filter out most of the non-columnar objects from the point cloud objects. Although there are large spatial morphological differences at the tops of columnar objects, they all have vertical columnar parts in the middle. The columnar parts of roadside trees are very rough and discrete, while the columnar parts of traffic columns are more regular and smooth. The columnar parts of roadside trees are rough and discrete, and have no fixed geometric structure, so there are large differences in the center position and diagonal length of each cross-sectional slice. Compared with roadside trees, the columnar parts of traffic pillars are regular and smooth cylinders, and the differences between the center position and diagonal length of each cross-sectional slice are small. The deviations of the center position and diagonal length between the minimum circumscribed moments of adjacent cross-sectional slices are checked in turn, and the frequency of occurrences greater than the deviation threshold is counted. If the deviation statistical frequency is less than the frequency threshold, the point cloud object is extracted as a traffic pillar.

[0126] In this embodiment, when establishing the error model, a time-varying model of the UAV lidar point cloud that conforms to the characteristics of multiple sorties is first established.

[0127] The positioning error of high-precision POS is related to the quality of GNSS signals. During the flight of the drone, since it is in the air, it is less affected by the multipath effect and the signal quality is often good, so it rarely loses lock. Its positioning accuracy can be within 10cm, or even up to 2cm. The high-precision POS system can also control the angle measurement accuracy within 0.05°.

[0128] The position and attitude parameters are expressed by the following formula:

[0129]

[0130] Among them, B0, L0, H0, r0, p0, and h0 are the observation values ​​of trajectory position and posture obtained by POS solution, that is, POS trajectory point values, dB, dL, dH, dr, dp, and dh are the error values ​​contained in the trajectory, and the random errors are all unknown quantities.

[0131] The position and attitude errors of POS exist in the whole flight strip scanning work. To fit them with the cubic spline function, it is necessary to express the error values ​​of the six parameters in the trajectory in each time period one by one. All error values ​​can be expressed by the cubic polynomial function:

[0132]

[0133] Where a, b, c, d, e, and f are the coefficients of the cubic polynomial respectively. In the subscript, i is the flight zone number, j is the time segment interval number, and 0, 1, 2, and 3 are the coefficients of the cubic polynomial respectively. i，j ) to the power of .

[0134] Substitute the above error values ​​into the following error vector formula, Δ W is the point error vector:

[0135]

[0136] Then in a certain period of time, the time is [t i,j ,t i,j+1 ], the kth point at time t, its position error ΔW becomes as follows:

[0137]

[0138]

[0139] in, The superscripts i, j, and k in represent the kth point of the jth segment of the i-th flight strip. Suppose the vector KA i，j，k KB i，j，k , KC i，j，k , KD i，j，k , KE i，j，k , KF i，j，k With x i，j , where x i，j yes The unknowns in are to be solved, and the rest are known numbers. Order:

[0140]

[0141] K i，j，k =[KA i，j，k KB i，j，k KC i，j，k KD i，j，k KE i，j，k KF i，j，k ]

[0142] but:

[0143] After solving all the unknowns x, the cubic spline function is established and the error value of each point can be obtained. If there are n+1 time nodes and the node time interval is Δt, there are 24n unknown parameters in total.

[0144] Solution of the error fitting model:

[0145] If it is the adjustment of two flight path point clouds, the distance between the corresponding relationships with the same name is set as:

[0146] l=p T -p S

[0147] The superscripts “S” and “T” represent the first and second flight strips, respectively. The total number of unknown parameters to be solved is 24 (n S +n T ), then all unknown parameters are:

[0148]

[0149] All known numbers can be expressed as:

[0150]

[0151] according to Fig.17 The corresponding formula of the cubic spline function is used as the constraint condition equation group Cx=0, with a total of 12+18(n S +n T ) conditions, then the adjustment function is:

[0152]

[0153] Since the above formula is highly linear, it is difficult to solve the parameters by linearization, and in general, there is no solution in the usual sense, but there is a least squares solution, V T PV = min, where P is the weight matrix. Since the UAV platform needs to maintain a relatively stable altitude on the route, and the external conditions in the air are roughly the same during the measurement process, the process can be regarded as an equal-precision observation, and the weight matrix is ​​set to the unit matrix. The least squares method is used to solve the parameters of the system of equations with constraints, and the solved function becomes The goal is to make f T f=min.

[0154] When establishing generalized corresponding points of the same name, this embodiment sequentially performs point primitive matching, combined distance error equation construction, and conversion parameter solution.

[0155] Because the accuracy of the time-varying function fitting model is related to the spline time interval T, the external accuracy can reflect whether the fitting conditions of the time-varying pose model are reasonable, such as whether the ground control points are sufficient and whether the size of T is appropriate. If there are enough ground control points, the control point spacing is small, and the size of T is appropriate, the RMSE values ​​of the internal accuracy and the external accuracy are almost the same. When T is constant, if the control point spacing becomes smaller, the internal accuracy may be higher than the external accuracy. The smaller T is, the more unknown parameters there are, the more homonymous correspondences are required, the number of equations increases, and the solution takes more time and storage space. As the unknown parameters increase, the stability of the solution decreases. The stability of the solution is affected by the control point spacing and T. When T is too small, overfitting is prone to occur. If the control point spacing or T is too large, underfitting will occur, resulting in reduced adjustment accuracy. Therefore, it is necessary to study the reasonable setting of T and to arrange ground control points with short spacing and as many numbers as possible. When the ground control point spacing is short and T is appropriately small, the pose error fitting is more accurate, the adjustment accuracy is higher, and the possibility of overfitting is smaller. The smaller the control point spacing, the more the same-name correspondences, and the greater the workload. When the number of control points increases to a certain number, the adjustment accuracy will no longer improve due to the influence of random measurement errors. The general principle of control point layout is to reduce the number as much as possible while ensuring accuracy.

[0156] Without considering the correspondence of the same name matching, if it is guaranteed that there are 3 or more control points in each spline time interval T. Let v represent the speed of the measuring vehicle, and the measuring distance in the T time period is T×v. Assuming a straight line with a constant speed, the average spacing of the control points should be no more than T×v÷3. Table 1 lists the minimum control points that need to be deployed for T=10s. It can be seen that when driving on a highway, the spacing between ground control points can be appropriately increased and the number of ground control points can be reduced. In the case of actually considering the revisit point cloud registration, the actual control points deployed are less than the number of preset control points, but the offset caused by the attitude error is not only related to the scanning distance measurement, but also to the driving direction. In order to be able to solve the attitude error offset, control points need to be deployed where the vehicle turns sharply.

[0157] The above is only an embodiment of the present invention, and the common knowledge such as the known specific technical solutions and / or characteristics in the solution is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical solution of the present invention, several modifications and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A UAV LiDAR flight zone adjustment method integrating semantic features, characterized in that: The following steps are involved: Step 1: Multi-level non-rigid registration for semantic objects: The flight path point cloud that has been pre-processed is divided into multiple segmented point clouds that can be regarded as rigid registration units; by separating the segmented point clouds into object point clouds of different categories, the matching relationship between objects is established by using the similarity of semantic features, and the generalized corresponding points of the same name are established between semantic objects of the same name by using the similarity of point primitive features; In step 1, the object point cloud includes a linear object point cloud, a planar object point cloud and a cylindrical object point cloud; In step 1, the matching relationship between objects is established by using the similarity of semantic features, that is, a single object in each category set of linear object point cloud, planar object point cloud and cylindrical object point cloud is associated and matched with a single object in the category set and in other category sets; In step 1, the matching relationship between objects is established by using the similarity of semantic features, including sequentially performing point cloud monomer separation, feature descriptor calculation, and feature similarity matching; In step 1, establishing generalized corresponding points of the same name by using the similarity of point primitive features between semantic objects of the same name includes sequentially performing point primitive matching, constructing a combined distance error equation, and solving a conversion parameter; Step 2: Multi-flight point cloud flight strip adjustment taking into account non-rigid deformation: establish the pose error observation equation for each flight and each flight strip point cloud in the measured area, and use the cubic spline function to approximate the trajectory pose error in time state, and establish a UAV lidar point cloud time-varying model that conforms to the characteristics of multiple flights; Step three, by diagonalizing the coefficient matrix of the overall error equation of different flight strips for different sorties, the final flight strip adjustment is calculated through the UAV lidar point cloud time-varying model.

2. A UAV laser radar flight zone adjustment method integrating semantic features according to claim 1, characterized in that: In step one, the flight strip point cloud that has undergone point cloud preprocessing is obtained by scanning multiple UAV laser radars to form a multi-strip initial point cloud and then processed according to the preprocessing strategy.

3. A UAV laser radar flight zone adjustment method integrating semantic features according to claim 2, characterized in that: The preprocessing strategy is to perform ground filtering, point cloud time and space division, and conjugate point cloud detection in sequence.

4. A UAV laser radar flight zone adjustment method integrating semantic features according to claim 3, characterized in that: The point cloud is divided into time and space according to the heading rule, time rule and speed rule in sequence.

5. A UAV laser radar flight zone adjustment method integrating semantic features according to claim 2, characterized in that: Before step 3, the error equation is constructed. In step 3, the coefficient equation is diagonalized and solved by the LM algorithm.

Images