A UAV positioning and mapping method and system based on semantic information matching

Abstract

The present invention discloses a method and system for UAV positioning and mapping based on semantic information matching, belonging to the field of artificial intelligence. Aiming at the problems of low accuracy and slow efficiency in existing UAV positioning and mapping, the present invention provides a method for UAV positioning and mapping based on semantic information matching, including the following steps: obtaining laser data, latitude-longitude-altitude data, and RPY angle data; performing semantic segmentation and semantic matching on the laser data; then fusing and positioning with translational changes and rotational changes; and finally outputting the result. The present invention reduces the data processing volume by eliminating laser data, improves work efficiency, and at the same time can effectively associate data information between adjacent frames through semantic matching, thereby obtaining positioning information more accurately and synchronously improving the mapping credibility. The accuracy of the entire method is effectively guaranteed, and at the same time, the robustness and anti-interference ability are stronger. The system of the present invention has a simple composition, high work efficiency, fast data processing, and ensures accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and more specifically, relates to a method and system for positioning and mapping an unmanned aerial vehicle based on semantic information matching. Background Art

[0002] Currently, unmanned aerial vehicles (UAVs) are increasingly used in the military and industrial fields. Especially in scene measurement, material transportation, etc., their economic practicality, light weight and convenience are more prominent. And real-time positioning of UAVs and constructing an environmental map is an important application direction. Small UAVs mostly adopt the remote control method. Their load capacity is relatively low, and they cannot effectively carry sensing devices that can work stably for a long time. Moreover, their working radius is limited by the controllable radius, so there are great limitations in map construction. Medium and large UAVs are equipped with sensing and navigation control systems such as cameras and lasers. Their load capacity and endurance are relatively good, and the control radius can usually reach several kilometers, and they can even complete tasks autonomously without remote control. Currently, the market applications of intelligent devices with positioning and mapping requirements are mostly vehicles on structured roads (such as the assisted driving function under high-precision maps), indoor robots (such as sweeping robots, food delivery robots), etc. The applications of UAVs in the external environment mainly focus on camera aerial photography, swarm performances, etc., and a small part is used for surveying, transportation and other tasks. For such intelligent devices, their perception fusion positioning and mapping methods mostly adopt laser point cloud matching methods or information fusion with RTK, GPS, IMU, etc., and are updated after reaching the convergence standard.

[0003] However, in the existing multi-sensor fusion positioning and mapping (SLAM, Simultaneous Localization and Mapping) solutions: the working environments of intelligent vehicles and robots mostly have strong structural characteristics, such as roads, rooms, etc., and their distribution structures on the ground are mostly wheeled or legged. Therefore, according to their characteristics, the influence in the height axis direction can be better removed in the selection of SLAM solutions. However, the flight state of UAVs is relatively flexible, and it must be able to effectively iterate the changes in the height axis direction. And the current positioning and mapping applications on UAVs are divided into two main directions: vision and laser: a. The accuracy of visual distance measurement is relatively poor, and its error offset is difficult to converge at long distances. Moreover, visual sensors mostly rely on the principle of visible light, so their performance is poor under external exposure, shadow and other conditions, and it is difficult to meet the robustness requirements; b. In the laser solution, the common SLAM solutions are mainly multi-source fusion point cloud matching, with low overall accuracy and complex algorithm design structures.

[0004] For example, Chinese Patent Application No. CN201810597661.5, with a publication date of July 20, 2018, this patent discloses a method for multi-scenario positioning and 3D map construction of an unmanned aerial vehicle (UAV) based on 3D lidar. When there is no GPS signal in an indoor environment, a SLAM algorithm based on a 3D laser sensor is established for positioning and 3D map construction. In an outdoor environment, the position information provided by RTK is mainly used for 3D map construction. After the RTK loses the GPS signal, SLAM is used for assisted positioning. It realizes the rapid construction of indoor and outdoor 3D maps, provides a convenient and efficient method for obtaining environmental 3D maps, and greatly improves the efficiency of indoor and outdoor environmental 3D map construction. Based on the theoretical basis of optimizing the SLAM algorithm in the graph, the SLAM algorithm for the UAV platform based on the 3D laser sensor is completed, realizing the construction of the environmental 3D map in an environment without GPS signal, solving the problems of UAV indoor positioning and indoor-outdoor positioning fusion, and meeting the requirements of multi-scenario UAV positioning. On the basis of obtaining high-precision pose information, laser data is fused to complete the construction of the 3D map. The disadvantages of this patent are as follows: RTK information and the edge surface features of the laser point cloud are used for matching and fusion positioning and mapping. This solution has low accuracy in point cloud feature extraction, and the inter-frame correlation information is not used, resulting in a decrease in mapping accuracy and relatively high requirements for result convergence.

[0005] Another example is Chinese Patent Application No. CN202111297853.2, with a publication date of January 28, 2022. This patent discloses a method and device for 3D map construction and positioning of an unmanned aerial vehicle based on SLAM. The method includes: performing noise reduction preprocessing on multiple frames of initial point cloud data obtained by a lidar device to obtain multiple frames of first point cloud data; filtering ground static targets and dynamic targets from each frame of the first point cloud data to obtain second point cloud data; matching four coplanar points of adjacent two frames of the second point cloud data based on a first matching algorithm to obtain a first point cloud matching result; and performing iterative calculation on the first point cloud matching result based on a second matching algorithm to obtain each frame of the second point cloud data and the first navigation attitude information of the corresponding UAV. The disadvantages of this patent are as follows: The algorithm is complex to run, with low efficiency and low overall mapping accuracy. Summary of the Invention

[0006] 1. Problems to be Solved

[0007] Aiming at the problems of low accuracy and slow efficiency in the existing UAV positioning and mapping, the present invention provides a UAV positioning and mapping method and system based on semantic information matching. By removing laser data to reduce the data processing volume, the present invention improves the working efficiency. At the same time, through semantic matching, the data information between adjacent frames can be effectively associated, so as to obtain more accurate positioning information, and the mapping credibility is also improved synchronously. While the accuracy of the whole method is effectively guaranteed, the robustness and anti-interference ability are stronger. The system of the present invention has a simple composition, high working efficiency, fast data processing and ensures accuracy.

[0008] 2. Technical solution

[0009] To solve the above problems, the present invention adopts the following technical solutions.

[0010] A UAV positioning and mapping method based on semantic information matching, comprising the following steps:

[0011] S1: Obtain multiple frames of laser point cloud data, and at the same time obtain multiple frames of latitude-longitude-altitude data and multiple frames of RPY angle data synchronized with the multiple frames of laser point cloud data in time. An initial coordinate system is obtained based on the first frame of latitude-longitude-altitude data and the first frame of RPY angle data, and a map is initialized and constructed based on the first frame of laser point cloud data;

[0012] S2: Perform semantic segmentation on each remaining frame of laser point cloud data in step S1, and remove dynamic targets in the laser point cloud data;

[0013] S3: Perform semantic matching on two adjacent frames of laser point cloud data in step S2 to obtain the pose change between the two adjacent frames of laser point cloud data;

[0014] S4: Calculate the translational change of the latitude-longitude-altitude data of adjacent frames for the remaining frames, and calculate the rotational change of the RPY angle data of adjacent frames for the remaining frames;

[0015] S5: Fuse and locate the translational change and rotational change obtained in step S4 with the pose change obtained in step S3 to obtain the fused pose data, and then output the result.

[0016] Furthermore, step S2 specifically includes the following steps:

[0017] S21: Downsample the laser point cloud data;

[0018] S22: Perform target detection on the downsampled laser point cloud data, classify the targets into static targets and dynamic targets, and label the targets with corresponding classification labels;

[0019] S23: Remove dynamic targets in the laser point cloud data in step S22.

[0020] Further, in step S22, the static targets are further divided into artificial objects and natural environments. Different artificial object targets are correspondingly set with different labels, while the natural environment is not labeled.

[0021] Further, the specific steps for removing dynamic targets in step S23 are as follows: When the central pose calculated for the same marked dynamic target changes by more than the threshold in three consecutive frames, it is considered to be changing, and it is removed.

[0022] Further, before the result is output in step S5, map calibration and update are also included, which specifically includes: taking the difference between the pose data after fusing the previous and the next frames. If the difference meets any one of the following three conditions, the map is updated and then the result is output: a: The displacement in other axes except the height axis exceeds 1.0 m; b: The displacement in the height axis exceeds 0.5 m; c: The angular deviation in any axis exceeds 10°.

[0023] Further, the map calibration and update modes are divided into the following types:

[0024] When the matching degree between adjacent two-frame lidar point cloud data in step S3 is higher than the set value, only the non-overlapping part of the map is updated;

[0025] When the matching degree between adjacent two-frame lidar point cloud data in step S3 is lower than the set value, but the fused pose data is within the positioning convergence range, the lidar point cloud data after the current pose transformation is updated on the original map;

[0026] When the fused pose data exceeds the positioning convergence range, the map is not updated; when this situation occurs for five consecutive frames of fused pose data, the existing map is backed up, the newly observed lidar point cloud data is added to the current existing map, and a map warning signal is given.

[0027] Further, when the continuously updated map file size exceeds the preset value, the map file is truncated for storage, and the steps of S1 to S5 are repeated for positioning and mapping.

[0028] Further, the map output results include two methods: continuously updated submaps; a global overview map, and the global overview map includes several submaps generated by truncating when exceeding the preset value size.

[0029] A system using the unmanned aerial vehicle positioning and mapping method based on semantic information matching as described in any one of the above, comprising:

[0030] Lidar: used to obtain lidar point cloud data;

[0031] GPS: used to obtain latitude, longitude, and altitude data;

[0032] Inertial navigation RTK: used to obtain RPY angle data;

[0033] Translation and rotation change module: used to calculate the translation change from the data read by GPS and calculate the rotation change from the data read by inertial navigation RTK;

[0034] Map building initialization module: used to build an initial environment map;

[0035] Semantic segmentation module: used to perform semantic segmentation on each frame of lidar point cloud data and remove dynamic targets from the lidar point cloud data;

[0036] Semantic matching module: used to perform semantic matching on two adjacent frames of lidar point cloud data to obtain pose changes;

[0037] Fusion positioning module: used to fuse the results in the semantic matching module with the results in the translation and rotation change module;

[0038] Output module: used to output the results.

[0039] 3. Beneficial effects

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] (1) The present invention initializes map building with the first frame of lidar point cloud data, then performs semantic segmentation on each frame of lidar point cloud data to remove dynamic targets from the lidar point cloud data, reducing the subsequent calculation amount of data and simplifying the model of the lidar point cloud; then, through semantic matching, the data information between adjacent frames can be effectively associated, so as to obtain more accurate positioning information, making its positioning accuracy higher than that using line and plane feature extraction and matching, and the map building credibility is improved synchronously while the operation real-time performance is high; finally, the results of semantic matching are fused with translation and rotation changes to further obtain relatively high-precision positioning information, update the map with the positioning information, and output the results; the accuracy of the whole method is effectively guaranteed, and the robustness and anti-interference ability are stronger, and the designed algorithm has a low structural complexity, improving work efficiency and reducing development costs;

[0042] (2) Before semantic segmentation, the present invention first downsamples the laser point cloud data by voxel filtering, thereby reducing the computational pressure of the points while retaining the point cloud structure as much as possible, thereby further improving the computational efficiency; and according to the attributes of the static targets, the static targets are divided into man-made objects and natural environments, and the laser point cloud data is more finely labeled, providing a more detailed basis for the subsequent map establishment, ensuring the comprehensiveness of the map establishment; and further specifically limiting the method of eliminating dynamic targets, ensuring the accuracy and completeness of dynamic target elimination, reducing the error caused by the elimination of targets, and reducing the workload of subsequent target processing;

[0043] (3) In order to further ensure the accuracy of map establishment, the present invention also adds a map correction and update step before the result output; and by subtracting the fused pose data of the previous and next two frames, the map is updated if the difference satisfies any of the three items, and the update is comprehensive to ensure the accuracy of the final map establishment; at the same time, different update methods are used according to different situations to avoid a large-scale update of the existing determined environment or to avoid the situation where the map construction is inaccurate due to a large deviation of certain data. The most effective update method is selected for different situations to improve work efficiency while saving resources and reducing time and manpower and material costs;

[0044] (4) In the system of the present invention, each module works independently and is interdependent, with a simple composition and high working efficiency. At the same time, the entire system has high robustness and anti-interference capabilities. The positioning accuracy achieved by semantic matching of lidar data is higher, and the system can effectively associate data between adjacent frames to obtain more accurate positioning information, providing a strong foundation and guarantee for drone positioning and mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with specific embodiments and drawings.

[0047] Example 1

[0048] As drone applications become more and more widespread, using drones for real-time positioning and building environmental maps of unknown environments is a very important application direction. Existing methods for building environmental maps using drones mostly use complex algorithms, with low mapping accuracy, poor real-time performance and poor operability. This application proposes a drone positioning and mapping method based on semantic information matching, such as Figure 1 As shown, the above defects can be overcome, which specifically includes the following steps:

[0049] S1: Acquire multiple frames of laser point cloud data, and simultaneously acquire multiple frames of latitude and longitude data and multiple frames of RPY angle data that are time-synchronized with the multiple frames of laser point cloud data. Both the latitude and longitude data and the RPY angle data are measurements of the drone. The time synchronization here means that each frame of laser point cloud data, each frame of latitude and longitude data, and each frame of RPY angle data are received at the same time to ensure data synchronization and reduce errors. An initial coordinate system is obtained based on the first frame of latitude and longitude data and the first frame of RPY angle data, and a map is initialized and constructed based on the first frame of laser point cloud data. Specifically, the subsidiary coordinate information of the first frame of laser point cloud data is converted from the latitude and longitude data to obtain the axial distance and the PRY angle data to obtain the RPY (Roll, Pitch, Yaw, three attitude angles), thereby obtaining the initial coordinate system, that is, the data of the first frame of laser point cloud is located by the first frame of latitude and longitude data and the first frame of RPY angle data, and a map is established based on this. At the same time, before performing step S1, the data coordinate alignment and information source clock synchronization have been configured;

[0050] S2: semantically segment each frame of laser point cloud data remaining in step S1, and remove dynamic targets in the laser point cloud data. Since dynamic targets have little reference significance for subsequent solution, dynamic targets in the laser point cloud data are removed to reduce the amount of subsequent data calculations and simplify the laser point cloud model to improve efficiency. It specifically includes the following steps:

[0051] S21: Downsampling the laser point cloud data can be performed by voxel filtering (for example, taking a voxel leaf with a length, width and height of 0.2m*0.2m*0.2m, the point cloud can be gridded to reduce the dimension, and its own structures such as corner points, planes, and frames will not be greatly affected) to improve processing efficiency; and the voxel filtering method can retain the point cloud structure as much as possible while reducing the calculation pressure of the number of points, further improving processing efficiency;

[0052] S22: Perform object detection on the downsampled lidar point cloud data, classify the objects into two categories: static objects and dynamic objects, and assign corresponding classification labels to the objects. That is, in this step, object detection is performed on the lidar point cloud to correspondingly segment a series of point clusters, and then classification labels are assigned. The classification labels divide the objects into two parts: static objects and dynamic objects, simplifying the model and improving efficiency. Furthermore, static objects are further divided into man-made objects and natural environments according to their attributes. Man-made objects include ground road bridges, walls, vehicles, utility poles, high towers, etc. Due to the different characteristics of different objects, for example, ground road bridges appear as a large number of continuous planar points in the lidar point cloud data (this continuity means that the surface normal vectors of this series of points are basically the same and have good boundary characteristics), while vehicles appear as a point cloud with a box shape (for this point cloud with a box shape, the boundary formed by the connected domains on each axis has good prism characteristics), walls have continuous planar points perpendicular to the ground, and utility poles and high towers present narrow and long point clouds perpendicular to the ground. The lidar point clouds generated by man-made objects have obvious separable characteristics, so different labels are set for different man-made object targets. Natural environments include trees (tree crowns and trunks), grass, slopes, etc. The point clouds generated by such natural environments do not have obvious separable characteristics, so labels are not set for the point clouds of such natural environments. However, such point cloud data is still input as part of the data into the subsequent steps. This step further makes a closer division of static objects, providing a more detailed basis for the subsequent map building and ensuring the comprehensiveness and accuracy of map building.

[0053] S23: Remove the dynamic objects from the lidar point cloud data in step S22. The specific steps for removal are as follows: Dynamic objects are removed through inter-frame solution iteration. When the center pose calculated for the same marked dynamic object changes by more than the threshold in three consecutive frames, it is considered to be changing and is removed. Specifically, when the center pose calculated for the same marked dynamic object changes by more than a certain threshold in three consecutive frames (the changes in the three translational axes exceed 0.1 m, and the changes in the three rotational angles exceed 0.5°. Converted to speed, with a laser frequency of 10 HZ, that is, when the speed of any one of the three axes exceeds 1 m / s and the angular speed of any one of the three axes exceeds 5° / s), it is considered to be changing and is removed from the environment. When observed later, it is still screened according to this method. It should be noted here that the threshold is not fixed and can be adjusted adaptively according to the on-site situation and component attributes.

[0054] S3: Semantically match the adjacent two-frame lidar point cloud data in step S2 to obtain the pose change of the adjacent two-frame lidar point cloud data. After performing semantic segmentation on each frame of lidar point cloud data through step S2, the objects in the lidar point cloud data environment are divided into several labeled targets and some point clouds that do not belong to the class characteristics. Since in consecutive frames, the change of each target in the point cloud corresponds to the change relationship of the lidar itself, the change relationship can be solved using a point cloud matching algorithm. The matching algorithm can use NDT (Normal Distributions Transform) or ICP (Iterative Closest Point) method. In this embodiment, the NDT algorithm is adopted, that is, the normal distribution situation between different frames of point clouds is compared. After the matching is completed, the backend uses a non-linear optimization method, such as the steepest descent method, Newton method, LM (Levenberg-Marquardt) method, etc., to minimize the matching error function term (implemented using C++ libraries such as ceres or g2o on the embedded development board), and the optimal change relationship is obtained. This method of extracting semantics and performing matching is more targeted than the traditional extraction and matching of line and surface features, greatly improving the matching accuracy (for example, in some usage environments, the number of algorithm optimization iterations can be set to 30 - 50 times, the grid resolution can be set to 1.0m, and the search step size can be set to 0.2m, etc.). After semantic matching, the pose change input by the lidar point cloud data can be obtained, that is, a set of matrices [R, T] about the rotation and translation relationship is obtained;

[0055] Meanwhile, in this step, considering that when there are few semantic targets that can be effectively separated in the lidar point cloud data environment, for example, less than 2 semantic point clusters, it is considered that the current frame cannot use semantic matching, and the NDT method is directly used (at this time, the downsampled point cloud is directly matched) to obtain the spatial distribution density of the point cloud, and it is directly matched with the next frame of point cloud to solve the pose change, fully considering the influencing factors to further ensure the accuracy of mapping;

[0056] S4: Calculate the translational change of the adjacent two-frame latitude, longitude, and altitude data for the remaining frames. Specifically, the latitude, longitude, and altitude data can be measured by the GPS on the UAV, and the coordinate transformation of the latitude, longitude, and altitude data is performed, that is, the latitude, longitude, and altitude data is aligned to the body coordinate and the difference is taken to obtain a set of translational change matrices T'; calculate the rotational change of the adjacent two-frame RPY angle data for the remaining frames. Specifically, the RPY angle data can be measured by the inertial RTK on the UAV, the RPY angle data is converted to the body coordinate, and the change of the RPY between frames can be obtained by taking the difference through the RTK angle integration relationship to obtain a set of rotational change matrices R';

[0057] S5: Fuse and localize the translational and rotational changes obtained in step S4 with the pose change obtained in step S3 to obtain the fused pose data, and then output the results. In this step, since the results obtained through semantic matching are directly obtained from the point cloud relationship, there are certain errors. Therefore, in the fusion localization step, various results need to be optimized to make the results converge. Usually, for the fusion of such two groups of [R, T] relationships, methods such as Kalman filtering can be used; after the fusion localization is completed, a pose transformation [R k+1 , T k+1 with relatively high accuracy is output, which represents the localization result at the (k + 1)-th moment. Update the map with this localization result, and then output the localization result and the map result. The results can be encapsulated into messages and local files for use by downstream software or later offline analysis.

[0058] In the present invention, the obtained lidar point cloud data is used to initialize mapping with the first frame of lidar point cloud data, then semantic information extraction and data processing are performed on each frame of lidar data, and then the semantic information is solved using a matching method to obtain the pose change input by the lidar; from the obtained GPS data, coordinate transformation is performed on it to align the longitude, latitude, and altitude data to the body coordinate to obtain the translational change; from the obtained inertial RTK data, coordinate transformation is performed on it to convert the RPY (Roll, Pitch, Yaw, three attitude angles) information to the body coordinate to obtain the rotational change. The results of semantic matching, the GPS translational change, and the inertial RTK rotational change are fused to obtain the localization information, and the map is updated with this localization information, and then the localization result and the map result are output. The entire method strengthens the association between inter-frame data through semantic matching, thereby obtaining more accurate localization information, and the credibility of mapping is also improved synchronously, with strong operation real-time performance; the algorithm structure used in the entire method has a low complexity and strong operation real-time performance.

[0059] Embodiment 2

[0060] Basically the same as Embodiment 1, in order to further ensure the integrity and accuracy of map drawing, in this embodiment, before the result output in step S5, there is also a step of map calibration and update. When the laser point cloud data is only the first frame, that is, the initial frame, there is no fused positioning information. At this time, the pose information of the UAV when determining the first frame of laser point cloud data is determined according to RTK and GPS information. Therefore, when the map result is only the first frame of laser point cloud data, the current frame of laser point cloud data is used as the environmental map, which has absolute pose accuracy. As the laser point cloud continues to be input, the latitude, longitude, and altitude data (GPS data) and the RPY angle data (inertial navigation RTK data) are continuously input and solved, and the map is updated according to the change of the fused positioning result. Specifically, it includes: subtracting the pose data after fusing the previous and the next frames. If the difference meets any one of the following three conditions, the map is updated and then the result is output: a: The displacement in other axes except the height axis exceeds 1.0 m; b: The displacement in the height axis exceeds 0.5 m; c: The angular deviation in any axis exceeds 10°. The update is comprehensive to ensure the accuracy of the final map establishment.

[0061] Furthermore, the map calibration and update mode in this embodiment is divided into the following types. It should be noted that the matching degree of the laser point cloud data is characterized by the match score. Since the semantic matching in this application uses the NDT algorithm, which will generate a match score. The lower the match score, the higher the coincidence degree of the adjacent two frames of laser point cloud data, and the higher the matching degree is considered:

[0062] (1) When the matching degree between the adjacent two frames of laser point cloud data in step S3 is higher than the set value, the map is only updated for the non-overlapping part; that is, when the matching degree of the adjacent two frames of laser point cloud data is high, that is, the match score of the NDT algorithm is less than 1.0, the map is only updated for the non-overlapping part to avoid large-scale updating of the existing determined environment. At this time, the updated details in the environment are depicted on the original map;

[0063] (2) When the matching degree between the adjacent two frames of laser point cloud data in step S3 is lower than the set value, that is, when the matching degree of the adjacent two frames of laser point cloud data is not high, the match score of the NDT algorithm is greater than or equal to 1.0, but the fused pose data is within the positioning convergence range (the positioning convergence here is known and is comprehensively determined by the own attributes of each sensor (lidar, GPS, inertial navigation RTK), etc. The positioning convergence can judge whether the sensor has a fault or a sudden situation). In this case, the UAV has a large pose change, and there are more updates in the observed environment. At this time, the laser point cloud data after the current pose transformation is updated on the original map;

[0064] (3) When the fused pose data exceeds the positioning convergence range, it indicates that the data may have mutated beyond the convergence range. In this case, for safety reasons, the map is not updated. When this situation occurs in the fused pose data for five consecutive frames, the existing map is backed up, the newly observed lidar point cloud data is added to the current existing map, and a map warning signal is given to promptly alert the staff and ensure the timeliness and safety during the mapping process. The most effective update method is selected according to different situations to improve work efficiency, save resources, and reduce time, manpower, and material costs.

[0065] Meanwhile, further considering the preservation and security of the file, when the size of the continuously updated map file exceeds the preset value (5MB in this embodiment), the map file is truncated for storage, and the steps of S1 - S5 are repeated for positioning and mapping. By limiting the size of the continuously updated map file, when it exceeds the preset value, the map file will be truncated for storage to ensure the correct storage of data and avoid the phenomenon of subsequent inability to open or save due to an overly large map file. And two corresponding map update output results are set, including: continuously updated sub - maps; a global overview map, which includes several sub - maps truncated due to exceeding the preset value size and data maps reserved due to warnings caused by possible positioning errors (corresponding to the third case of the map update method).

[0066] Embodiment 3

[0067] A system using the method for UAV positioning and mapping based on semantic information matching as described in any one of the above, including:

[0068] Lidar: used to obtain lidar point cloud data;

[0069] GPS: used to obtain the longitude, latitude, and altitude data of the UAV body;

[0070] Inertial navigation RTK: used to obtain the RPY angle data of the UAV body;

[0071] Translation change and rotation change module: used to calculate the translation change by resolving the data read by GPS and calculate the rotation change by resolving the data read by inertial navigation RTK;

[0072] Mapping initialization module: used to establish an initial environment map, using the data of GPS and inertial navigation RTK as the current positioning and taking the first - frame lidar point cloud data as the initial environment map;

[0073] Semantic segmentation module: used to perform semantic segmentation on each frame of lidar point cloud data and remove dynamic targets from the lidar point cloud data;

[0074] Semantic matching module: It is used to perform semantic matching on adjacent two-frame lidar point cloud data to obtain pose changes. Specifically, the NDT algorithm is adopted in this module for semantic matching. After the matching is completed, the backend uses a nonlinear optimization method to minimize the matching error function term;

[0075] Fusion positioning module: It is used to fuse the results in the semantic matching module and the results in the translation change and rotation change module. Specifically, the Kalman filter algorithm is adopted in this module to fuse the relationship between the two sets of data. The result in the semantic matching module is the pose change after semantic matching of adjacent two-frame lidar point cloud data; the results in the translation change and rotation change module are the translation change of adjacent two-frame longitude-latitude-altitude data and the rotation change of adjacent two-frame RPY angle data;

[0076] Output module: It is used to output the results. Specifically, the output results include the output of the positioning result and the output of the map result. The positioning result output directly outputs the positioning data obtained by the fusion positioning module; at the same time, the positioning data in the fusion positioning module is synchronously sent to the map update module. The map update module updates the map according to the positioning data on the basis of the map construction initialization module and then outputs the global map result.

[0077] The modules of the system of the present invention work independently while depending on each other, with a simple composition and high working efficiency; it can be used to efficiently and real-time provide accurate positioning and environmental maps, and draw a global map, so as to provide environmental information for subsequent work tasks. At the same time, the whole system has high robustness and anti-interference ability, and has functions such as software safety inspection and reset startup.

[0078] The embodiments described in the present invention are only used to describe the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. Without departing from the design idea of the present invention, various deformations and improvements made by those skilled in the art to the technical solutions of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for positioning and mapping of unmanned aerial vehicles based on semantic information matching, characterized in that: The following steps are involved: S1: Acquire multiple frames of laser point cloud data, and simultaneously acquire multiple frames of latitude, longitude, and height data and multiple frames of RPY angle data that are time synchronized with the multiple frames of laser point cloud data, obtain the initial coordinate system based on the first frame of latitude, longitude, and height data and the first frame of RPY angle data, and initialize and construct the map based on the first frame of laser point cloud data; S2: Perform semantic segmentation on each frame of laser point cloud data remaining in step S1 to remove dynamic targets in the laser point cloud data; S3: performing semantic matching on two adjacent frames of laser point cloud data in step S2 to obtain the position and posture changes of the two adjacent frames of laser point cloud data; S4: Calculate the translation change of the latitude and longitude data of two adjacent frames for the latitude and longitude data of the remaining frames, and calculate the rotation change of the RPY angle data of two adjacent frames for the RPY angle data of the remaining frames; S5: Fusing the translation change and rotation change obtained in step S4 with the posture change obtained in step S3 to obtain fused posture data, and then outputting the result; Before the result output in step S5, the map correction update is also included, which specifically includes: subtracting the fused pose data of the previous and next frames, and the result output is then updated after the difference satisfies any of the following three conditions: a: the displacement in other axes other than the height axis exceeds 1.0m; b: the displacement in the height axis exceeds 0.5m; c: the angular deviation in any axis exceeds 10°; The map correction update modes are divided into the following categories: When the matching degree between two adjacent frames of laser point cloud data in step S3 is higher than the set value, the map only updates the non-overlapping part; When the matching degree between two adjacent frames of laser point cloud data in step S3 is lower than the set value, but the fused posture data is within the positioning convergence range, the laser point cloud data after the current posture transformation is updated on the original map; When the fused pose data exceeds the positioning convergence range, the map will not be updated. When this happens for five consecutive frames of fused pose data, the existing map will be backed up, the newly observed laser point cloud data will be added to the existing map, and a map warning signal will be given. When the size of the continuously updated map file exceeds a preset value, the map file is stored and truncated, and steps S1 to S5 are repeated to perform positioning and mapping.

2. The method for positioning and mapping an unmanned aerial vehicle based on semantic information matching according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21: downsampling the laser point cloud data; S22: Target detection is performed on the downsampled laser point cloud data, and the targets are classified into two categories: static targets and dynamic targets, and the targets are labeled with classification labels accordingly; S23: Eliminate dynamic targets in the laser point cloud data in step S22.

3. The method for positioning and mapping an unmanned aerial vehicle based on semantic information matching according to claim 2, characterized in that: In step S22, the static targets are divided into man-made objects and natural environments. Different labels are set for different man-made objects, while no label is set for the natural environment.

4. A method for UAV positioning and mapping based on semantic information matching according to claim 2 or 3, characterized in that: The specific steps for removing dynamic targets in step S23 are as follows: when the center pose calculated for the same marked dynamic target exceeds the threshold in three consecutive frames, it is considered to be changing and is removed.

5. A method for UAV positioning and mapping based on semantic information matching according to claim 1, characterized in that: The map output results include two methods: continuously updated sub - maps; a global overview map, which includes several sub - maps generated by truncating maps larger than a preset size.

6. A system using the method for UAV positioning and mapping based on semantic information matching according to any one of claims 1-5, characterized in that: Including: LiDAR: used to obtain LiDAR point cloud data; GPS: used to obtain latitude, longitude, and altitude data; Inertial navigation RTK: used to obtain RPY angle data; Translation and rotation change module: used to calculate the translation change by resolving the data read by GPS and calculate the rotation change by resolving the data read by inertial navigation RTK; Mapping initialization module: used to establish an initial environment map; Semantic segmentation module: used to perform semantic segmentation on each frame of LiDAR point cloud data and remove dynamic objects in the LiDAR point cloud data; Semantic matching module: used to perform semantic matching on two adjacent frames of LiDAR point cloud data to obtain pose changes; Fusion positioning module: used to fuse the results in the semantic matching module with the results in the translation and rotation change module; Output module: used to output the results.

Images