High-precision map construction method for sky-air-ground integrated cross-country environment

Abstract

The invention belongs to the technical field of ground unmanned platforms, and particularly relates to a sky-air-ground integrated cross-country environment high-precision map construction method. Comprising the following steps: 1, acquiring a cross-country environment high-precision map; and 2, making a cross-country environment high-precision map. According to the invention, automatic extraction of elements is realized by combining an automatic extraction technology; in the aspect of map making, making specifications are provided, and the running efficiency of a high-precision geographic information system is improved while it is guaranteed that the cross-country maneuvering requirement of a ground unmanned platform in the cross-country environment is met; incremental data updating is carried out on the high-precision map based on a real-time mobile environment perception result of the ground unmanned platform so as to ensure the timeliness of the high-precision map. Through the method, the blank of a high-precision map technology in the field of automatic driving in the cross-country environment is filled up, and support is provided for improving the cross-country maneuverability of a ground unmanned platform.

Description

Technical Field

[0001] The present invention belongs to the technical field of ground unmanned platforms, and specifically relates to a method for constructing a high-precision map of an integrated "sky-air-ground" off-road environment. Background Art

[0002] At present, the autonomous maneuvering function of ground unmanned platforms in off-road environments is realized based on traditional autonomous navigation systems. Autonomous navigation systems usually include three layers of perception, planning and control, such as Figure 1 As shown. In the current research and application fields of autonomous navigation systems, environmental perception and positioning are considered to be one of the most challenging topics. Autonomous navigation systems generally rely on global satellite navigation systems, dynamic real-time differential methods, and inertial navigation systems to achieve positioning. However, navigation systems based on satellite communication technology are easily affected by terrain, forming multipath effects, and it is difficult to ensure the stability of positioning. At the same time, the commonly used sensors carried by ground unmanned platforms have performance boundary gaps, laser sensors cannot break through the detection of sand and dust, and high-resolution camera equipment cannot cope with severe weather. These are defects that artificial intelligence equipment is difficult to solve in the short term. Therefore, ground unmanned platforms face huge challenges in autonomous maneuvers in off-road environments where prior information is extremely lacking.

[0003] The above factors severely limit the use of ground unmanned platforms in off-road environments, and the emergence of high-precision maps provides a new way to solve the above problems. Figure 2 As shown in the figure, high-precision maps are the "neural network" of autonomous navigation systems. The preset digital information is not only a part of the perception system, but also an important component of the decision-making system. It serves the perception, decision-making, positioning and control of autonomous navigation and is the core technical support for solving large-scale fully autonomous maneuvers. With the support of high-precision maps, ground unmanned platforms can automatically search for passable areas in off-road environments, and then quickly and reasonably determine the passing trajectory, thereby improving the speed of autonomous driving.

[0004] In the field of autonomous driving in urban environments, the technology is relatively mature. In the development of high-precision map products, with the help of professional data collection vehicles, changes in the real world are collected, and the automated production tools based on deep learning automatically reduce noise, classify, and extract the collected data. High-precision data has efficiently covered more than 400,000 kilometers of highways across the country. In terms of intelligent updates of high-precision maps, relying on the MapLearning map learning system, a large amount of multi-source sensor data from crowdsourcing, OEM manufacturers, etc. is collected, processed, and integrated in real time, changes are automatically discovered and processed, and released in real time through the distribution platform, using the latest high-precision maps to escort the safety of autonomous navigation.

[0005] However, in the field of off-road autonomous driving, due to the limitations of passing performance and environmental adaptability, the method of collecting data for high-precision maps through professional collection vehicles and using a large number of end-side vehicles to obtain crowdsourced data is obviously not suitable for the mobility needs of off-road environments. In off-road environments, more attention is paid to unstructured environments under off-road conditions. Its typical characteristics are inconsistent environmental characteristics, lack of traffic rules, and complex road environments. The research on high-precision maps for off-road environments is still in its infancy, and there are still a series of challenges such as difficulty in data acquisition, high processing costs, and complex standardization issues.

[0006] Existing off-road environment maps are mainly series-scale topographic maps, which are mainly used to view geographical locations and topography in order to familiarize yourself with the environment. However, series-scale topographic maps are far inferior to high-precision maps in terms of accuracy and element richness, and are mainly used by personnel and cannot be directly used for ground unmanned platforms. The element categories contained in series-scale electronic maps are relatively detailed, mainly used in vector forms such as points, lines, and surfaces, and the accuracy is relatively low, with a general positioning accuracy of meters; while off-road environment high-precision maps contain two-dimensional map elements in vector form, channel three-dimensional point clouds, and cost maps in raster form, with higher positioning accuracy than series-scale electronic maps. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] The technical problem to be solved by the present invention is: how to provide a method for constructing a high-precision map of an integrated "sky-air-ground" off-road environment.

[0009] (II) Technical solution

[0010] In order to solve the above technical problems, the present invention provides a method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground", the method comprising:

[0011] Step 1: Collection of high-precision maps of off-road environments;

[0012] Step 2: Create a high-precision map of the off-road environment.

[0013] Wherein, the step 1 includes:

[0014] Step 1.1: Use satellite images to obtain a basic prior base map of the entire off-road environment;

[0015] Step 1.2: Based on the basic prior base map, the off-road environment is divided into range areas, and the route is designed in combination with the digital elevation model. The UAV environmental perception equipment is used to obtain three-dimensional point cloud data and high-resolution image data;

[0016] Step 1.3: When the ground unmanned platform is driving in an off-road environment, it obtains a high-precision three-dimensional point cloud of the driving channel based on the environmental perception sensors it carries, and forms drivable path data.

[0017] Among them, in the step 1.2, when the UAV is equipped with a real-time environment perception pod used as an environment perception device, the real-time point cloud completes the real-time collection, solution, fusion and precise matching of multiple sensor data during the flight data collection process through the control unit installed in the laser radar system, so as to obtain the three-dimensional point cloud data of the target object with precise positioning;

[0018] In this process, a multi-modal perception fusion technology for accurate 3D reconstruction of natural scenes was proposed, and a multi-modal fusion perception system was used to carry out accurate 3D reconstruction of the natural environment. The steps are as follows:

[0019] (1) Modeling and calibration of multimodal systems based on simultaneous estimation of external model structures; design a multimodal fusion perception system for off-road natural environment scanning and reconstruction operations. Before use, the sensors need to be unified in the multimodal coordinate system; to improve the accuracy of calibration, a joint observation model of the fusion system and the perceived environment is established, and the system is calibrated using the method of simultaneous estimation of external model structures;

[0020] (2) Multi-scale adaptive natural environment reconstruction based on point cloud and image fusion; the multimodal fusion perception system integrates a laser radar module with long-distance sparse point cloud-based precise ranging and a depth camera with close-range high-precision speckle imaging, so it is compatible with the functions of long-range and short-range 3D reconstruction in natural environments; through a multi-scale adaptive 3D natural environment reconstruction method based on point cloud and image fusion, a scale adaptive switching strategy is adopted with depth threshold as the panoramic scanning indicator; for long-range reconstruction, by setting thresholds for point cloud feature points, the external points in the corresponding image feature points are eliminated to complete the laser-assisted image real scene reconstruction;

[0021] (3) DFNN-based reconstruction hole repair and visual-inertial measurement fusion motion correction-based real-scene reconstruction effect optimization; these two real-scene reconstruction optimization methods improve the hole problem of point cloud and image fusion real-scene reconstruction; the generator network model composed of the encoder and decoder is designed by referring to the generative adversarial network, and the discriminator network model and the generative network model are mirror images of each other; in addition, in order to solve the problems of holes in the real-scene reconstruction results that are easily caused by the three-dimensional reconstruction method of multiple views of images and the cross-layering of the reconstruction effect caused by the lack of translation increment under complex scene conditions, the system motion estimation is corrected by jointly minimizing the pre-integration constraint of the inertial measurement unit and the visual reprojection error in the sliding window.

[0022] Among them, in the step 1.3, the ground unmanned platform relies on its own laser scanner, global navigation and positioning system, and high-precision inertial navigation system to quickly build high-precision map data; the position and orientation measured by the global navigation and positioning system and INS are used to align the laser radar point cloud; the laser radar real-time mapping and SLAM positioning technology are used to achieve laser ranging, path planning and positioning; by recording the time from the emission of a single laser signal to the reception of the energy reflected by the ground object, and based on the position and posture of the laser scanning system measured at the moment the signal is emitted, the three-dimensional coordinates of the ground object target are calculated, and a three-dimensional terrain map is drawn;

[0023] Among them, the SLAM positioning technology used is a SLAM algorithm based on graph optimization, which is mainly divided into two modules, the front end and the back end;

[0024] The front end is responsible for obtaining the attitude from the odometer and laser data, and the back end uses loop detection to construct closed-loop constraints and obtain the optimized attitude by minimizing the observation and estimation residuals.

[0025] In the front-end, the initial posture information is first obtained from the odometer, and then the posture after laser constraint is obtained through continuous frame matching of laser data; in continuous frame matching, there are ICP, NDT, and brute force matching algorithms; brute force matching is used as the front-end method, that is, laser scanning matching is used. During the movement of the ground unmanned platform, the relative movement of the ground unmanned platform is obtained by matching the laser changes of the two frames before and after; one of the key modules is the three-dimensional window traversal optimization; this is an exhaustive search algorithm, for the ground unmanned platform in planar motion, the posture is decomposed into three dimensions, namely the x-axis, y-axis, and angle axis; by constructing a three-layer for loop, the x-direction, y-direction, and rotation angle direction are traversed and searched respectively to find the optimal posture matching result;

[0026] The front end is a local estimate, that is, it can only estimate the posture change of the ground unmanned platform at the current moment relative to the previous moment. In fact, due to the error introduced by the estimation, as the range of motion expands, the error will gradually accumulate, leading to the final wrong result. To solve this problem, the graph optimization back end introduces closed-loop detection. By judging whether the ground unmanned platform has returned to a certain point it has passed, a closed-loop constraint is constructed. Through nonlinear optimization, the errors in the motion process are dispersed to each posture involved in the optimization, thereby eliminating error accumulation.

[0027] Wherein, in the step 2, the preparation of the high-precision map of the off-road environment includes a large-scale basic base map, a local high-precision map, a cost map and a high-precision channel map;

[0028] The step 2 comprises:

[0029] Step 2.1: Construction of large-scale basic base map and local high-precision map;

[0030] Step 2.2: Cost map construction;

[0031] Step 2.3: High-precision channel map construction.

[0032] Among them, in the step 2.1, the construction of the large-scale basic base map depends on satellite images, and feature extraction is performed on the basis of the images to form a large-scale basic map. At the same time, the satellite images have their own elevation data and can be used directly to form an elevation map; the local high-precision map relies on drone data collection, and high-precision images are formed through high-precision image stitching technology under large parallax, and further feature extraction is performed on this basis to further supplement the large-scale basic map. The drone data also has its own elevation data, which can be used directly to form a high-precision elevation map.

[0033] Among them, in step 2.1, the high-precision image stitching technology under large parallax is specifically:

[0034] Firstly, the image registration method based on deep features is improved for the registration between the target image and the reference image. Specifically, a pre-trained VGG19 network is used for feature extraction, and image registration is performed in two stages. In the first stage, a homography transformation matrix is ​​quickly estimated using terminal features and applied to the target image, so that the target image and the reference image are preliminarily aligned. In the second stage, the aligned target image is used to register with the reference image, and a coarse-to-fine strategy is used for hierarchical refinement matching, and the matching process is adaptive. The improved method achieves the same effect as the current mainstream registration algorithm, and has a higher registration efficiency per unit time, especially when the image feature texture is not obvious.

[0035] On this basis, the traditional method of finding the best stitching seam based on quality evaluation is improved; according to the characteristics of UAV aerial images, structural similarity, color difference, and texture complexity are comprehensively considered, and a new quality evaluation function is proposed to evaluate each pixel on the stitching seam. Then the difference cost is updated and the stitching seam is re-estimated based on the evaluation result. The estimation and evaluation process is repeated until the stitching seam tends to be stable and the iteration is stopped; the obtained stitching seam preferentially passes through roads, woodlands and other areas, which is more in line with human visual perception.

[0036] Wherein, in the step 2.2, the cost map includes the cost map of the UAV and the cost map of the unmanned vehicle; the construction of the cost map is based on the three-dimensional point cloud data of the UAV and the unmanned vehicle to generate the cost map of the area; the cost map used for the autonomous driving planning and obstacle avoidance function of the ground unmanned platform is a two-dimensional grid map, which presents the passability and obstacle distribution information in the area; wherein, the global cost grid map is a large-scale full-scene map, which is regularly maintained and covered based on the map of the executable area;

[0037] The three-dimensional point cloud data collected by drones and ground unmanned platforms is denoised to eliminate the interference of noise points, and then the ground is segmented to extract the ground data and construct the cost map, including scene partitioning, feature extraction and cost map construction. Finally, the cost map is further processed to extract the skeleton line.

[0038] Wherein, in step 2.3, the construction of the high-precision channel map includes:

[0039] The ground unmanned platform's rapid environmental perception and mapping uses laser SLAM mapping technology, combined with the laser radar and IMU sensors carried by the unmanned vehicle, to enable the ground unmanned platform to build high-precision point cloud data of the channel in real time during driving, forming a high-precision channel map.

[0040] Among them, in the step 2.3, a tightly coupled laser-inertial odometry method is used to obtain the globally consistent position and posture of the target by optimizing the LiDAR odometry factor, the IMU pre-integration factor and the closed-loop detection factor; frame-local map matching is used instead of frame-global map matching to improve the efficiency of frame map matching, thereby greatly improving the real-time performance of map construction.

[0041] (III) Beneficial effects

[0042] The purpose of the present invention is to provide a method for constructing a high-precision map of an off-road environment that is integrated with "sky-air-ground". The method combines the three methods of "sky-air-ground" to obtain high-precision three-dimensional data, and extracts key elements during data collection. To improve data collection efficiency, the present invention will combine automatic extraction technology to achieve automatic extraction of elements. In terms of map production, production specifications are proposed to ensure that the off-road mobility requirements of ground unmanned platforms in off-road environments are met while improving the operating efficiency of high-precision geographic information systems. Based on the real-time mobility environment perception results of ground unmanned platforms, incremental data updates are performed on high-precision maps to ensure the timeliness of high-precision maps. Through the present invention, the gap in high-precision map technology in the field of autonomous driving in off-road environments will be filled, providing support for improving the off-road mobility capabilities of ground unmanned platforms.

[0043] Compared with the prior art, the key technical points of the present invention include:

[0044] (1) The present invention innovatively proposes a theoretical model of high-precision maps for off-road environments. Based on the off-road mobility requirements of ground unmanned platforms, the present invention proposes the information that the theoretical model should contain, and further refines the map layers that support autonomous driving in off-road environments for ground unmanned platforms. This can provide a reference standard for the subsequent development of high-precision maps for off-road environments and guide the iteration of related technical research.

[0045] (2) The present invention designs a method for constructing a high-precision map of an off-road environment that is an integrated "sky-air-ground" environment from a system level, filling the gap in high-precision map technology in the field of off-road autonomous driving and providing support for improving the off-road maneuverability of ground unmanned platforms.

[0046] (3) The present invention proposes a precise 3D reconstruction technology for natural scenes based on multimodal perception fusion. The data accuracy of the system based on structure is improved by multimodal system modeling and calibration based on simultaneous estimation of external participating model structure, natural environment reconstruction is achieved at multiple scales based on point cloud and image fusion, reconstruction holes are repaired based on DFNN, and the real scene reconstruction effect is optimized based on visual-inertial measurement fusion motion correction. Through the combination of the above methods, the accuracy of 3D reconstruction based on drone point cloud data is improved.

[0047] (4) The present invention proposes a high-precision image stitching technology under large parallax. By improving the image registration method based on depth features, it is used for the registration between the target image and the reference image; combining structural similarity, color difference, and texture complexity, the traditional method of finding the best stitching seam based on quality evaluation is improved. The probability of stitching image artifacts and distortion is reduced.

[0048] (5) The present invention proposes an automatic extraction technology of relevant map elements based on images. Combined with commonly used deep learning algorithms, the accuracy of target recognition in automatic target extraction is improved by combining multiple technical means such as receptive field enhancement and positioning feature enhancement, further improving the efficiency of element extraction and reducing the workload of element extraction.

[0049] (6) The present invention forms a cost map by integrating the point cloud information of the UAV and the unmanned vehicle, and further extracts the passable skeleton line based on the cost map, thereby improving the efficiency of path planning.

[0050] (7) The present invention proposes a method for constructing a high-precision channel map based on a tightly coupled laser-inertial odometry method. The globally consistent position and posture of the target (the moving carrier of the measurement process) is obtained by optimizing the LiDAR odometry factor, the IMU pre-integration factor, and the closed-loop detection factor. Frame-local map matching is used instead of frame-global map matching to improve the efficiency of frame map matching, thereby greatly improving the real-time performance of map construction.

[0051] Compared with the prior art, the beneficial effects of the present invention include:

[0052] (1) Before the present invention was proposed, the concept of high-precision maps for off-road environments had not been mentioned. The present invention innovatively proposed a theoretical model of high-precision maps for off-road environments, which is conducive to the large-scale promotion and use of high-precision map technology in the field of ground unmanned platforms and promotes the sustainable development of high-precision maps for off-road environments.

[0053] (2) High-precision maps for off-road environments can make up for the shortcomings of traditional autonomous navigation artificial intelligence methods. In complex off-road scenarios, the harsh environment causes the on-board sensors to be unable to analyze the passable area information due to bad natural weather, sand and dust, magnetic field effects, signal interference, obstacles or sensor dirt. The lane feature information of high-precision maps can help the autonomous navigation system to more accurately identify the passable area and ensure the safe passage of ground unmanned platforms.

[0054] (3) High-precision map data for off-road environments is not affected by factors such as weather and light detection distance, and can break through the performance boundaries of conventional sensors to achieve beyond-visual-range perception. It is difficult to directly obtain the surrounding environment information of the current location of the ground unmanned platform by relying on the information collected in real time by sensor equipment, such as surface material, road width, up and down slopes, curved road surface slopes, and road curvature; this type of information can be obtained, recorded, and stored in high-precision map data. Road traffic information exists objectively and will not change in the short term, which can provide auxiliary decision-making for ground unmanned platforms. At the same time, high-precision maps can also identify various static objects based on vehicle-mounted sensors, and then match and calculate the obstacle image band with the static carrier recorded on the high-precision map to obtain the precise position and status of the ground unmanned platform, so that the ground unmanned platform can obtain an ultra-long-distance perspective and a more accurate position.

[0055] (4) The high-precision map data of the urban environment is obtained through crowdsourcing data from professional data collection vehicles and numerous passenger cars. Compared with the data collection method of the high-precision map data of the urban environment, the present invention proposes an integrated "sky-air-ground" data collection method, which does not need to rely on large-scale platform data and is suitable for the construction of high-precision maps of irregular road environments in the wild. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Schematic diagram of the hierarchical structure of the autonomous navigation system.

[0057] Figure 2 Schematic diagram of the role of high-precision maps in autonomous navigation systems.

[0058] Figure 3 Schematic diagram of the theoretical model of high-precision maps for off-road environments.

[0059] Figure 4 Schematic diagram for building an integrated high-precision map of "sky-air-ground".

[0060] Figure 5 This is a schematic diagram of the UAV subsystem technology route.

[0061] Figure 6 This is a schematic diagram of the three-dimensional point cloud and image effects.

[0062] Figure 7 Schematic diagram of the front end and back end.

[0063] Figure 8 Schematic diagram for diagram optimization.

[0064] Fig. 9 Schematic diagram of SLAM mapping effect for ground unmanned platform.

[0065] Fig.10 Schematic diagram of the overall framework of the recognition neural network.

[0066] Fig.11 This is a schematic diagram of the road extraction effect.

[0067] Fig.12 Schematic diagram of water extraction effect.

[0068] Fig.13 This is a schematic diagram of the bridge extraction effect.

[0069] Fig.14 Schematic diagram of the costmap generation process.

[0070] Fig.15 This is the basic flowchart of point cloud ground segmentation using morphological methods.

[0071] Fig.16 This is a schematic diagram of the cost map effect.

[0072] Fig.17 This is a schematic diagram of the fusion results of UAV + unmanned vehicle 3D laser point cloud data.

[0073] Fig.18 is the fusion cost map.

[0074] Fig.19 is the skeleton line cost map.

[0075] Fig. 20 This is the flow chart of the laser-inertial odometer factor algorithm.

[0076] Fig.21 It is the geometric feature structure diagram of point cloud data.

[0077] Fig. 22 It is the geometric partition map of point cloud data.

[0078] Fig.23 Schematic diagram of sign point extraction.

[0079] Fig.24 Schematic diagram of edge point projection.

[0080] Fig.25 Schematic diagram of edge point plane.

[0081] Fig.26 This is a diagram of a scan context example.

[0082] Fig. 27 Schematic diagram of point cloud denoising. DETAILED DESCRIPTION

[0083] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and examples.

[0084] In order to solve the above technical problems, the present invention provides a method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground", the method comprising:

[0085] Step 1: Collection of high-precision maps of off-road environments;

[0086] Step 2: Create a high-precision map of the off-road environment.

[0087] Wherein, the step 1 includes:

[0088] Step 1.1: Use satellite images to obtain a basic prior base map of the entire off-road environment;

[0089] Step 1.2: Based on the basic prior base map, the off-road environment is divided into range areas, and the route is designed in combination with the digital elevation model. The UAV environmental perception equipment is used to obtain three-dimensional point cloud data and high-resolution image data;

[0090] Step 1.3: When the ground unmanned platform is driving in an off-road environment, it obtains a high-precision three-dimensional point cloud of the driving channel based on the environmental perception sensors it carries, and forms drivable path data.

[0091] Among them, in the step 1.2, when the UAV is equipped with a real-time environment perception pod used as an environment perception device, the real-time point cloud completes the real-time collection, solution, fusion and precise matching of multiple sensor data during the flight data collection process through the control unit installed in the laser radar system, so as to obtain the three-dimensional point cloud data of the target object with precise positioning;

[0092] In this process, a multi-modal perception fusion technology for accurate 3D reconstruction of natural scenes was proposed. The multi-modal fusion perception system was used to carry out accurate 3D reconstruction of the natural environment. The steps are as follows:

[0093] (1) Modeling and calibration of multimodal systems based on simultaneous estimation of external model structures; design a multimodal fusion perception system for off-road natural environment scanning and reconstruction operations. Before use, the sensors need to be unified in the multimodal coordinate system; to improve the accuracy of calibration, a joint observation model of the fusion system and the perceived environment is established, and the system is calibrated using the method of simultaneous estimation of external model structures;

[0094] (2) Multi-scale adaptive natural environment reconstruction based on point cloud and image fusion; the multimodal fusion perception system integrates a laser radar module with long-distance sparse point cloud-based precise ranging and a depth camera with close-range high-precision speckle imaging, so it is compatible with the functions of long-range and short-range 3D reconstruction in natural environments; through a multi-scale adaptive 3D natural environment reconstruction method based on point cloud and image fusion, a scale adaptive switching strategy is adopted with depth threshold as the panoramic scanning indicator; for long-range reconstruction, by setting thresholds for point cloud feature points, the external points in the corresponding image feature points are eliminated to complete the laser-assisted image real scene reconstruction;

[0095] (3) DFNN-based reconstruction hole repair and visual-inertial measurement fusion motion correction-based real-scene reconstruction effect optimization; these two real-scene reconstruction optimization methods improve the hole problem of point cloud and image fusion real-scene reconstruction; the generator network model composed of the encoder and decoder is designed by referring to the generative adversarial network, and the discriminator network model and the generative network model are mirror images of each other; in addition, in order to solve the problems of holes in the real-scene reconstruction results that are easily caused by the three-dimensional reconstruction method of multiple views of images and the cross-layering of the reconstruction effect caused by the lack of translation increment under complex scene conditions, the system motion estimation is corrected by jointly minimizing the pre-integration constraint of the inertial measurement unit and the visual reprojection error in the sliding window.

[0096] Among them, in the step 1.3, the ground unmanned platform relies on its own laser scanner, global navigation and positioning system, and high-precision inertial navigation system to quickly build high-precision map data; the position and orientation measured by the global navigation and positioning system and INS are used to align the laser radar point cloud; the laser radar real-time mapping and SLAM positioning technology are used to achieve laser ranging, path planning and positioning; by recording the time from the emission of a single laser signal to the reception of the energy reflected by the ground object, and based on the position and posture of the laser scanning system measured at the moment the signal is emitted, the three-dimensional coordinates of the ground object target are calculated, and a three-dimensional terrain map is drawn;

[0097] Among them, the SLAM positioning technology used is a SLAM algorithm based on graph optimization, which is mainly divided into two modules, the front end and the back end;

[0098] The front end is responsible for obtaining the attitude from the odometer and laser data, and the back end uses loop detection to construct closed-loop constraints and obtain the optimized attitude by minimizing the observation and estimation residuals.

[0099] In the front-end, the initial posture information is first obtained from the odometer, and then the posture after laser constraint is obtained through continuous frame matching of laser data; in continuous frame matching, there are ICP, NDT, and brute force matching algorithms; brute force matching is used as the front-end method, that is, laser scanning matching is used. During the movement of the ground unmanned platform, the relative movement of the ground unmanned platform is obtained by matching the laser changes of the two frames before and after; one of the key modules is the three-dimensional window traversal optimization; this is an exhaustive search algorithm, for the ground unmanned platform in planar motion, the posture is decomposed into three dimensions, namely the x-axis, y-axis, and angle axis; by constructing a three-layer for loop, the x-direction, y-direction, and rotation angle direction are traversed and searched respectively to find the optimal posture matching result;

[0100] The front end is a local estimate, that is, it can only estimate the posture change of the ground unmanned platform at the current moment relative to the previous moment. In fact, due to the error introduced by the estimation, as the range of motion expands, the error will gradually accumulate, leading to the final wrong result. To solve this problem, the graph optimization back end introduces closed-loop detection. By judging whether the ground unmanned platform has returned to a certain point it has passed, a closed-loop constraint is constructed. Through nonlinear optimization, the errors in the motion process are dispersed to each posture involved in the optimization, thereby eliminating error accumulation.

[0101] Wherein, in the step 2, the preparation of the high-precision map of the off-road environment includes a large-scale basic base map, a local high-precision map, a cost map and a high-precision channel map;

[0102] The step 2 comprises:

[0103] Step 2.1: Construction of large-scale basic base map and local high-precision map;

[0104] Step 2.2: Cost map construction;

[0105] Step 2.3: High-precision channel map construction.

[0106] Among them, in the step 2.1, the construction of the large-scale basic base map depends on satellite images, and feature extraction is performed on the basis of the images to form a large-scale basic map. At the same time, the satellite images have their own elevation data and can be used directly to form an elevation map; the local high-precision map relies on drone data collection, and high-precision images are formed through high-precision image stitching technology under large parallax, and further feature extraction is performed on this basis to further supplement the large-scale basic map. The drone data also has its own elevation data, which can be used directly to form a high-precision elevation map.

[0107] Among them, in step 2.1, the high-precision image stitching technology under large parallax is specifically:

[0108] Firstly, the image registration method based on deep features is improved for the registration between the target image and the reference image. Specifically, a pre-trained VGG19 network is used for feature extraction, and image registration is performed in two stages. In the first stage, a homography transformation matrix is ​​quickly estimated using terminal features and applied to the target image, so that the target image and the reference image are preliminarily aligned. In the second stage, the aligned target image is used to register with the reference image, and a coarse-to-fine strategy is used for hierarchical refinement matching, and the matching process is adaptive. The improved method achieves the same effect as the current mainstream registration algorithm, and has a higher registration efficiency per unit time, especially when the image feature texture is not obvious.

[0109] On this basis, the traditional method of finding the best stitching seam based on quality evaluation is improved; according to the characteristics of UAV aerial images, structural similarity, color difference, and texture complexity are comprehensively considered, and a new quality evaluation function is proposed to evaluate each pixel on the stitching seam. Then the difference cost is updated and the stitching seam is re-estimated based on the evaluation result. The estimation and evaluation process is repeated until the stitching seam tends to be stable and the iteration is stopped; the obtained stitching seam preferentially passes through roads, woodlands and other areas, which is more in line with human visual perception.

[0110] Wherein, in the step 2.2, the cost map includes the cost map of the UAV and the cost map of the unmanned vehicle; the construction of the cost map is based on the three-dimensional point cloud data of the UAV and the unmanned vehicle to generate the cost map of the area; the cost map used for the autonomous driving planning and obstacle avoidance function of the ground unmanned platform is a two-dimensional grid map, which presents the passability and obstacle distribution information in the area; wherein, the global cost grid map is a large-scale full-scene map, which is regularly maintained and covered based on the map of the executable area;

[0111] The three-dimensional point cloud data collected by drones and ground unmanned platforms is denoised to eliminate the interference of noise points, and then the ground is segmented to extract the ground data and construct the cost map, including scene partitioning, feature extraction and cost map construction. Finally, the cost map is further processed to extract the skeleton line.

[0112] Wherein, in step 2.3, the construction of the high-precision channel map includes:

[0113] The ground unmanned platform's rapid environmental perception and mapping uses laser SLAM mapping technology, combined with the laser radar and IMU sensors carried by the unmanned vehicle, to enable the ground unmanned platform to build high-precision point cloud data of the channel in real time during driving, forming a high-precision channel map.

[0114] Among them, in the step 2.3, a tightly coupled laser-inertial odometry method is used to obtain the globally consistent position and posture of the target by optimizing the LiDAR odometry factor, the IMU pre-integration factor and the closed-loop detection factor; frame-local map matching is used instead of frame-global map matching to improve the efficiency of frame map matching, thereby greatly improving the real-time performance of map construction.

[0115] Example 1

[0116] In this embodiment, in the field of off-road environment maps, series scale maps are usually used, and there is no high-precision map specifically used for autonomous driving of unmanned platforms. For this reason, the present invention first proposes a theoretical model of high-precision maps for off-road environments. On the basis of the theoretical model of high-precision maps for urban environments, unnecessary information such as lane speed limits, highway toll information, and traffic restriction information are removed, and information specific to off-road environments is added to form model information that conforms to off-road environments, as shown in Table 1 below;

[0117] Table 1 Information included in the high-precision map of off-road environment

[0118]

[0119] Based on the information contained, a high-precision map base layer, a local high-precision map, a ground unmanned platform dedicated layer and a positioning layer are constructed to obtain a theoretical model of a high-precision map for off-road environments. Figure 3 shown.

[0120] Based on the theoretical model, the present invention provides a method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground". The technical solution process adopted is as follows: Figure 4 As shown. The present invention adopts space-based services + UAV environment perception system + ground platform environment perception system to form an integrated "sky-air-ground" data collection method to provide support for the construction method of high-precision maps of off-road environments. On this basis, the map information is further extracted to produce a high-precision map.

[0121] 1. Collection of high-precision maps for off-road environments

[0122] The off-road environment is complex, the terrain is undulating, and there are no roads in most areas. The vehicle-mounted mobile measurement system currently used to obtain high-precision maps of urban environments cannot obtain environmental data for the entire area. The present invention intends to use the following three methods to obtain high-precision map data for off-road environments: ① Use satellite images to obtain the basic prior base map of the entire area; ② Based on the satellite image base map, divide the off-road environment into large areas, combine the digital elevation model, design the route, and use drone environmental perception equipment to obtain three-dimensional point cloud data and high-resolution image data; ③ When the ground unmanned platform is driving in the off-road environment, based on the environmental perception sensors it carries, such as lidar, it obtains high-precision three-dimensional point clouds of the driving channel to form drivable path data.

[0123] 1.1 Basic prior base map of the off-road environment in the entire area based on space-based satellite images

[0124] Space-based services use satellite communications, satellite navigation and satellite remote sensing to achieve real-time observation of the Earth's surface terrain, obtain data on off-road maneuvering areas, and process them to complete the preliminary construction of a high-precision map of the off-road environment.

[0125] According to the requirements of autonomous maneuvering, coordinate satellites to collect prior map data of off-road areas, including remote sensing image data and high-resolution DEM / DSM data. In most conventional areas, such as forests and deserts, data processing is performed based on high-resolution remote sensing images and DEM data obtained by space-based satellites to extract two-dimensional map elements and create high-precision maps of off-road environments. In roadless areas of off-road environments, the timeliness of existing prior map data does not meet the requirements. It can be used as a prior basic map to integrate the laser point cloud and image data obtained by drones in real time to update and supplement the basic map data.

[0126] 1.2 High-precision data collection in rainy night environment based on air-based UAV environmental perception equipment

[0127] The UAV subsystem is mainly based on the UAV mobile platform carrying a real-time environmental perception pod to obtain high-precision 3D point cloud data and image data in off-road environments. Based on the real-time environmental perception pod, the precise 3D reconstruction technology of natural scenes based on multi-modal perception fusion is used to achieve 3D reconstruction of the mapping area.

[0128] As an important aerial mapping part of the high-precision map construction and navigation system for off-road environments, the UAV subsystem is designed to achieve aerial wide-area data collection, target recognition, real-time high-precision mapping, etc., and provide aerial environment perception data support for ground unmanned platforms.

[0129] The UAV subsystem collects environmental data and constructs 3D real-time point clouds through the real-time environment perception pod carried by the UAV, and realizes the acquisition of high-precision 3D point clouds and image data by combining the real-time solution processing of the supporting software. Based on the integrated technology of laser, image and high-precision inertial navigation, the real-time 3D reconstruction of the environment is realized, the 3D color point cloud data is solved on the UAV side, and the data is transmitted to the receiving end in the form of data packets through the communication link, and then high-precision data is generated and fused and displayed. The specific technical route design is as follows: Figure 5 As shown (the algorithms involved are relatively common and will not be expanded in detail).

[0130] When the drone is equipped with a real-time environmental perception pod, the real-time point cloud is completed through the control unit installed in the lidar system during the flight data collection process. It completes the real-time collection, solution, fusion and precise matching of multiple sensor data to obtain multi-information three-dimensional data for precise positioning of the target object.

[0131] Real-time and accurate 3D reconstruction of unstructured natural environments is a core issue in the off-road mobility application scenarios of ground unmanned platforms. However, the current problems are low system accuracy, poor reconstruction effect, and low real-time performance. To solve the above problems, the present invention proposes a natural scene accurate 3D reconstruction technology based on multimodal perception fusion. The multimodal fusion perception system is used to carry out accurate 3D reconstruction of the natural environment. The main steps are as follows:

[0132] (1) Multimodal system modeling and calibration based on simultaneous estimation of external participating motif structure. The present invention designs a multimodal fusion perception system for off-road natural environment scanning and reconstruction operations. Before use, the sensor needs to be unified into a multimodal coordinate system. In order to improve the accuracy of calibration, a joint observation model of the fusion system and the perceived environment is established, and the system is calibrated using the method of simultaneous estimation of external participating motif structure.

[0133] (2) Multi-scale adaptive natural environment reconstruction based on point cloud and image fusion. The multimodal system integrates a laser radar module capable of long-distance sparse point cloud-based precise ranging and a depth camera capable of close-range high-precision speckle imaging, so it is compatible with the functions of long-range and near-range three-dimensional reconstruction in natural environments. The present invention adopts a multi-scale adaptive three-dimensional natural environment reconstruction method based on point cloud and image fusion, and uses a scale adaptive switching strategy with a depth threshold as a panoramic scanning indicator. For long-range reconstruction, by setting a threshold for the point cloud feature points, the external points in the corresponding image feature points are eliminated to complete the laser-assisted image real-scene reconstruction.

[0134] (3) DFNN-based reconstruction hole repair and real-scene reconstruction effect optimization based on visual-inertial measurement fusion motion correction. These two real-scene reconstruction optimization methods improve the hole problem of point cloud and image fusion real-scene reconstruction. The generator network model composed of the encoder and decoder is designed based on the generative adversarial network, and the discriminator network model and the generative network model are mirror images of each other. In addition, in order to solve the problems of holes in the real-scene reconstruction results that are easily caused by the three-dimensional reconstruction method of multiple views of images and the cross-layering of the reconstruction effect caused by the lack of translation increment under complex scene conditions (such as corner areas), the system motion estimation is corrected by jointly minimizing the pre-integration constraint of the inertial measurement unit and the visual reprojection error in the sliding window.

[0135] The UAV subsystem obtains high-precision 3D point cloud data and image data through the real-time environment perception pod. Figure 6 shown.

[0136] 1.3 High-precision data collection of off-road environment based on ground-based unmanned platform environmental perception equipment

[0137] Ground unmanned platforms can quickly build high-precision map data by relying on their own laser scanners, global navigation and positioning systems, and high-precision inertial navigation systems (INS). The laser radar point cloud can be aligned using the position and orientation measured by the global navigation and positioning system and INS. The laser radar real-time mapping and positioning (SLAM) technology is used, which integrates laser ranging, inertial navigation (IMU) and other technologies to achieve laser ranging, path planning and positioning. By recording the time it takes for a single laser signal to be emitted and received from the energy reflected by the ground object, and based on the position and attitude of the laser scanning system measured at the moment the signal is emitted, the three-dimensional coordinates of the ground object target can be calculated and a three-dimensional terrain map can be drawn.

[0138] The SLAM algorithm used in the present invention is a SLAM algorithm based on graph optimization, which is mainly divided into two modules, the front end and the back end. Figure 7 shown.

[0139] The front end is responsible for obtaining the attitude from the odometer and laser data, and the back end uses loop detection to construct closed-loop constraints and obtain the optimized attitude by minimizing the observation and estimation residuals.

[0140] On the front end, the initial posture information is first obtained from the odometer, and then the posture after laser constraint is obtained through continuous frame matching of laser data. In continuous frame matching, there are algorithms such as ICP, NDT, and brute force matching. Brute force matching is used as the front-end method, that is, laser scanning matching is used. During the movement of the ground unmanned platform, the relative movement of the ground unmanned platform is obtained by matching the laser changes of the two frames before and after. One of the key modules is the three-dimensional window traversal optimization. This is an exhaustive search algorithm. For ground unmanned platforms with planar motion, the posture can be decomposed into three dimensions, namely the x-axis, y-axis, and angle axis. By constructing a three-layer for loop, the x-direction, y-direction, and rotation angle direction are traversed and searched respectively to find the optimal posture matching result.

[0141] The front end is a local estimate, that is, it can only estimate the posture change of the ground unmanned platform at the current moment relative to the previous moment. In fact, due to the error introduced by the estimation, as the range of motion expands, the error will gradually accumulate, leading to the final wrong result. To solve this problem, the graph optimization backend introduces loop closure detection (Loop Closure). By judging whether the ground unmanned platform has returned to a certain point that has been passed, a closed-loop constraint is constructed. Through nonlinear optimization, the error in the motion process is dispersed to each posture involved in the optimization, thereby eliminating error accumulation. The core of graph optimization is to construct an error function, such as Figure 8 shown.

[0142] In the figure, the ground unmanned platform moves from point i to point j. Based on the front end, we calculated the posture of the ground unmanned platform at point j, that is, x i , then at x i The coordinates of the coordinate system are At this point we obtain a laser observation and obtain x through laser frame matching. j In x i The coordinate is z ij . Assuming that the observed and actual motions are perfect, then In form, there are

[0143]

[0144] In practice, the exact x i and x j The above error should be minimized. This process constructs an unconstrained nonlinear optimization problem. Through laser SLAM technology, graph optimization technology and other means, the ground unmanned platform can quickly build high-precision map data through the sensors it carries. The collection effect is as follows Fig. 9 shown.

[0145] 2. Production of high-precision maps for off-road environments

[0146] Based on the collected high-precision map data of off-road environment, high-precision map of off-road environment is produced, which mainly includes large-scale basic base map, local high-precision map, cost map and high-precision channel map.

[0147] 2.1 Construction of large-scale basic base maps and local high-precision maps

[0148] The construction of large-scale basic base maps mainly relies on satellite images, and feature extraction is carried out on the basis of images to form large-scale basic maps. At the same time, satellite images have their own elevation data and can be used directly to form elevation maps. Local high-precision maps mainly rely on data collected by drones. High-precision images are formed through high-precision image stitching technology under large parallax, and further feature extraction is carried out on this basis to further supplement the large-scale basic maps. The data collected by drones also has its own elevation data, which can be used directly to form high-precision elevation maps.

[0149] (1) High-precision image stitching technology under large parallax

[0150] At present, most image stitching scenes are often accompanied by large parallax. If the large parallax is not handled properly, it will cause artifacts, distortion and other problems in the stitched image. To address the above defects, this solution has made innovations in image registration and image fusion, as follows:

[0151] First, the image registration method based on deep features is improved for the registration between the target image and the reference image. Specifically, a pre-trained VGG19 network is used for feature extraction, and image registration is performed in two stages. In the first stage, a homography transformation matrix is ​​quickly estimated using terminal features and applied to the target image, so that the target image and the reference image are preliminarily aligned. In the second stage, the aligned target image is used to register with the reference image, and a coarse-to-fine strategy is used for hierarchical refinement matching, and the matching process is adaptive. The improved method achieves the same effect as the current mainstream registration algorithm, and has a higher registration efficiency per unit time, especially when the image feature texture is not obvious.

[0152] On this basis, the traditional method of finding the best seam based on quality evaluation is improved. According to the characteristics of drone aerial images, a new quality evaluation function is proposed to evaluate each pixel on the seam, and then the difference cost is updated and the seam is re-estimated according to the evaluation result. The estimation and evaluation process is repeated until the seam tends to be stable and the iteration is stopped. The obtained seam preferentially passes through roads, woodlands and other areas, which is more in line with human visual perception.

[0153] (2) Automatic extraction technology of relevant map elements based on images

[0154] Through the recognition of neural networks, obstacles and inaccessible areas can be automatically identified and mapped. Image-based extraction of relevant map elements includes satellite images and drone images, and the extracted objects include roads, bridges, vegetation, buildings, water bodies, obstacles and other information. The deep learning method based on convolutional neural networks is used for target detection, training and inference, achieving high flexibility and robustness, such as Fig.10 shown.

[0155] The traditional feature extraction method is used to extract shallow features, and the neural network method is used to extract deep abstract semantic features, and then the features are fused to form a dimensional feature vector for matching. Finally, a combination of multiple technical means is used to match and identify the same target to improve the target recognition accuracy, mainly including the following categories.

[0156] a) Sample feature enhancement

[0157] Compared with target detection in ordinary scene images, targets in optical remote sensing images are smaller, with greater scale differences, more complex surrounding environments, and more common abnormal samples. Therefore, the number of "difficult samples" will be larger. Therefore, effectively extracting the features of "difficult samples" can improve the accuracy and recall rate of target detection at the same time. "Online difficult example parsing" is a special sample sampling method that can effectively mine the features of "difficult samples". Correct use of "online difficult example parsing" can also take the features of "difficult samples" into account in the network structure, thereby effectively improving the performance of the detector on the data set.

[0158] b) Improved receptive field

[0159] The reduction of feature receptive field will lead to the loss of global information, affecting the classification effect and thus the detection accuracy. The receptive field can be increased by adding "hole convolution" to the network structure to improve the detection accuracy and recall rate.

[0160] c) Enhanced positioning features

[0161] Introducing "online hard example parsing" and "atrous convolution" in deep learning methods solves the problems of a large number of "hard samples" in optical remote sensing images and poor receptive field when improving resolution. However, using only deep network features for target detection will inevitably lose a lot of effective positioning information, so it is necessary to introduce positioning information.

[0162] The use of multi-scale feature combination can effectively solve the problem of poor positioning of small-scale targets in optical remote sensing images. In practical applications, a more efficient multi-scale feature combination method can be used: using the characteristics of deep convolutional neural networks—the shallow network extracts primary or intermediate features such as edges and textures, while the deep network extracts high-level semantic features that are beyond human intuitive understanding for effective multi-scale feature combination. The use of multi-scale networks for feature combination can introduce shallow network information that is conducive to positioning while ensuring high-level semantic features. Through algorithm improvements, it is expected that the multi-scale feature combination method will be used to further improve the overall performance of detection.

[0163] d) Transfer Learning

[0164] Although the amount of remote sensing image data is huge, there are relatively few remote sensing image data with specific targets, and data annotation consumes a lot of manpower and material resources, and is relatively difficult to collect. In order to ensure the accuracy of target recognition, the transfer learning method is used to study how to effectively obtain high-precision recognition results in convolutional neural networks under small sample conditions.

[0165] e) Human-assisted recognition and interactive interpretation

[0166] Although the use of intelligent recognition methods for target recognition is widely used, in some cases the accuracy is limited and it is difficult to achieve the level of commercial application. It is still necessary to consider using artificial interpretation features to assist target recognition to enhance the reliability of the algorithm. In addition, when the current automatic or intelligent recognition algorithms cannot identify certain military facilities that have been severely damaged, it is still necessary to use artificial interpretation methods for identification.

[0167] Through the above method, two-dimensional map elements such as roads, bridges, vegetation, obstacles, buildings, water bodies, and geology are extracted to generate a high-precision basic map. Manual inspection is then carried out, and omissions and incorrect element classifications are supplemented and corrected manually to achieve the extraction of all map elements. The extraction results of some map elements are as follows: Figure 11-13 shown.

[0168] 2.2 Costmap

[0169] Costmaps mainly include UAV costmaps and unmanned vehicle costmaps. The costmap is constructed based on the three-dimensional point cloud data of UAVs and unmanned vehicles to generate costmaps of the region. The costmap used for autonomous driving planning, obstacle avoidance and other functions of ground unmanned platforms is a two-dimensional grid map that presents the passability and obstacle distribution information in the region. Among them, the global cost grid map is a large-scale full-scene map, which is regularly maintained and updated based on the usable area map.

[0170] The 3D point cloud data collected by the UAV and ground unmanned platform is denoised to remove the interference of noise points, and then the ground is segmented to extract the ground data and construct the cost map, including scene partitioning, feature extraction and cost map construction. Finally, the cost map is further processed to extract the skeleton line. The cost map generation process is as follows: Fig.14 .

[0171] 1) Point cloud denoising (Noise Remove)

[0172] The radius filter is used to filter out isolated noise points. The basic process is as follows:

[0173] Construct a KD tree for the point cloud.

[0174] Traverse the point cloud and for each point, find n points whose three-dimensional distance is less than R based on the KD tree.

[0175] For n neighboring points, if n is less than the neighboring point number threshold N, the currently traversed point is regarded as a noise point.

[0176] like Fig. 27 shown.

[0177] The removal of low (elevation) noise points is mainly based on elevation features. The basic process is as follows:

[0178] a) After filtering out isolated points, the point cloud is normalized in elevation, that is, the elevation of the point cloud is scaled to between [0, 1].

[0179] b) Project the point cloud after normalization of elevation onto the xOy plane to form a two-dimensional image with normalized elevation as the attribute.

[0180] c) Dilate the elevation image and erode the pixels at the low elevation points to obtain a new two-dimensional image, in which the pixel reference value is the normal ground point elevation.

[0181] d) Traverse the point cloud and filter out the points whose elevation is significantly lower than the corresponding two-dimensional elevation image as low (elevation) noise points.

[0182] 2) Ground segmentation

[0183] Considering the general scene of point cloud, the ground segmentation here uses the method of progressive morphological filtering. After preprocessing the original point cloud (low point noise removal and isolated point noise removal), a 2D grid with a linear gradient size is established based on the morphological filtering method. According to the height difference distance from the point to the lowest point of the local grid, the non-ground points are filtered in turn to remove various non-ground point clouds from low to high, including pedestrians, vehicles, roadside traffic signs, roadside trees and buildings, etc. The ground point cloud left after filtering mainly includes road surface point cloud and roadside ground point cloud and some noise points that have not been completely filtered. Larger grids are mainly used to filter large-scale objects, such as buildings on the roadside, and smaller grids are mainly used to remove objects with insignificant elevation, such as fire hydrants and flower beds. This method has a small amount of computation and can filter out obstacles of different sizes in the vehicle-mounted point cloud road scene, which can better meet the needs of ground segmentation. The basic morphological method process:

[0184] a) Grid the point cloud data based on the lowest point elevation;

[0185] b) Perform corrosion operation on the lowest point grid with a certain window size w; compare the elevation values ​​of each grid unit in the window by sliding the window, and replace the elevation of the current window center grid with the lowest elevation, i.e. the elevation after corrosion;

[0186] c) Dilation operation: using a window of the same size, during the sliding process, the elevation of the center grid of the current window is replaced by the maximum elevation, i.e. the elevation after dilation;

[0187] d) The difference between the expanded elevation and the original elevation value is calculated. If the difference Δh is less than a certain threshold H, the current point is determined to be a ground point;

[0188] e) Increase the window size, perform erosion and dilation operations again, and add the ground point to the ground point set. Repeat the above process until the window size w is larger than the preset threshold W.

[0189] f) Get the final ground point set.

[0190] Note: As the window size increases, the size of the elevation threshold H will also change to ensure that the basic terrain tolerance is retained under a larger window. Generally speaking, if the window size increases linearly, the elevation threshold H will also increase linearly. That is,

[0191] H=αW+β

[0192] The basic process of ground point segmentation is as follows Fig.15 :

[0193] Note: If DEM (elevation map) or ground contour lines are used as input, it is not necessary to perform ground segmentation at the algorithm level. You only need to use DEM (elevation map) as a reference to perform a simple elevation difference judgment.

[0194] 3) Driving area segmentation

[0195] The main premise of road surface extraction is that there are relatively clear elevation differences on the roadside, such as curbs, flower beds, etc. Based on the trajectory file of the post-processed point cloud, the location information of the unmanned vehicle when driving is loaded as the initial point of the road surface. Multiple regional growth strategies are used, with the initial point set as the seed point, and the distance is unlimited in the x direction of the world coordinate. If there is no significant difference in elevation between the lowest point in the growth direction and the current direction, the road surface point set is added; the distance-restricted regional growth is performed in the y direction of the world coordinate, and the maximum distance of the growth is limited. If there is no significant difference in elevation between the lowest point in the growth direction and the current direction within a certain distance, the road surface point set is added. The growth in the x direction is mainly used to generate the main road area, and the growth in the y direction is mainly used to deal with the missing road surface caused by the occlusion of some vehicles. This method can meet the mapping requirements in the extraction of drivable areas in the main road scenes.

[0196] The basic flow of this method is as follows:

[0197] a) Load the spatial three-dimensional coordinates of the driving trajectory points;

[0198] b) Based on the ground segmentation, a grid is constructed with ground points;

[0199] c) Project the vehicle trajectory points to the corresponding cells of the grid and mark them as initial seed points;

[0200] d) Based on the seed point, the region is grown perpendicular to the driving direction of the trajectory, and the grids with a height difference Δh less than the threshold H in the adjacent cells are included in the pavement unit set Gcell;

[0201] e) For the pavement unit set, regional growth is performed in the parallel driving direction, and the grids with the height difference Δh less than the threshold H among the adjacent cells are included in the pavement unit set Gcell;

[0202] f) Traverse all the road surface units of the grid, judge all the points in the grid, and take the points with a height difference less than the lowest point of the grid unit as road surface points.

[0203] Note: The initial trajectory point frequency is relatively high, and it can be thinned to a certain extent according to the driving speed and grid cell size, which will not affect the result of regional growth.

[0204] The resulting cost map is stored in raster form, with pixel values ​​representing the passability of the driverless vehicle, 0-128 representing passable, and 129-255 representing impassable (different pixel values ​​represent different obstacles). Fig.16 As shown:

[0205] After the features at the above levels are processed, the data can be integrated in the same coordinate system. Since high-precision maps are mainly used for autonomous driving of unmanned vehicles and are not maps presented to operators, they are relatively abstract. The following effects are only for display purposes and cannot represent the entire fused map information.

[0206] Data fusion can be performed for aerial drone point cloud data and ground unmanned vehicle point cloud data, such as Fig.17 shown.

[0207] Based on the fused point cloud, the generated cost map is as follows Fig.18 shown.

[0208] Based on the cost map, the passable skeleton line is automatically extracted through continuous splicing of the passable area, and the path planning efficiency can be improved by an order of magnitude, such as Fig.19 shown.

[0209] 2.3 High-precision channel diagram

[0210] The ground unmanned platform's rapid environmental perception and mapping uses laser SLAM mapping technology, combined with the laser radar and IMU sensors carried by the unmanned vehicle, to enable the ground unmanned platform to build high-precision point cloud data of the channel in real time during driving, forming a high-precision channel map.

[0211] The present invention uses a tightly coupled laser-inertial odometer method to optimize the LiDAR odometer factor, IMU pre-integration factor and closed-loop detection factor to obtain the globally consistent position and posture of the target (the moving carrier of the measurement process). Frame-local map matching is used instead of frame-global map matching to improve the efficiency of frame map matching, thereby greatly improving the real-time performance of map construction.

[0212] The process structure of the tightly coupled laser-inertial odometry calculation method is as follows: Fig. 20 shown.

[0213] The figure above shows the factor graph of the algorithm. There are three main factors in the figure. The first is the IMU pre-integration factor (orange), which is obtained by integrating the IMU measurements between two adjacent keyframes. The second is the laser odometry factor (green), which is obtained by matching the frame graph between each keyframe and the previous n keyframes. The third is the loop detection factor (black), which is obtained by matching the frame graph between each keyframe and the 2m+1 keyframes that are adjacent in time sequence to the candidate loop keyframe.

[0214] ①LiDAR odometer factor

[0215] In the present invention, only the features of key frames are used, and the features of non-key frames are all discarded. The feature extraction method of point cloud feature extraction is used to extract the features of the i-th key frame. The former is an edge feature, and the latter is a plane feature. i+1 When it arrives, the feature set of the previous n+1 key frames {F i-n , ..., F i} and pose estimation {T i-n , ..., T i}Build local map M i ,in

[0216]

[0217] and is the feature of the i-th key frame transformed into the world coordinate system. and is the local edge feature map and local plane feature map in the world coordinate system. Then, by matching and as well as and Establish point-line and point-plane distance constraints. Optimize T by minimizing all constraints. i+1 This technique uses the combined point cloud of the past n+1 keyframes instead of the global point cloud and the latest frame for matching. The distance between a feature and its associated edge or plane can be calculated as follows:

[0218]

[0219] Where k, u, v, w are the indices of the features in their corresponding sets. One edge feature and is The two points that constitute the corresponding edge feature in A plane feature in and is The three points that constitute the corresponding plane features. Then the Gauss-Newton method is used to solve the optimal transformation and minimize the residual model as shown below:

[0220]

[0221] Finally, we can get the state x i and x i+1 Relative conversion between ΔT i，i+1 , which is the radar odometry factor connecting the two poses, as shown below:

[0222]

[0223] Note that there is an alternative method to obtain ΔTi，i+1 , this method is to convert the sub-keyframe to x i coordinate system. In other words, matching F i+1 To the voxel map is the map that represents the x i In this way, the real transformation ΔT i，i+1 Can be obtained directly.

[0224] Laser odometry uses feature points to match point cloud data of adjacent frames. Laser SLAM uses feature point-based correction to match input point cloud data with the map. There are two types of feature points and corresponding feature areas: (1) using edge points and edge lines where edge points are located; (2) using plane points and planes where plane points are located.

[0225] The following methods are mainly used for feature point extraction:

[0226] S is the point i in the kth frame point cloud data A set of continuous points nearby, the following expression means:

[0227]

[0228] The type of point i is determined by the modulus of the normalized vector sum of point i and the midpoint of S. If the modulus is large, it is an edge point, and if the modulus is small, it is a plane point. |S| represents the number of points in S. Fig.21 As shown, the modulus of the vector sum of edge points is generally large, and the vector sum is not a zero vector, while the modulus of the vector sum of the corresponding plane points is generally small, and the vector sum is a zero vector.

[0229] The point cloud data obtained from each scan is evenly divided into four regions, each of which contains at most 2 edge points and 4 plane points. Unreliable points need to be discarded, such as Fig. 22 Point 1 in (a) is located in a plane parallel to the laser beam; Fig. 22 The point 1 in (b) is actually in a plane feature. It may be regarded as an edge point due to occlusion. Therefore, it is not a true edge point and may not be observed again in adjacent frame data. These two types of points need to be excluded. There are also outliers: the distance between a point and its two adjacent points on the left and right is too large.

[0230] Therefore, the following conditions need to be met when selecting feature points:

[0231] First, the number of feature points meets the maximum value of the sub-region. The number of feature points in each sub-region is certain to prevent too many feature points. Second, the points around the feature points cannot be selected again. The number of feature points in each sub-region is relatively small, which further prevents the concentrated distribution of feature points. Third, the feature points are plane points that are not parallel to the laser beam, plane points that are not obscured, and outliers. These are all unreliable point cloud data and cannot be obtained in two consecutive frames of point cloud data and cannot be used as feature points.

[0232] Fig.23 The edge points (yellow) and plane points (red) found are shown in the figure. Since the lidar is in motion, there is still point cloud distortion at this time, which has not been corrected and cannot be corrected for the time being.

[0233] After extracting the feature points, we need to further find the relationship between the feature points, otherwise the feature point extraction loses its meaning. For edge points, we find the relationship between the edge points and the straight line where the edge points are located. For plane points, we find the relationship between the plane points and the plane. Let P k The starting timestamp is t k The end timestamp is t k+1 After distortion correction, P k Reproject to t k+1 Get P k When estimating the LiDAR motion, the reprojected point cloud data (after distortion correction) is used. k And the new point cloud data (not corrected for distortion) P k+1 .like Fig.24 shown.

[0234] E k+1 and H k+1 Respectively expressed in P k+1 (without distortion correction) to find the edge points and plane points. Correspondingly, we need to k Find the k+1 The edge line corresponding to the edge point in the middle and H k+1 The plane corresponding to the midplane point.

[0235] Each time the iterative calculation is performed, E k+1 and H k+1 The feature points in are reprojected to the start of k+1 sweeps t k+1 (This is because the only reference we have is the previous frame rectified and reprojected to t k+1 Data P k .),use and Indicates that the reprojection process is to the k+1th sweep start t k+1 A set of points. and For each point in P k Find the nearest neighbor point (their timestamps are all t k+1 , so a match can be made).

[0236] Edge points and edge lines: Fig.25 In (a), i is A point in P k In the example, j is the nearest neighbor point of i, and l is the nearest neighbor point of i in two adjacent scans of j, that is, j and l are not points obtained by the same scan (a scan can only contain one point in the edge line, otherwise it is not an edge line. The purpose of two adjacent scans is to prevent excessive motion distortion from causing incorrect edge line feature extraction. Due to structural problems, the difference in the timestamps corresponding to the same azimuth angle of the laser used in different scans is still quite large, approximately one scan cycle). Therefore, j and l constitute the edge line corresponding to i, and they are both P k The edge points in the set i. The points i, j, and l belong to j and l belong to P k , and in different scan lines.

[0237] Plane points and planes: Fig.25 In (b), i is A point in In order to determine a plane, at least three points that are not on the same line are required. l is the point closest to i in the same scan as j, and m is the point closest to i in two adjacent scans of j, that is, j and m are not points obtained from the same scan, and j and l are points from the same scan. j, l and m constitute the plane corresponding to i, and they are all P k Points in the plane.

[0238] Points i, j, l, and m belong to the following sets: i belongs to j, l and m belong to The corresponding scan has been analyzed before.

[0239] According to the matching principle, an optimization problem can be constructed: solving the transformation relationship T so that the distance between edge points and edge lines is the shortest, and the distance between plane points and planes is the shortest.

[0240] The distance between a point and a line is calculated as follows: The cross product represents the area of ​​the parallelogram composed of vectors, area / side=height. The height here is the distance. The calculation formula is as follows:

[0241]

[0242] Represents the point in the k+1th frame of data reprojected during the iteration, and the timestamp is Transform to the beginning of the k+1th frame data t k+1 , a uniform motion model was used.

[0243] It represents the corrected k-th frame data. After reprojection, the timestamps of all points are the end of the k-th frame data. k+1 .

[0244] For point and plane distances we have:

[0245]

[0246] The cross product represents the area of ​​the parallelogram composed of vectors, and the cross product plus the dot product represents the volume of the solid, volume / area=height. The height here is the distance.

[0247] The formula is as follows, Represents the point in the k+1th frame of data reprojected during the iteration, and the timestamp is Transform to the beginning of the k+1th frame data A uniform motion model was used.

[0248] It represents the corrected k-th frame data. After reprojection, the timestamps of all points are the end of the k-th frame data. k+1 .

[0249] Ideally, the distances are all 0, that is, edge points are on the edge line, and plane points are on the plane. But this is impossible. You can only find a minimum value that is not 0.

[0250] ②IMU pre-integration factor

[0251] The essence of the IMU pre-integration factor is the motion posture of the moving object, including the direction of motion, speed of motion, etc. The above data can be directly obtained through the IMU device. The IMU device is mainly a nine-axis sensor composed of a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer. The nine-axis sensor outputs acceleration, angular velocity, and magnetic declination. Through the attitude fusion algorithm, the acceleration, angular velocity, and magnetic declination are fused and calculated to obtain the position attitude data.

[0252] The angular velocity and acceleration measurements from the IMU are defined as follows:

[0253]

[0254] In the formula and is the measurement value of IMU at time t and in B coordinate system. t and is subject to a slowly varying bias b t and white noise n t Influence. EW t is the rotation matrix from coordinate system W to B. g is the gravity vector fixed in the W coordinate system.

[0255] Now we can use the IMU measurements to infer the target's motion. The position, attitude, and velocity of the target at time t+Δt are calculated as shown in the following formulas:

[0256]

[0257] In the formula And it is assumed that the angular velocity and acceleration of the base B are constant during the above integration process.

[0258] The IMU pre-integration method is used to obtain the relative motion of the body in adjacent time stamps. The pre-integrated measurement value ΔV between time i and j is ij ,ΔP ij and ΔR ij It can be calculated according to the following formula:

[0259]

[0260] In addition to effectiveness, applying IMU pre-integration also provides a factor graph constraint - IMU pre-integration factor. The IMU bias is jointly optimized in the factor graph along with the radar odometry factor.

[0261] ③ Closed-loop detection factor

[0262] Closed-loop detection refers to the recognition of the target having reached a certain scene, so that the constructed map has closed-loop differential capabilities. If only the association between adjacent timestamps is considered, then the errors generated previously will inevitably accumulate to the next moment, causing cumulative errors in the entire SLAM algorithm process. The results of long-term estimates will be unreliable, or in other words, it will be impossible to build a globally consistent trajectory and map. In simple terms, during the scanning process of the target, for the places or feature points that have been scanned before, when building the map, you can realize that this place, this feature point, is where "I" have been. Then match the map generated at this moment with the map just generated.

[0263] The closed loop detection factor can provide some constraints with longer time intervals in addition to adjacent frames. This is because it can detect that the device has passed the same place and collected similar data. The key to closed loop detection is how to effectively detect that the device has passed the same place.

[0264] Closed loop detection is a difficult problem. If the closed loop detection is successful, it can significantly reduce the cumulative error and help the target to perform obstacle avoidance navigation more accurately and quickly. However, incorrect detection results may make the map construction error larger and cause the map construction to fail. Therefore, closed loop detection is very necessary for large-area and large-scene map construction.

[0265] Traditional laser SLAM closed-loop detection mainly uses the ICP algorithm (iterative closest point) or Bag-of-Words (BOW) to match and identify scene point cloud data. The former compares the spatial position distribution of point clouds, and the latter compares the spatial features of point clouds, and combines feature extraction, feature encoding, feature classification and matching algorithms to calculate scene similarity to achieve closed-loop detection. However, the ICP algorithm has low computational efficiency, and the feature extraction, encoding, and classification of the bag-of-words technology are usually based on artificially constructed feature spaces, and its adaptability is limited. The constructed closed-loop detection method has low accuracy and poor reliability, and cannot meet the high-precision laser SLAM positioning and composition requirements.

[0266] Compared with visual loop detection, loop detection of 3D point clouds is more difficult. The main reason is that 3D point clouds lack color information, texture information, etc., and cannot extract the unique features of traditional images (ORB, SIFT, etc.). If the point cloud data is not preprocessed, only geometric matching can be performed, which is very time-consuming. This project uses the scan context dimensionality reduction method to reduce the 3D point cloud to a 2D image. The 2D image retains the depth information representation. Through a method similar to loop detection for color images, feature extraction and description are also performed on the reduced 2D point cloud.

[0267] Dimensionality reduction methods such as Fig.26 As shown in the figure, in the final two-dimensional graph, the horizontal axis represents the annular dimension, the vertical axis represents the radial dimension, and the pixel values ​​in the matrix are taken from the corresponding annular and radial intersection areas. The image after dimensionality reduction is intuitively felt like a "bumpy" mountain map, which better preserves the geometric structure information.

[0268] Due to the different radar perspectives, that is, when the radar rotates a certain angle at the same location, the value of the column vector remains unchanged, but there will be an offset. The behavior of the row vector is that the order of the elements in the vector will change, but the row vector will not be offset. In order to solve the problem of column vector offset, the similarity score uses the minimum distance after adjusting the column order as the similarity.

[0269] After the closed-loop detection is completed, the error within each closed-loop is differentially processed to reduce the overall error of mapping and improve data accuracy.

[0270] ④Point cloud data motion compensation

[0271] Since the laser radar body rotates in the vertical plane, the point cloud data output by the radar must be motion compensated in real time, and the point cloud data collected when the radar rotates must be transformed into the geodetic coordinate system. The angle measurement unit outputs angle information to perform real-time motion compensation on the point cloud. Motion compensation is mainly based on the characteristics of the motor to compensate for point cloud motion. Assuming the motor speed is N and its reduction ratio is k, the relative speed of the coaxial radar to the carrier can be obtained as n=N×k (hereinafter referred to as the revolution speed). The derivative of the radar revolution speed with respect to time can be obtained to obtain the number of revolution angles per microsecond θ. After the starting time of the motor is determined, the time of each point in the radar is obtained relative to the motor zero position time t, and the relative relationship of any radar point cloud with respect to the carrier coordinate system can be obtained. In the static case, the point only has a rotation transformation relationship between the carrier coordinate system, and its rotation matrix is ​​shown in the following formula.

[0272]

[0273] The time when any point of the laser radar is clouded can be obtained through the time when the laser radar UDP data block is released and the release order and release cycle of each line radar.

[0274] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for constructing a high-precision map of an integrated "sky-air-ground" off-road environment, characterized in that: The method comprises: Step 1: Collection of high-precision maps of off-road environments; Step 2: Create a high-precision map of the off-road environment.

2. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 1, characterized in that: The step 1 includes: Step 1.1: Use satellite images to obtain a basic prior base map of the entire off-road environment; Step 1.2: Based on the basic prior base map, the off-road environment is divided into range areas, and the route is designed in combination with the digital elevation model. The UAV environmental perception equipment is used to obtain three-dimensional point cloud data and high-resolution image data; Step 1.3: When the ground unmanned platform is driving in an off-road environment, it obtains a high-precision three-dimensional point cloud of the driving channel based on the environmental perception sensors it carries, and forms drivable path data.

3. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 2, characterized in that: In step 1.2, when the drone is equipped with a real-time environment perception pod used as an environment perception device, the real-time point cloud completes the real-time collection, solution, fusion and precise matching of multiple sensor data during the flight data collection process through a control unit installed in the laser radar system to obtain three-dimensional point cloud data for precise positioning of the target object; In this process, a multi-modal perception fusion technology for accurate 3D reconstruction of natural scenes was proposed, and a multi-modal fusion perception system was used to carry out accurate 3D reconstruction of the natural environment. The steps are as follows: (1) Modeling and calibration of multimodal systems based on simultaneous estimation of external model structures; design a multimodal fusion perception system for off-road natural environment scanning and reconstruction operations. Before use, the sensors need to be unified in the multimodal coordinate system; to improve the accuracy of calibration, a joint observation model of the fusion system and the perceived environment is established, and the system is calibrated using the method of simultaneous estimation of external model structures; (2) Multi-scale adaptive natural environment reconstruction based on point cloud and image fusion; the multimodal fusion perception system integrates a laser radar module with long-distance sparse point cloud-based precise ranging and a depth camera with close-range high-precision speckle imaging, so it is compatible with the functions of long-range and short-range 3D reconstruction in natural environments; through a multi-scale adaptive 3D natural environment reconstruction method based on point cloud and image fusion, a scale adaptive switching strategy is adopted with depth threshold as the panoramic scanning indicator; for long-range reconstruction, by setting thresholds for point cloud feature points, the external points in the corresponding image feature points are eliminated to complete the laser-assisted image real scene reconstruction; (3) DFNN-based reconstruction hole repair and visual-inertial measurement fusion motion correction-based real-scene reconstruction effect optimization; these two real-scene reconstruction optimization methods improve the hole problem of point cloud and image fusion real-scene reconstruction; the generator network model composed of the encoder and decoder is designed by referring to the generative adversarial network, and the discriminator network model and the generative network model are mirror images of each other; in addition, in order to solve the problems of holes in the real-scene reconstruction results that are easily caused by the three-dimensional reconstruction method of multiple views of images and the cross-layering of the reconstruction effect caused by the lack of translation increment under complex scene conditions, the system motion estimation is corrected by jointly minimizing the pre-integration constraint of the inertial measurement unit and the visual reprojection error in the sliding window.

4. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 3, characterized in that: In step 1.3, the ground unmanned platform relies on its own laser scanner, global navigation and positioning system, and high-precision inertial navigation system to quickly build high-precision map data; uses the position and orientation measured by the global navigation and positioning system and INS to align the laser radar point cloud; uses laser radar real-time mapping and SLAM positioning technology to achieve laser ranging, path planning and positioning; by recording the time from the emission of a single laser signal to the reception of energy reflected by the ground object, and based on the position and posture of the laser scanning system measured at the moment the signal is emitted, the three-dimensional coordinates of the ground object target are calculated, and a three-dimensional terrain map is drawn; Among them, the SLAM positioning technology used is a SLAM algorithm based on graph optimization, which is mainly divided into two modules, the front end and the back end; The front end is responsible for obtaining the attitude from the odometer and laser data, and the back end uses loop detection to construct closed-loop constraints and obtain the optimized attitude by minimizing the observation and estimation residuals. In the front-end, the initial posture information is first obtained from the odometer, and then the posture after laser constraint is obtained through continuous frame matching of laser data; in continuous frame matching, there are ICP, NDT, and brute force matching algorithms; brute force matching is used as the front-end method, that is, laser scanning matching is used. During the movement of the ground unmanned platform, the relative movement of the ground unmanned platform is obtained by matching the laser changes of the two frames before and after; one of the key modules is the three-dimensional window traversal optimization; this is an exhaustive search algorithm, for the ground unmanned platform in planar motion, the posture is decomposed into three dimensions, namely the x-axis, y-axis, and angle axis; by constructing a three-layer for loop, the x-direction, y-direction, and rotation angle direction are traversed and searched respectively to find the optimal posture matching result; The front end is a local estimate, that is, it can only estimate the posture change of the ground unmanned platform at the current moment relative to the previous moment. In fact, due to the error introduced by the estimation, as the range of motion expands, the error will gradually accumulate, leading to the final wrong result. To solve this problem, the graph optimization back end introduces closed-loop detection. By judging whether the ground unmanned platform has returned to a certain point it has passed, a closed-loop constraint is constructed. Through nonlinear optimization, the errors in the motion process are dispersed to each posture involved in the optimization, thereby eliminating error accumulation.

5. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 4, characterized in that: In step 2, the preparation of the high-precision map of the off-road environment includes a large-scale basic base map, a local high-precision map, a cost map and a high-precision channel map; The step 2 comprises: Step 2.1: Construction of large-scale basic base map and local high-precision map; Step 2.2: Cost map construction; Step 2.3: High-precision channel map construction.

6. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 5, characterized in that: In the step 2.1, the construction of the large-scale basic base map depends on satellite images, and elements are extracted based on the images to form a large-scale basic map. At the same time, the satellite images have their own elevation data and can be used directly to form an elevation map; the local high-precision map relies on drone data collection, and high-precision images are formed through high-precision image stitching technology under large parallax, and on this basis, elements are further extracted to further supplement the large-scale basic map. The drone data also has its own elevation data, which can be used directly to form a high-precision elevation map.

7. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 1, characterized in that: In step 2.1, the high-precision image stitching technology under large parallax is specifically as follows: Firstly, an image registration method based on deep features is improved for registration between target images and reference images. Specifically, a pre-trained VGG19 network is used for feature extraction, and image registration is performed in two stages. In the first stage, a homography transformation matrix is ​​quickly estimated using terminal features and applied to the target image, so that the target image and the reference image are preliminarily aligned. In the second stage, the aligned target image is used to register with the reference image, and a coarse-to-fine strategy is used to perform hierarchical refinement matching, and the matching process is adaptive; the improved method achieves the same effect as the current mainstream registration algorithm, and has a higher registration efficiency per unit time, especially when the image feature texture is not obvious; On this basis, the traditional method of finding the best stitching seam based on quality evaluation is improved; according to the characteristics of UAV aerial images, structural similarity, color difference, and texture complexity are comprehensively considered, and a new quality evaluation function is proposed to evaluate each pixel on the stitching seam. Then the difference cost is updated and the stitching seam is re-estimated based on the evaluation result. The estimation and evaluation process is repeated until the stitching seam tends to be stable and the iteration is stopped; the obtained stitching seam preferentially passes through roads, woodlands and other areas, which is more in line with human visual perception.

8. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 7, characterized in that: In step 2.2, the cost map includes a cost map for drones and a cost map for unmanned vehicles; the cost map is constructed based on the three-dimensional point cloud data of drones and unmanned vehicles to generate a cost map of the region; the cost map used for autonomous driving planning and obstacle avoidance functions of ground unmanned platforms is a two-dimensional grid map, which presents the passability and obstacle distribution information in the region; among them, the global cost grid map is a large-scale full-scene map, which is regularly maintained and updated based on the map of the executable area; The three-dimensional point cloud data collected by drones and ground unmanned platforms is denoised to eliminate the interference of noise points, and then the ground is segmented to extract the ground data and construct the cost map, including scene partitioning, feature extraction and cost map construction. Finally, the cost map is further processed to extract the skeleton line.

9. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 8, characterized in that: In step 2.3, the construction of the high-precision channel map includes: The ground unmanned platform's rapid environmental perception and mapping uses laser SLAM mapping technology, combined with the laser radar and IMU sensors carried by the unmanned vehicle, to enable the ground unmanned platform to build high-precision point cloud data of the channel in real time during driving, forming a high-precision channel map.

10. The method for constructing a high-precision map of an integrated off-road environment of "sky-air-ground" as claimed in claim 9, characterized in that: In step 2.3, a tightly coupled laser-inertial odometry method is used to obtain a globally consistent position and posture of the target by optimizing the LiDAR odometry factor, the IMU pre-integration factor and the closed-loop detection factor; frame-local map matching is used instead of frame-global map matching to improve the efficiency of frame map matching, thereby greatly improving the real-time performance of map construction.

Images