Topographic mapping methods, devices and equipment

Abstract

This application relates to a method, apparatus, and equipment for topographic mapping. The method includes: synchronizing point cloud data acquired by a laser radar onboard an aircraft and motion data acquired by an inertial navigation device onboard the aircraft; performing real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data; fusing the synchronized point cloud data and motion data to obtain optimized pose point cloud data; and converting the real-time pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination topographic map. The solution provided in this application can address both the real-time and accuracy requirements for topographic maps and meet the practical needs of intelligent flying vehicles.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to methods, devices and equipment for topographic mapping. Background Art

[0002] Intelligent flying cars are a key research topic in the field of intelligent driving and a key development direction for future transportation. Their autonomous driving capabilities play a crucial role. For autonomous driving to work, they require prior access to terrain maps or topographic information within a certain distance of their current location to mitigate risks. Terrain data also serves as prerequisite information for flight trajectory planning.

[0003] However, in related technologies, either the generated terrain maps have low accuracy or poor real-time performance, which cannot meet the actual needs of intelligent flying cars that require high real-time performance and accuracy. Summary of the Invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a topographic mapping method, device and computer-readable storage medium, which can take into account the real-time and accuracy requirements of topographic maps and meet the actual needs of intelligent flying cars.

[0005] A first aspect of the present application provides a topographic mapping method, comprising:

[0006] Synchronize the point cloud data acquired by the laser radar onboard the aircraft and the motion data acquired by the inertial navigation equipment onboard the aircraft;

[0007] Perform real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data;

[0008] The synchronized point cloud data and motion data are fused to obtain optimized pose point cloud data;

[0009] The instant pose point cloud data and the optimized pose point cloud data are converted and processed to obtain elevation raster data to generate a destination terrain map.

[0010] In one embodiment, fusing the synchronized point cloud data and motion data to obtain optimized pose point cloud data includes:

[0011] The positioning and mapping SLAM framework is used to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data.

[0012] In one embodiment, the performing real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data includes: performing real-time dead reckoning processing on the latest historical optimized pose point cloud data, the synchronized point cloud data, and the synchronized motion data to obtain real-time pose point cloud data.

[0013] In one embodiment, performing real-time dead reckoning processing based on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data to obtain instantaneous pose point cloud data includes:

[0014] Based on the synchronized motion data and the latest historical optimized pose point cloud data, the pose of the aircraft is predicted to obtain the predicted pose data of the aircraft;

[0015] The predicted pose data and the synchronized point cloud data are encapsulated into real-time pose point cloud data.

[0016] In one embodiment, the synchronization of point cloud data acquired by a laser radar carried by an aircraft and motion data acquired by an inertial navigation device carried by the aircraft includes:

[0017] The motion data is interpolated using the motion data of the adjacent frames before and after the current frame of the point cloud data, so that the interpolated motion data is aligned with the current frame of the point cloud data acquired by the lidar to achieve synchronization of the point cloud data and the motion data.

[0018] In one embodiment, the method further comprises:

[0019] Synchronize image data acquired by the visual equipment on board the aircraft;

[0020] Perform semantic segmentation on the image data acquired by the synchronized visual device to obtain semantic category labels;

[0021] The step of performing dead reckoning processing on the synchronized point cloud data and motion data to obtain instant pose point cloud data includes: performing dead reckoning processing on the synchronized point cloud data and motion data to obtain instant pose point cloud data; and semantically labeling the instant pose point cloud data according to semantic category labels to obtain instant pose point cloud data containing semantic labels.

[0022] The step of fusing the synchronized point cloud data and the motion data to obtain the optimized pose point cloud data includes: fusing the synchronized point cloud data and the motion data to obtain the optimized pose point cloud data; semantically labeling the optimized pose point cloud data according to the semantic category labels to obtain the optimized pose point cloud data containing the semantic labels;

[0023] The step of converting the instant pose point cloud data and the optimized pose point cloud data into elevation raster data to generate a destination terrain map comprises converting the instant pose point cloud data containing semantic tags and the optimized pose point cloud data containing semantic tags to obtain elevation raster data to generate a destination terrain map.

[0024] In one embodiment, converting the instant pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination terrain map includes:

[0025] Converting the pose data and point cloud data in the instant pose point cloud data and the optimized pose point cloud data into point cloud data in a map coordinate system respectively;

[0026] The point cloud data of the respectively transformed map coordinate systems are subjected to elevation rasterization processing through spatial rasterization to obtain elevation raster data to generate a destination topographic map.

[0027] In one embodiment, performing elevation rasterization processing on the point cloud data of the respectively transformed map coordinate systems through spatial rasterization to obtain elevation raster data to generate a destination terrain map includes:

[0028] According to the principle that the processing priority of the instant pose point cloud data is higher than the processing priority of the optimized pose point cloud data, the point cloud data of the map coordinate system converted from the instant pose point cloud data and the point cloud data of the map coordinate system converted from the optimized pose point cloud data are converted into high-priority elevation raster data and low-priority elevation raster data respectively;

[0029] According to the priority of the elevation raster data, the elevation raster data is aggregated into elevation slices in matrix form to generate an elevation raster map; according to the purpose of use, a map generator corresponding to the purpose of use is used to convert the elevation raster map into a topographic map corresponding to the purpose of use.

[0030] In one embodiment, wherein the elevation raster data includes a global cell index, the method further comprises:

[0031] Calculate the elevation slice global index and raster index of the elevation raster data according to the global cell index of the elevation raster data;

[0032] Query the priority and update time of historical elevation raster data according to the elevation slice global index and raster index;

[0033] If the priority of the current elevation raster data is higher than that of the historical elevation raster data, and the update time of the current elevation raster data is later than that of the historical elevation raster data, the elevation slice will be updated.

[0034] A second aspect of the present application provides a topographic mapping device, the device comprising:

[0035] Synchronization module, used to synchronize the point cloud data acquired by the aircraft's onboard laser radar and the motion data acquired by the aircraft's onboard inertial navigation equipment;

[0036] The dead reckoning module is used to perform real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data;

[0037] The fusion optimization module is used to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data;

[0038] The conversion module is used to convert the instant pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination terrain map.

[0039] A third aspect of the present application provides an electronic device, including:

[0040] processor; and

[0041] The memory stores executable codes, and when the executable codes are executed by the processor, the processor executes the above method.

[0042] The technical solution provided by this application may have the following beneficial effects:

[0043] The technical solution provided by this application, on the one hand, because the optimized pose point cloud data obtained by fusing the synchronized point cloud data and motion data contains the pose information of the aircraft with high precision, the accuracy of the subsequently generated terrain map is guaranteed; on the other hand, because the aircraft's position is calculated in real time, it has advantages in terms of calculation speed over other technologies, such as the calculation speed of the SLAM framework, with faster calculation speed and less latency. Therefore, when converting the optimized pose point cloud data and the real-time pose point cloud data into elevation raster data and generating the destination terrain map, the real-time pose point cloud data can be used for conversion first, thereby making the real-time performance of the destination terrain map stronger. Therefore, the solution provided by this application can take into account the requirements for real-time and precision of terrain maps and meet the actual needs of intelligent flying cars.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0046] Figure 1 1 is a flow chart of a topographic mapping method according to an embodiment of the present application;

[0047] Figure 2 is a flow chart of a topographic mapping method according to another embodiment of the present application;

[0048] Figure 3 1 is a flow chart of a topographic mapping method according to another embodiment of the present application;

[0049] Figure 4 is a schematic flow chart of a topographic mapping method according to another embodiment of the present application;

[0050] Figure 5 Schematic diagram of the organization of elevation slices and elevation grid units shown in an embodiment of the present application;

[0051] Figure 6 Schematic diagram of the structure of a topographic mapping device shown in an embodiment of the present application;

[0052] Figure 7 is a schematic structural diagram of a topographic mapping device according to another embodiment of the present application;

[0053] Figure 8 It is a structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although the accompanying drawings illustrate embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0055] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0056] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0057] In the field of intelligent driving, related technologies use various methods to generate topographic maps for intelligent flying vehicles. However, these methods often suffer from low accuracy or poor real-time performance, failing to meet the practical needs of intelligent flying vehicles, which require both real-time performance and high accuracy. To address these issues, embodiments of the present application provide a topographic mapping method that balances the real-time and high-accuracy requirements of topographic maps, meeting the practical needs of intelligent flying vehicles.

[0058] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0059] See also Figure 1 , is a flow chart of a topographic mapping method according to an embodiment of the present application, which mainly includes:

[0060] Step S101: Synchronize the point cloud data acquired by the laser radar carried by the aircraft and the motion data acquired by the inertial navigation device carried by the aircraft.

[0061] In the embodiments of the present application, the inertial navigation device carried by the aircraft is a general term for positioning devices such as the Inertial Measurement Unit (IMU) and Global Navigation Satellite System (GNSS) carried by the aircraft. Because different sensors may have inconsistent sampling frequencies even after hardware synchronization, that is, the data acquired by the sensors may be out of sync in terms of timestamps. The subsequent fusion of sensor data requires synchronization of these sensor data. Therefore, it is necessary to synchronize the point cloud data acquired by the aircraft's lidar and the motion data acquired by the aircraft's inertial navigation device.

[0062] Step S102: Perform real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data.

[0063] This step can be to construct an observation equation with the synchronized point cloud data and a motion equation with the synchronized motion data after the point cloud data acquired by the lidar and the motion data acquired by the inertial navigation device are synchronized, and the optimal value of the point cloud data in the two equations is solved in real time to obtain the instant pose point cloud data.

[0064] Step S103: Fusing the synchronized point cloud data and motion data to obtain optimized pose point cloud data.

[0065] It should be noted that the above steps S102 and S103 may be executed simultaneously or in sequence, that is, step S102 may be executed before or after step S103, or may be executed synchronously.

[0066] Step S104: converting the instant pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination terrain map.

[0067] From the above Figure 1 It can be seen from the example topographic mapping method that, on the one hand, since the optimized pose point cloud data obtained by fusing the synchronized point cloud data and motion data contains the pose information of the aircraft with high precision, the accuracy of the subsequently generated topographic map is guaranteed; on the other hand, since the aircraft's position is calculated in real time, it has advantages in terms of calculation speed over other technologies, such as the calculation speed of the SLAM framework, with faster calculation speed and less delay. Therefore, when converting the optimized pose point cloud data and the real-time pose point cloud data into elevation raster data and generating the destination topographic map, the real-time pose point cloud data can be used for conversion first, so that the real-time performance of the generated destination topographic map is stronger. Therefore, the solution provided by this application can take into account the requirements for the real-time and accuracy of the topographic map and meet the actual needs of intelligent flying cars.

[0068] See also Figure 2 , is a flow chart of a topographic mapping method according to another embodiment of the present invention. In this embodiment, the latest optimized pose point cloud data is taken into account during the dead reckoning process.

[0069] The method mainly includes:

[0070] Step S201: Synchronize the point cloud data acquired by the laser radar carried by the aircraft and the motion data acquired by the inertial navigation device carried by the aircraft.

[0071] In this step, the motion data can be interpolated using the motion data of the adjacent frames before and after the current frame of the point cloud data, so that the interpolated motion data is aligned with the current frame of the point cloud data obtained by the lidar to achieve synchronization of the point cloud data and the motion data.

[0072] It should be noted that in the above embodiment, since each frame of motion data or point cloud data includes the timestamp of when it was acquired, the timestamp can be used to determine the motion data of the frames immediately preceding and following the current frame of the point cloud data. This step can interpolate the motion data using the motion data of the frames immediately preceding and following the current frame of the laser point cloud data, aligning the interpolated motion data with the current frame of the point cloud data acquired by the lidar to achieve synchronization between the point cloud data and the motion data. As for the specific interpolation method, a linear interpolation algorithm or other interpolation algorithms can be used.

[0073] Step S202: Perform real-time dead reckoning processing based on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data to obtain instantaneous pose point cloud data.

[0074] Since the optimized pose point cloud data is obtained by continuously iteratively optimizing the synchronized point cloud data, when the latest historical optimized pose point cloud data, i.e., the latest optimized pose point cloud data in the historical optimized pose point cloud data, exists, the dead reckoning module can perform real-time dead reckoning processing based on the latest historical optimized pose point cloud data, the synchronized point cloud data, and the synchronized motion data to obtain the instantaneous pose point cloud data.

[0075] It should be noted that before performing the dead reckoning processing, the synchronized lidar point cloud data may be pre-processed by de-distortion, invalid point filtering, etc., and the pre-processed point cloud data may be sent to the dead reckoning module for processing.

[0076] Step S203: Using the positioning and mapping SLAM framework to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data.

[0077] This step can use a positioning and mapping SLAM framework, where the SLAM framework module uses the front-end odometer and back-end optimization module to perform fusion processing to obtain a series of optimized pose point cloud data. The processing of the positioning and mapping SLAM framework can be implemented using related technologies and is not limited in this application.

[0078] It should be noted that before performing fusion optimization, the synchronized lidar point cloud data can be preprocessed by dedistortion, invalid point filtering, etc., and the preprocessed point cloud data can be sent to the SLAM framework module (also called fusion optimization module) for processing.

[0079] It should also be noted that the above steps S202 and S203 may be executed simultaneously or in sequence, that is, step S202 may be executed before or after step S203, or may be executed synchronously.

[0080] Step S204: converting the pose data and point cloud data in the instant pose point cloud data and the optimized pose point cloud data into point cloud data in a map coordinate system respectively.

[0081] The pose data in the instant pose point cloud data represents the pose of the LiDAR coordinate system in the map coordinate system, and the point cloud data in the instant pose point cloud data represents the point cloud in the LiDAR coordinate system. Therefore, the pose data and point cloud data in the instant pose point cloud data and optimized pose point cloud data are converted into point cloud data in the map coordinate system. The conversion between the LiDAR coordinate system and the map coordinate system can be achieved using relevant technologies and is not limited in this application.

[0082] Step S205: performing elevation rasterization processing on the point cloud data of the respectively transformed map coordinate systems through spatial rasterization to obtain elevation raster data to generate a destination terrain map.

[0083] Raster data is a data format that divides space into regular grids, each called a cell, and assigns corresponding attribute values ​​to each cell to represent entities. This step uses the spatial grid to perform elevation rasterization on the point cloud data converted to the map coordinate system, generating elevation raster data. This elevation raster data is then used to regenerate the destination terrain map.

[0084] It can be seen from this embodiment that, in addition to performing real-time dead reckoning processing based on the synchronized point cloud data and the synchronized motion data to obtain instantaneous pose point cloud data, the present application solution can further refer to the latest historical optimized pose point cloud data as a motion prediction benchmark. At the same time, based on the latest historical optimized pose point cloud data, the synchronized point cloud data and the synchronized motion data, real-time dead reckoning processing can be performed to obtain instantaneous pose point cloud data, which can make the instantaneous pose point cloud data more accurate.

[0085] See also Figure 3 , is a flow chart of a topographic mapping method according to another embodiment of the present application. In this embodiment, the principle that the processing priority of the instant pose point cloud data is higher than the processing priority of the optimized pose data is taken into consideration.

[0086] The method mainly includes:

[0087] Step S301: Synchronize the point cloud data acquired by the laser radar carried by the aircraft and the motion data acquired by the inertial navigation device carried by the aircraft.

[0088] The implementation of step S301 is the same as the implementation of step S201 in the aforementioned embodiment. Please refer to the relevant description of the aforementioned embodiment and will not be described in detail here.

[0089] Step S302: Perform real-time dead reckoning processing based on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data to obtain instantaneous pose point cloud data.

[0090] Since the optimized pose point cloud data is obtained by iteratively optimizing the synchronized point cloud data, there is the latest optimized pose point cloud data in history, i.e., the latest optimized pose point cloud data in the historical optimized pose point cloud data. Specifically, based on the latest optimized pose point cloud data, the synchronized point cloud data, and the synchronized motion data, real-time dead reckoning processing is performed to obtain the instantaneous pose point cloud data, which can be achieved by the following steps S1 and S2:

[0091] Step S3021: Predict the posture of the aircraft based on the synchronized motion data and the latest historical optimized posture point cloud data to obtain the predicted posture data of the aircraft.

[0092] The motion data obtained by the inertial navigation device can be used to obtain the change in the aircraft's posture through a complementary filtering algorithm. Then, if there is the latest historical optimized posture point cloud data, the latest historical optimized posture point cloud data is used as the motion prediction benchmark. Combined with the posture change obtained based on the motion data, the aircraft's posture is predicted to obtain the aircraft's predicted posture data, making the aircraft's predicted posture data more accurate.

[0093] It should be noted that the present application solution may also not consider the latest historical optimized pose point cloud data, and directly use the synchronized point cloud data and synchronized motion data to perform real-time dead reckoning processing.

[0094] Step S3022: Encapsulate the predicted pose data and the synchronized point cloud data into real-time pose point cloud data.

[0095] This step encapsulates the predicted pose data and the synchronized point cloud data into real-time pose point cloud data, which means that the real-time pose point cloud data includes the predicted pose data and synchronized point cloud data of the aircraft.

[0096] It should be noted that before performing the dead reckoning processing, the synchronized lidar point cloud data may be pre-processed by de-distortion, invalid point filtering, etc., and the pre-processed point cloud data may be sent to the dead reckoning module for processing.

[0097] Step S303: Using the positioning and mapping SLAM framework to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data.

[0098] This step can use a positioning and mapping SLAM framework, where the SLAM framework module uses the front-end odometer and back-end optimization module to perform fusion processing to obtain a series of optimized pose point cloud data. The processing of the positioning and mapping SLAM framework can be implemented using related technologies and is not limited in this application.

[0099] It should be noted that before performing fusion optimization, the synchronized lidar point cloud data can be preprocessed by de-distortion, invalid point filtering, etc., and the preprocessed point cloud data can be sent to the SLAM framework module for processing.

[0100] It should also be noted that the above steps S302 and S303 may be executed simultaneously or in sequence, that is, step S302 may be executed before or after step S303, or may be executed synchronously.

[0101] Step S304: According to the principle that the processing priority of the instant pose point cloud data is higher than the processing priority of the optimized pose point cloud data, the instant pose point cloud data and the optimized pose point cloud data are converted into high-priority elevation raster data and low-priority elevation raster data respectively.

[0102] In this application, considering the importance of real-time performance and ensuring the real-time nature of the terrain map, the processing priority of the instant pose point cloud data can be set higher than the processing priority of the optimized pose point cloud data. Therefore, in this step, the instant pose point cloud data and the optimized pose point cloud data are converted into high-priority elevation raster data and low-priority elevation raster data, respectively, in accordance with the principle that the processing priority of the instant pose point cloud data is higher than the processing priority of the optimized pose point cloud data.

[0103] Step S305: According to the priority of the elevation raster data, the elevation raster data is aggregated into elevation slices in a matrix form to generate an elevation raster map; according to the purpose of use, a map generator corresponding to the purpose of use is used to convert the elevation raster map into a topographic map corresponding to the purpose of use.

[0104] This step aggregates high-priority and low-priority elevation raster data into matrix-shaped elevation slices based on the priority of the elevation raster data to generate an elevation raster map. Elevation slices are also elevation raster units arranged in a matrix form. Here, multiple elevation raster units are aggregated and positioned as one elevation slice. Figure 5 , is a schematic diagram of the organization of elevation slices and elevation grid units shown in an embodiment of the present application.

[0105] For example, this step can be based on the purpose of use and use a map generator corresponding to the purpose of use to convert the elevation grid map into an ESDF (Euclidean Signed Distance Functions) map that is convenient for path planning, a grid map with low data volume and easy to transmit, and an elevation map that can be expressed in multiple resolutions to suit different scale requirements. It should be noted that in the above embodiment, the elevation grid data contained in each elevation slice is actually organized in matrix form, such as Figure 5 As shown in the figure, the elevation slice is an N*N matrix. For the elevation slice in the i-th row and j-th column, it is actually an N*N matrix, where the element in the i-th row and j-th column is an elevation raster data; each elevation slice contains a timestamp, an elevation slice global index, an elevation data matrix, an update time matrix and a priority matrix.

[0106] It should be noted that the elevation raster data contains a global unit index. The method may further include: calculating the elevation slice global index and raster index of the elevation raster data based on the global unit index of the elevation raster data; querying the priority and update time of the historical elevation raster data according to the elevation slice global index and raster index; if the priority of the current elevation raster data is higher than the priority of the historical elevation raster data, and the update time of the current elevation raster data is later than the update time of the historical elevation raster data, then updating the elevation slice.

[0107] As can be seen from this embodiment, in the embodiment of the present application, the elevation grid data obtained by converting the optimized pose point cloud data and the real-time pose point cloud data mainly includes information such as the timestamp, resolution, the global unit index of the elevation grid data, the elevation data, and the priority of the elevation grid data. Since the elevation grid data is converted from the optimized pose point cloud data or the real-time pose point cloud data, the priority of the elevation grid data actually corresponds to the processing priority of the optimized pose point cloud data or the real-time pose point cloud data. As mentioned above, the real-time pose point cloud data is obtained by real-time dead reckoning of the aircraft's position based on point cloud data and motion data, while the optimized pose point cloud data is obtained by fusing synchronized sensor data using the SLAM framework. The real-time performance of the real-time pose point cloud data is better than that of the optimized pose point cloud data. Therefore, in order to ensure real-time performance, the processing priority of the real-time pose point cloud data is higher than that of the optimized pose point cloud data. Accordingly, the real-time pose point cloud data is converted to the high-priority elevation grid data, while the optimized pose point cloud data is converted to the low-priority elevation grid data. In this way, subsequent modules will give priority to processing high-priority elevation raster data (that is, give priority to processing real-time pose point cloud data), thereby ensuring the real-time nature of the terrain map.

[0108] See also Figure 4, is a flow chart of a topographic mapping method according to another embodiment of the present application. This embodiment further considers image data acquired by a visual device and semantic labeling.

[0109] This application solution combines the advantages of SLAM framework optimization mapping and real-time terrain rapid mapping based on short-term dead reckoning, and semantically labels the real-time pose point cloud data and optimized pose point cloud data to provide basic semantic data for subsequent map generation and conversion.

[0110] The method mainly includes:

[0111] Step S401: Synchronize the point cloud data acquired by the laser radar carried by the aircraft, the motion data acquired by the inertial navigation device carried by the aircraft, and the image data acquired by the visual device carried by the aircraft, and proceed to step S402.

[0112] The image data acquired by the visual device carried by the synchronized aircraft here is similar to the point cloud data acquired by the laser radar carried by the synchronized aircraft and the motion data acquired by the inertial navigation device carried by the aircraft in the aforementioned embodiment. The difference is that the synchronized object this time has added image data.

[0113] This step can interpolate the motion data using the motion data of the adjacent frames before and after the current frame of the laser point cloud data, so that the interpolated motion data is aligned with the current frame of the point cloud data acquired by the laser radar to achieve synchronization of the point cloud data and the motion data. Furthermore, the synchronization of the image data can be achieved by performing time-domain nearest neighbor matching on the image based on the timestamp of the current frame of the point cloud data acquired by the laser radar. Among them, the time-domain nearest neighbor matching can be implemented using algorithms such as the nearest neighbor method, which is not limited in this application.

[0114] Step S402: Perform semantic segmentation on the image data acquired by the synchronized visual device to obtain semantic category labels, and then proceed to step S403.

[0115] In the embodiment of the present application, semantic segmentation can be performed on the image data acquired by the visual device, and semantic category labels can be obtained through a semantic segmentation network based on deep learning. Specifically, semantic segmentation can be performed on the image data acquired by the visual device to obtain semantic category labels. The image data can be input into the semantic segmentation network to obtain the semantic segmentation result image S of the kth frame image. k , when the semantic segmentation result map S k When the value of the pixel in the i-th row and j-th column is between the lower limit and upper limit of the semantic category label of the p-th category in the semantic segmentation result map, the semantic category label of the pixel in the i-th row and j-th column in the image data is the p-th category.

[0116] Step S403: Based on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data, real-time dead reckoning processing is performed to obtain instant pose point cloud data, and then step S404 is entered.

[0117] Step S403 may include:

[0118] Step S4031: Predict the posture of the aircraft based on the synchronized motion data and the latest historical optimized posture point cloud data to obtain the predicted posture data of the aircraft.

[0119] The motion data obtained by the inertial navigation device can be used to obtain the change in the aircraft's posture through a complementary filtering algorithm. Then, if there is the latest historical optimized posture point cloud data, the latest historical optimized posture point cloud data is used as the motion prediction benchmark. Combined with the posture change obtained based on the motion data, the aircraft's posture is predicted to obtain the aircraft's predicted posture data, making the aircraft's predicted posture data more accurate.

[0120] Step S4032: Encapsulate the predicted pose data and the synchronized point cloud data into real-time pose point cloud data.

[0121] This step encapsulates the predicted pose data and the synchronized point cloud data into real-time pose point cloud data, which means that the real-time pose point cloud data includes the predicted pose data and synchronized point cloud data of the aircraft.

[0122] It should be noted that before performing the dead reckoning processing, the synchronized lidar point cloud data may be pre-processed by de-distortion, invalid point filtering, etc., and the pre-processed point cloud data may be sent to the dead reckoning module for processing.

[0123] Step S404: semantically labeling the instant pose point cloud data according to the semantic category label to obtain instant pose point cloud data containing the semantic label.

[0124] This step uses semantic category labels to semantically label the instant pose point cloud data to obtain instant pose point cloud data containing semantic labels, which can provide basic semantic data for subsequent map generation and conversion.

[0125] Step S405: Use the positioning and mapping SLAM framework to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data, and then proceed to step S406.

[0126] This step can use a positioning and mapping SLAM framework, where the SLAM framework module uses the front-end odometer and back-end optimization module to perform fusion processing to obtain a series of optimized pose point cloud data. The processing of the positioning and mapping SLAM framework can be implemented using related technologies and is not limited in this application.

[0127] It should be noted that before performing fusion optimization, the synchronized lidar point cloud data can be preprocessed by de-distortion, invalid point filtering, etc., and the preprocessed point cloud data can be sent to the SLAM framework module for processing.

[0128] It should also be noted that the above steps S403 and S405 may be executed simultaneously or in sequence.

[0129] Step S406: semantically labeling the optimized pose point cloud data according to the semantic category label to obtain optimized pose point cloud data containing the semantic label.

[0130] This step uses semantic category labels to semantically label the optimized pose point cloud data to obtain optimized pose point cloud data containing semantic labels, which can provide basic semantic data for subsequent map generation and conversion.

[0131] Step S407: converting the pose data and point cloud data in the instant pose point cloud data containing semantic tags and the optimized pose point cloud data containing semantic tags into point cloud data in a map coordinate system respectively.

[0132] The pose data in the instant pose point cloud data is the pose of the laser radar coordinate system in the map coordinate system, and the point cloud data in the instant pose point cloud data is the point cloud in the laser radar coordinate system. Therefore, the pose data and point cloud data in the instant pose point cloud data containing semantic tags and the optimized pose point cloud data containing semantic tags are converted into point cloud data in the map coordinate system. The conversion between the laser radar coordinate system and the map coordinate system can be achieved by using relevant technologies, and this application does not limit it.

[0133] Step S408: According to the principle that the processing priority of the instant pose point cloud data containing semantic tags is higher than the processing priority of the optimized pose point cloud data containing semantic tags, the point cloud data of the map coordinate system converted from the instant pose point cloud data containing semantic tags and the point cloud data of the map coordinate system converted from the optimized pose point cloud data containing semantic tags are converted into high-priority elevation raster data and low-priority elevation raster data, respectively.

[0134] In this application, considering the importance of real-time performance and ensuring the real-time performance of the terrain map, the processing priority of the instant pose point cloud data can be set higher than the processing priority of the optimized pose point cloud data. Therefore, in accordance with the principle that the processing priority of the instant pose point cloud data containing semantic tags is higher than the processing priority of the optimized pose point cloud data containing semantic tags, this step converts the point cloud data of the map coordinate system converted from the instant pose point cloud data containing semantic tags and the point cloud data of the map coordinate system converted from the optimized pose point cloud data containing semantic tags into high-priority elevation raster data and low-priority elevation raster data, respectively.

[0135] Step S409: According to the priority of the elevation raster data, the elevation raster data is aggregated into elevation slices in a matrix form to generate an elevation raster map; according to the purpose of use, a map generator corresponding to the purpose of use is used to convert the elevation raster map into a topographic map corresponding to the purpose of use.

[0136] This step aggregates high-priority elevation raster data and low-priority elevation raster data into elevation slices in a matrix form according to the priority of the elevation raster data to generate an elevation raster map. Elevation slices are also elevation raster units arranged in a matrix form. Here, multiple elevation raster units are aggregated and positioned as one elevation slice. Then, according to the purpose of use, the elevation raster map can be converted into an ESDF map that is convenient for path planning, a grid map with low data volume and easy to transmit, and an elevation map that can be expressed in multiple resolutions to suit different scale requirements.

[0137] It should be noted that the elevation grid data includes a global cell index. The method may further include:

[0138] According to the global cell index of the elevation raster data, the elevation slice global index and raster index of the elevation raster data are calculated; according to the elevation slice global index and raster index, the priority and update time of the historical elevation raster data are queried; if the priority of the current elevation raster data is higher than the priority of the historical elevation raster data, and the update time of the current elevation raster data is later than the update time of the historical elevation raster data, the elevation slice is updated.

[0139] It can be seen from this embodiment that in the embodiment of the present application, in addition to the point cloud data obtained by the lidar and the motion data obtained by the inertial navigation device carried by the aircraft, the image data obtained by the visual device carried by the aircraft is further combined, and the image data is semantically segmented to obtain semantic category labels. The semantic category labels are used to semantically mark the instant pose point cloud data and the optimized pose point cloud data, thereby providing basic semantic data for subsequent map generation and conversion.

[0140] The solution provided in this application combines real-time terrain conversion based on dead reckoning and a terrain optimization process based on SLAM framework optimization, and has the characteristics of real-time measurement and precision optimization, which can guide the aircraft to perform terrain obstacle avoidance and other functional applications; in addition, based on the elevation grid map response mechanism with semantic information and update priority elevation grid data, it can effectively integrate real-time map data, optimized map data and semantic detection results into a unified processing flow, facilitating the maintenance and update of the elevation grid map; in addition, the elevation grid map, as the basic map data of the solution of this application, can be easily converted into other types of map data to meet different functional application requirements.

[0141] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a topographic mapping device, an electronic device and corresponding embodiments.

[0142] See also Figure 6 , is a schematic diagram of the structure of a topographic mapping device 300 according to an embodiment of the present application. For ease of description, only the parts related to the embodiment of the present application are shown. Figure 6 The exemplary topographic mapping device 300 mainly includes a synchronization module 301, a dead reckoning module 302, a fusion optimization module 303 and a conversion module 304. Among them:

[0143] Synchronization module 301, for synchronizing point cloud data acquired by the laser radar carried by the aircraft and motion data acquired by the inertial navigation device carried by the aircraft;

[0144] The dead reckoning module 302 is used to perform real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data;

[0145] A fusion optimization module 303 is used to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data;

[0146] The conversion module 304 is used to convert the instant pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination terrain map.

[0147] See also Figure 7 , is a structural diagram of a topographic mapping device shown in another embodiment of the present application.

[0148] Figure 7 The exemplary topographic mapping device 400 mainly includes a synchronization module 301, a dead reckoning module 302, a fusion optimization module 303, and a conversion module 304. The functions of the synchronization module 301, the dead reckoning module 302, the fusion optimization module 303, and the conversion module 304 can be found in Figure 6 Description in .

[0149] Fusion Optimization Module 303 can utilize a positioning and mapping SLAM framework to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data. Therefore, Fusion Optimization Module 303 can also be referred to as SLAM framework module 303. Fusion Optimization Module 303 utilizes its front-end odometry and back-end optimization modules to perform fusion processing, obtaining a series of optimized pose point cloud data. The processing within the positioning and mapping SLAM framework can be implemented using related technologies and is not limited in this application.

[0150] It should be noted that before the fusion optimization module 303 performs fusion optimization, the preprocessing module (not shown in the figure) can also first perform preprocessing such as dedistortion and invalid point filtering on the synchronized lidar point cloud data, and send the preprocessed point cloud data to the fusion optimization module 303 for processing.

[0151] The dead reckoning module 302 can perform real-time dead reckoning processing based on the latest historical optimized pose point cloud data, the synchronized point cloud data, and the synchronized motion data to obtain instantaneous pose point cloud data. For example, the dead reckoning module 302 can predict the pose of the aircraft based on the synchronized motion data and the latest historical optimized pose point cloud data to obtain predicted pose data for the aircraft. The predicted pose data and the synchronized point cloud data are then packaged into instantaneous pose point cloud data.

[0152] It should be noted that before the dead reckoning module 302 performs dead reckoning processing, a pre-processing module (not shown in the figure) may further perform pre-processing on the synchronized lidar point cloud data, such as de-distortion and invalid point filtering, and then send the pre-processed point cloud data to the dead reckoning module 302 for processing.

[0153] Among them, the synchronization module 301 can use the motion data of the adjacent frames before and after the current frame of the point cloud data to interpolate the motion data, so that the motion data obtained after interpolation is aligned with the current frame of the point cloud data obtained by the laser radar to achieve synchronization of the point cloud data and the motion data.

[0154] The topographic mapping device may further include: a semantic processing module 305 .

[0155] The synchronization module 301 can also synchronize image data acquired by the visual device carried by the aircraft, that is, the synchronization module 301 synchronizes the point cloud data acquired by the laser radar carried by the aircraft, the motion data acquired by the inertial navigation device carried by the aircraft, and the image data acquired by the visual device carried by the aircraft;

[0156] Semantic processing module 305, used to perform semantic segmentation on the image data acquired by the synchronized visual device to obtain semantic category labels;

[0157] The dead reckoning module 302 performs dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data, and then semantically labels the real-time pose point cloud data according to the semantic category labels obtained by the semantic processing module 305 to obtain real-time pose point cloud data containing semantic labels;

[0158] After the fusion optimization module 303 fuses the synchronized point cloud data and motion data to obtain optimized pose point cloud data, the optimized pose point cloud data is semantically labeled according to the semantic category labels obtained by the semantic processing module 305 to obtain optimized pose point cloud data containing semantic labels;

[0159] The conversion module 304 converts the instant pose point cloud data containing semantic tags and the optimized pose point cloud data containing semantic tags to obtain elevation raster data to generate a destination terrain map.

[0160] The conversion module 304 converts the pose data and point cloud data in the instant pose point cloud data and the optimized pose point cloud data into point cloud data in a map coordinate system, respectively; performs elevation rasterization processing on the point cloud data in the map coordinate system that have been converted respectively through a spatial grid to obtain elevation raster data to generate a destination terrain map. For example, the conversion module 304 converts the point cloud data in the map coordinate system converted from the instant pose point cloud data and the point cloud data in the map coordinate system converted from the optimized pose point cloud data into high-priority elevation raster data and low-priority elevation raster data, respectively, in accordance with the principle that the processing priority of the instant pose point cloud data is higher than the processing priority of the optimized pose point cloud data; aggregates the elevation raster data into elevation slices in a matrix form according to the priority of the elevation raster data to generate an elevation raster map; and converts the elevation raster map into a terrain map corresponding to the purpose of use using a map generator corresponding to the purpose of use according to the purpose of use.

[0161] The topographic mapping apparatus may further include: an updating module 306 .

[0162] When the elevation raster data contains a global unit index, the update module 306 calculates the elevation slice global index and raster index of the elevation raster data according to the global unit index of the elevation raster data; queries the priority and update time of the historical elevation raster data according to the elevation slice global index and raster index; if the priority of the current elevation raster data is higher than the priority of the historical elevation raster data, and the update time of the current elevation raster data is later than the update time of the historical elevation raster data, then updates the elevation slice.

[0163] The topographic mapping device provided in the present application combines the real-time terrain conversion based on dead reckoning and the terrain optimization process based on SLAM framework optimization, and has the characteristics of real-time measurement and precision optimization, and can guide the aircraft to perform terrain obstacle avoidance and other functional applications; in addition, based on the elevation grid map response mechanism with semantic information and update priority elevation grid data, it can effectively integrate the real-time map data, optimized map data and semantic detection results into a unified processing flow, which facilitates the maintenance and update of the elevation grid map; in addition, the elevation grid map, as the basic map data of the present application solution, can be easily converted into other types of map data to meet different functional application requirements.

[0164] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated again here.

[0165] Figure 8 It is a structural diagram of an electronic device shown in an embodiment of the present application.

[0166] See also Figure 8 , the electronic device 400 includes a memory 410 and a processor 420.

[0167] The processor 420 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0168] Memory 410 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. ROM may store static data or instructions required by processor 420 or other modules of the computer. Permanent storage may be a readable and writable storage device. A permanent storage device may be a non-volatile storage device that retains stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (e.g., a magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (e.g., a floppy disk, optical drive). System memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. System memory may store some or all instructions and data required by the processor during operation. In addition, memory 410 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 410 may include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0169] The memory 410 stores executable codes. When the executable codes are processed by the processor 420 , the processor 420 may execute part or all of the above-mentioned methods.

[0170] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0171] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0172] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for topographic mapping, characterized in that: The method is applied to an aircraft and comprises: Synchronize the point cloud data acquired by the laser radar onboard the aircraft and the motion data acquired by the inertial navigation equipment onboard the aircraft; Performing real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data, including: performing real-time dead reckoning processing on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data to obtain real-time pose point cloud data; The synchronized point cloud data and motion data are fused to obtain optimized pose point cloud data; The instant pose point cloud data and the optimized pose point cloud data are converted and processed to obtain elevation raster data to generate a destination terrain map.

2. The method for topographic mapping according to claim 1, characterized in that: The step of fusing the synchronized point cloud data and motion data to obtain optimized pose point cloud data includes: The positioning and mapping SLAM framework is used to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data.

3. The topographic mapping method according to claim 1, characterized in that: The real-time dead reckoning processing is performed based on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data to obtain instant pose point cloud data, including: Based on the synchronized motion data and the latest historical optimized pose point cloud data, the pose of the aircraft is predicted to obtain the predicted pose data of the aircraft; The predicted pose data and the synchronized point cloud data are encapsulated into real-time pose point cloud data.

4. The method for topographic mapping according to claim 1, wherein: The point cloud data acquired by the laser radar carried by the synchronous aircraft and the motion data acquired by the inertial navigation device carried by the aircraft include: The motion data is interpolated using the motion data of the adjacent frames before and after the current frame of the point cloud data, so that the interpolated motion data is aligned with the current frame of the point cloud data acquired by the lidar to achieve synchronization of the point cloud data and the motion data.

5. The method for topographic mapping according to claim 1, characterized in that: The method further comprises: Synchronize image data acquired by the visual equipment on board the aircraft; Perform semantic segmentation on the image data acquired by the synchronized visual device to obtain semantic category labels; The step of performing dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data further includes: performing dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data; semantically labeling the real-time pose point cloud data according to semantic category labels to obtain real-time pose point cloud data containing semantic labels; The step of fusing the synchronized point cloud data and the motion data to obtain the optimized pose point cloud data includes: fusing the synchronized point cloud data and the motion data to obtain the optimized pose point cloud data; semantically labeling the optimized pose point cloud data according to the semantic category labels to obtain the optimized pose point cloud data containing the semantic labels; The step of converting the instant pose point cloud data and the optimized pose point cloud data into elevation raster data to generate a destination terrain map comprises converting the instant pose point cloud data containing semantic tags and the optimized pose point cloud data containing semantic tags to obtain elevation raster data to generate a destination terrain map.

6. The method for topographic mapping according to claim 1, characterized in that: The converting and processing the instant pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination terrain map includes: Converting the pose data and point cloud data in the instant pose point cloud data and the optimized pose point cloud data into point cloud data in a map coordinate system respectively; The point cloud data of the respectively transformed map coordinate systems are subjected to elevation rasterization processing through spatial rasterization to obtain elevation raster data to generate a destination topographic map.

7. The method for topographic mapping according to claim 6, characterized in that: The step of performing elevation rasterization processing on the point cloud data of the respectively transformed map coordinate systems through the spatial raster to obtain elevation raster data to generate a destination terrain map includes: According to the principle that the processing priority of the instant pose point cloud data is higher than the processing priority of the optimized pose point cloud data, the point cloud data of the map coordinate system converted from the instant pose point cloud data and the point cloud data of the map coordinate system converted from the optimized pose point cloud data are converted into high-priority elevation raster data and low-priority elevation raster data respectively; According to the priority of the elevation raster data, the elevation raster data is aggregated into elevation slices in matrix form to generate an elevation raster map; according to the purpose of use, a map generator corresponding to the purpose of use is used to convert the elevation raster map into a topographic map corresponding to the purpose of use.

8. The method for topographic mapping according to claim 7, characterized in that: in, The elevation raster data includes a global cell index, and the method further includes: Calculate the elevation slice global index and raster index of the elevation raster data according to the global cell index of the elevation raster data; Query the priority and update time of historical elevation raster data according to the elevation slice global index and raster index; If the priority of the current elevation raster data is higher than that of the historical elevation raster data, and the update time of the current elevation raster data is later than that of the historical elevation raster data, the elevation slice will be updated.

9. A topographic mapping device, characterized in that: The device comprises: Synchronization module, used to synchronize the point cloud data acquired by the aircraft's onboard laser radar and the motion data acquired by the aircraft's onboard inertial navigation equipment; The dead reckoning module is used to perform real-time dead reckoning processing on the synchronized point cloud data and motion data to obtain real-time pose point cloud data. This includes: performing real-time dead reckoning processing based on the latest historical optimized pose point cloud data, synchronized point cloud data, and synchronized motion data to obtain real-time pose point cloud data; The fusion optimization module is used to fuse the synchronized point cloud data and motion data to obtain optimized pose point cloud data; The conversion module is used to convert the instant pose point cloud data and the optimized pose point cloud data to obtain elevation raster data to generate a destination terrain map.

10. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 8.

Images