A multi-layer grid map based scene quick reconstruction method

Abstract

This invention provides a method for rapid scene reconstruction based on multi-layer raster maps, comprising: acquiring LiDAR point cloud data and camera image data of the target scene; preprocessing the LiDAR point cloud data and camera image data, including data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering; inputting the preprocessed LiDAR point cloud data and camera image data into a scene element extraction module to obtain scene elements of the target scene; inputting the scene elements into a multi-layer map generation module to obtain a multi-layer raster map; and inputting the multi-layer raster map into a rapid scene reconstruction module to obtain the reconstructed scene. This invention's method for rapid scene reconstruction based on multi-layer raster maps can prioritize reconstruction using more reliable data from the multi-layer raster map, while simultaneously using other data to supplement the scene, improving the real-time performance of scene reconstruction and solving the problems of complex multi-layer map data and poor real-time performance in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method for rapid scene reconstruction based on multi-layer raster maps. Background Technology

[0002] Scene reconstruction is a key task in the fields of autonomous driving and intelligent transportation, and the real-time performance of reconstruction algorithms has a direct impact on the practical application of intelligent transportation systems. Scene reconstruction relies on the constructed map, and common map formats include raster maps, point cloud maps, and layered maps.

[0003] Multi-layer raster maps are a special form of 3D data representation that contains the spatial geometric information of the observed scene. Compared with other 3D data formats, multi-layer raster maps do not require storing the topological structure between discrete points, offering simpler, more flexible, and more powerful representation capabilities. They exhibit better performance during processing, making them a crucial data source for autonomous driving environmental perception. The current scene is constructed using multi-layer raster maps and provided to the autonomous driving decision-making module for planning and decision-making. Autonomous driving demands high real-time performance of the input scene data. However, real-world scenes are often complex, and different types of sensors are affected by the environment to varying degrees. The reliability of different layers in a multi-layer raster map also varies, and ensuring the accuracy of each layer can lead to a decrease in real-time performance. This paper first processes the original multi-layer raster map, prioritizing the reconstruction of highly reliable map layers, while supplementing the prioritized layers with broader and more detailed map layers. Real-time, dense scene data is crucial for achieving dynamic obstacle avoidance and path planning in high-level autonomous driving. Summary of the Invention

[0004] In view of this, the present invention provides a method for rapid scene reconstruction based on multi-layer raster maps to solve the above problems.

[0005] This invention provides a method for rapid scene reconstruction based on multi-layer raster maps, comprising: acquiring LiDAR point cloud data and camera image data of the target scene; preprocessing the LiDAR point cloud data and camera image data, including data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering; inputting the preprocessed LiDAR point cloud data and camera image data into a scene element extraction module to obtain scene elements of the target scene; inputting the scene elements into a multi-layer map generation module to obtain a multi-layer raster map; and inputting the multi-layer raster map into a rapid scene reconstruction module to obtain the reconstructed scene.

[0006] In another implementation of the present invention, the acquisition of LiDAR point cloud data and camera image data of the target scene includes: acquiring LiDAR point cloud data and timestamps of the LiDAR point clouds of the target scene; acquiring camera image data and timestamps of the camera images of the target scene; preprocessing the LiDAR point cloud data and camera image data, the preprocessing including data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering, including: determining whether the time difference between the LiDAR and the camera exceeds a threshold; if it does, the data is invalid, and the LiDAR point cloud data and camera image data of the target scene are reacquired; if it does not exceed the threshold, the camera is calibrated using extrinsic parameters; filtering the LiDAR point clouds and retaining the point clouds within the region of interest; and performing raster filtering and dimensionality reduction on the retained LiDAR point clouds.

[0007] In another implementation of the present invention, scene elements include the target category, center point location, three-dimensional size data, ground point cloud, ground model, and single target point cloud data; the preprocessed LiDAR point cloud data and camera image data are input into the scene element extraction module to obtain the scene elements of the target scene, including: processing the preprocessed camera image data using a trained deep learning model to obtain the target category, center point location, and three-dimensional size data in the camera image data; and performing ground point cloud extraction and non-ground point cloud clustering processing on the preprocessed LiDAR point cloud data to obtain ground point cloud, ground model, and single target point cloud data.

[0008] In another implementation of the present invention, the method for rapid scene reconstruction based on multi-layer raster maps further includes: performing reliability analysis on the detection results of camera image data, the results of ground point cloud extraction, and the results of non-ground point cloud clustering according to scene elements, and obtaining reliability analysis results; setting status labels for the detection results of camera image data, the results of ground point cloud extraction, and the results of non-ground point cloud clustering based on the reliability analysis results; and determining the reliability status of camera image data detection, ground point cloud extraction, and non-ground point cloud clustering through the reliability analysis results.

[0009] In another implementation of the present invention, reliability analysis is performed on the detection results of camera image data, the extraction results of ground point clouds, and the clustering results of non-ground point clouds according to scene elements to obtain reliability analysis results. This includes: performing reliability analysis on the detection results of camera image data based on the target category, center point position, and three-dimensional size to obtain reliability analysis results of the detection results; performing reliability analysis on the extraction results of ground point clouds based on ground point clouds and ground models to obtain reliability analysis results of the extraction results; and performing reliability analysis on the clustering results of non-ground point clouds based on single target point clouds to obtain reliability analysis results of the clustering results.

[0010] In another implementation of the present invention, the multi-layer raster map includes a center point map, a 3D dimension map, a ground point map, and a non-ground point cloud map. The process involves inputting scene elements into a multi-layer map generation module to obtain the multi-layer raster map, including: generating a base raster map based on the multi-layer map generation module; finding the raster point in the base raster map that is closest to each center point in the center point location, setting `cond` to 1, and obtaining the center point map; finding the raster point in the base raster map that is closest to each element in the 3D dimension data, setting `cond` to 1, and simultaneously setting `cond` to 1 for raster points near the raster point that are inside the target, and obtaining the 3D dimension map; finding the raster point in the base raster map that is closest to each point cloud in the ground point cloud, setting `cond` to 1, and obtaining the ground point map; and finding the raster point in the base raster map that is closest to each point cloud in the single target point cloud data, setting `cond` to 1, and obtaining the non-ground point cloud map.

[0011] In another implementation of the present invention, the grid points inside the target satisfy the following condition:

[0012]

[0013] Among them, (x o ,y o ,z o () is the grid point in the base grid map that is closest to the corresponding element in the 3D dimension data.

[0014] In another implementation of the present invention, a multi-layer raster map is input into a scene fast reconstruction module to obtain a reconstructed scene, including: the scene fast reconstruction module merges the center point map, the three-dimensional size map, the ground point map, and the non-ground point cloud map to generate map data for scene reconstruction.

[0015] The fast scene reconstruction method based on multi-layer raster maps of the present invention has the following beneficial effects:

[0016] 1. The multi-layer raster map scene reconstruction method proposed in this invention can balance the real-time performance and accuracy of map building, and improve the accuracy of map building while ensuring that important information is prioritized for mapping.

[0017] 2. The scene reconstruction method proposed in this invention has good robustness. By analyzing the characteristics of each layer of the raster map layer by layer, the reliability of the raster map used for merging is ensured. The real-time performance of the algorithm is further improved by removing unreliable information.

[0018] 3. The constructed grid map can prioritize the construction of drivable areas. For devices with limited computing resources, such as autonomous vehicles or roadside equipment of intelligent transportation systems, it can prioritize the use of limited computing resources to ensure the normal operation of functions such as autonomous driving, and has good applicability. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. By reading the detailed description of the embodiments below, the advantages and benefits of the solutions will become clear to those skilled in the art. The accompanying drawings are only for illustrating preferred embodiments and are not intended to limit the present invention.

[0020] In the attached diagram:

[0021] Figure 1 This is a flowchart illustrating the steps of a method for rapid scene reconstruction based on a multi-layer raster map, according to an embodiment of the present invention.

[0022] Figure 2 This is a software framework diagram of one embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram of the workflow of the preprocessing module according to an embodiment of the present invention.

[0024] Figure 4 This is a schematic diagram illustrating the workflow of a scene element extraction module according to an embodiment of the present invention.

[0025] Figure 5 This is a schematic diagram illustrating the workflow of a multi-layer map generation module according to an embodiment of the present invention.

[0026] Figure 6 This is a schematic diagram illustrating the workflow of a scene rapid reconstruction module according to an embodiment of the present invention.

[0027] Figure 7 This is a schematic diagram of a multi-layer map constructed according to an embodiment of the present invention.

[0028] Figure 8 This is a schematic diagram illustrating the construction of a scenario according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0030] Figure 1 A flowchart illustrating the steps of a rapid scene reconstruction method based on a multi-layer raster map provided in this embodiment of the invention is shown below. Figure 1 As shown, this embodiment mainly includes the following steps:

[0031] Collect lidar point cloud data and camera image data of the target scene.

[0032] The lidar point cloud data and camera image data are preprocessed, including data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering.

[0033] The preprocessed LiDAR point cloud data and camera image data are input into the scene element extraction module to obtain the scene elements of the target scene.

[0034] The scene elements are input into the multi-layer map generation module to obtain a multi-layer raster map.

[0035] Input the multi-layer raster map into the scene quick reconstruction module to obtain the reconstructed scene.

[0036] The fast scene reconstruction method based on multi-layer raster maps of the present invention has the following beneficial effects:

[0037] 1. The multi-layer raster map scene reconstruction method proposed in this invention can balance the real-time performance and accuracy of map building, and improve the accuracy of map building while ensuring that important information is prioritized for mapping.

[0038] 2. The scene reconstruction method proposed in this invention has good robustness. By analyzing the characteristics of each layer of the raster map layer by layer, the reliability of the raster map used for merging is ensured. The real-time performance of the algorithm is further improved by removing unreliable information.

[0039] 3. The constructed grid map can prioritize the construction of drivable areas. For devices with limited computing resources, such as autonomous vehicles or roadside equipment of intelligent transportation systems, it can prioritize the use of limited computing resources to ensure the normal operation of functions such as autonomous driving, and has good applicability.

[0040] In another implementation of the present invention, the acquisition of LiDAR point cloud data and camera image data of the target scene includes: acquiring LiDAR point cloud data and timestamps of the LiDAR point clouds of the target scene; acquiring camera image data and timestamps of the camera images of the target scene; preprocessing the LiDAR point cloud data and camera image data, the preprocessing including data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering, including: determining whether the time difference between the LiDAR and the camera exceeds a threshold; if it does, the data is invalid, and the LiDAR point cloud data and camera image data of the target scene are reacquired; if it does not exceed the threshold, the camera is calibrated using extrinsic parameters; filtering the LiDAR point clouds and retaining the point clouds within the region of interest; and performing raster filtering and dimensionality reduction on the retained LiDAR point clouds.

[0041] For example, such as Figure 3 As shown, the system acquires LiDAR point cloud data and its timestamp, as well as camera image data and its timestamp. It then determines whether the time difference between the LiDAR and camera exceeds a threshold. If it does, the data is invalid, and the LiDAR point cloud data and camera image data for the target scene are reacquired. If not, the camera's extrinsic parameters are calibrated. The specific determination process is as follows:

[0042] |t cam -t LiDAR |<T thresh

[0043] Among them, t cam t is the timestamp of the image data. LiDAR T is the timestamp for the lidar point cloud data. thresh T is the time difference threshold. thresh Set to 0.5 times the time it takes for the camera to acquire a single frame image.

[0044] Furthermore, the lidar point cloud is filtered to retain the point cloud within the region of interest. The specific processing is as follows:

[0045] PTS = {pts i |pts i ∈PTS o ,pts i ∈ROI}

[0046] Among them, PTS o For the original point cloud collection, pts i Let i be the i-th point in the original point cloud, and ROI be the region of interest. The ROI range is set to... PTS is the filtered point cloud set.

[0047] Preferably, the filtered lidar point cloud is subjected to raster filtering for dimensionality reduction, and the specific processing is as follows:

[0048] PTS V =voxel(PTS)

[0049] Here, `voxel()` is the rasterization function, the raster size is set to 0.1 meters, and the PTS value is... V This is the rasterized point cloud data.

[0050] In another implementation of the present invention, scene elements include the target category, center point location, three-dimensional size data, ground point cloud, ground model, and single target point cloud data; such as Figure 2 As shown, the preprocessed LiDAR point cloud data and camera image data are input into the scene element extraction module to obtain the scene elements of the target scene, including: processing the preprocessed camera image data through a trained deep learning model to obtain the target category, center point position, and three-dimensional size data in the camera image data; and performing ground point cloud extraction and non-ground point cloud clustering processing on the preprocessed LiDAR point cloud data to obtain ground point cloud, ground model, and single target point cloud data.

[0051] For example, such as Figure 4 As shown, a trained deep learning model is used to process the preprocessed image to obtain the target category (CLS), center point location (CTR), and 3D size data (TDS) of the image data. Ground point cloud extraction and non-ground point cloud clustering are then performed on the preprocessed point cloud data to obtain the ground point cloud (PTS). g Ground Model MP g Single target point cloud data PTS sg Outputs scene element information such as the category of the extracted target, center point location, 3D dimensions, ground point cloud, ground model, and single target point cloud.

[0052] Specifically, the steps for processing images using a trained deep learning model are as follows:

[0053] (1) Construct a monocular 3D detection model M based on deep learning dl ;

[0054] (2) The deep learning model M is trained using labeled data. dl ;

[0055] (3) Use the trained model M dl Make predictions about the image to obtain the prediction results;

[0056] (4) Use non-maximum values ​​to filter the prediction results of the image to obtain the final target category CLS, center point position CTR, and three-dimensional size data TDS;

[0057] Furthermore, the specific steps for extracting ground point clouds and clustering non-ground point clouds from point cloud data are as follows:

[0058] (1) Obtain the point cloud PTS after rasterization. V ;

[0059] (2) Using ground extraction algorithms to extract data from the raster point cloud PTS V Extracting ground point PTS g Non-ground point PTS ng ;

[0060] (3) Use the least squares method to analyze the ground point PTS g Processing and extracting the ground model; the extracted ground model is as follows:

[0061] ax + by + cz + d = 0

[0062] (4) Processing non-terrestrial point clouds using clustering methods (PTS) ng Extract the point cloud dataset PTS belonging to a single target. sg .

[0063] In another implementation of the present invention, the method for rapid scene reconstruction based on multi-layer raster maps further includes: performing reliability analysis on the detection results of camera image data, the results of ground point cloud extraction, and the results of non-ground point cloud clustering according to scene elements, and obtaining reliability analysis results; setting status labels for the detection results of camera image data, the results of ground point cloud extraction, and the results of non-ground point cloud clustering based on the reliability analysis results; and determining the reliability status of camera image data detection, ground point cloud extraction, and non-ground point cloud clustering through the reliability analysis results.

[0064] For example, such as Figure 5 As shown, the reliability of image detection results is assessed using the target's category, center point location, and 3D dimensions, and status flags are set accordingly. Ground point cloud and ground model data are used to assess the reliability of ground point cloud extraction results, and status flags are set accordingly. A single target point cloud is used to assess the reliability of non-ground point cloud clustering results, and status flags are set accordingly. The reliability status of image detection, ground point cloud extraction, and non-ground point cloud clustering is obtained, and a basic raster map is generated. l0 And the raster map of the detection results, including the center point map. CTR 3D map TDS Ground point map G Non-terrestrial point cloud map NG Output the generated multi-layer raster map {map CTR map TDS map G mapNG} and the reliability status results of scene elements {con CTR ,con TDS ,con G ,con NG}

[0065] In another implementation of the present invention, reliability analysis is performed on the detection results of camera image data, the extraction results of ground point clouds, and the clustering results of non-ground point clouds according to scene elements to obtain reliability analysis results. This includes: performing reliability analysis on the detection results of camera image data based on the target category, center point position, and three-dimensional size to obtain reliability analysis results of the detection results; performing reliability analysis on the extraction results of ground point clouds based on ground point clouds and ground models to obtain reliability analysis results of the extraction results; and performing reliability analysis on the clustering results of non-ground point clouds based on single target point clouds to obtain reliability analysis results of the clustering results.

[0066] For example, the specific steps for judging the reliability of image detection results are as follows:

[0067] (1) Obtain the target's category (CLS), center point location (CTR), and three-dimensional dimension data (TDS);

[0068] (2) Determine if the number of each type of target in CLS is 0. If all are 0, then con CTR ,con TDS If set to unreliable, proceed with the step of judging the reliability of the ground point cloud extraction results; otherwise, proceed with step (3).

[0069] (3) Calculate the distance d(CTR) between each pair of all elements in CTR. i CTR j ), and determine d(CTR) i CTR j )<d thresh d thresh Set it to 0.15. If none of these conditions are met, then con CTR Set to valid, otherwise set to invalid. Execute (4) after the status flags are set;

[0070] (4) Perform a judgment on TDS, as follows:

[0071]

[0072] If none of them are true, then con TDS Set to valid, otherwise set to invalid. After the status flag is set, proceed with the step of determining the reliability of the ground point cloud extraction results;

[0073] Furthermore, the reliability of the ground point cloud extraction results is assessed through the following steps:

[0074] (1) Obtain ground point cloud PTS g The extracted ground model ax+by+cz+d=0;

[0075] (2) Extract the feature vector of the ground point cloud, as follows:

[0076] {λ1,λ2,λ3}=PCA(PTS g )

[0077] (3) Perform the judgment, as follows:

[0078]

[0079] If all conditions are met, then execute (4); otherwise, set the flag con. G Invalid;

[0080] (4) Perform a judgment for each point, as follows:

[0081]

[0082] Count the number of point clouds that satisfy the above formula (num) gt And perform the judgment:

[0083] num gt >0.8*num all

[0084] If the condition is met, set the flag 'con'. G If valid, otherwise set the flag 'con'. G Invalid;

[0085] Furthermore, the specific steps for judging the reliability of non-terrestrial point cloud clustering results are as follows:

[0086] (1) Obtain the point cloud dataset PTS for a single target sg ;

[0087] (2) For each cluster cloud Extract feature vectors and count the number of point clouds. The counted number of point clouds is denoted as... The extracted feature vector is shown in the following formula:

[0088]

[0089] (3) Perform the following judgment for each cluster point cloud:

[0090]

[0091] If all conditions are met, then execute (4); otherwise, the flag con is set. NG Set to invalid;

[0092] (4) Perform a judgment on the number of point clouds in each point cloud cluster. If all conditions are met, then the flag bit con NG Set to valid, otherwise the con flag is enabled. NG Set to invalid.

[0093] In another implementation of the present invention, the multi-layer raster map includes a center point map, a 3D dimension map, a ground point map, and a non-ground point cloud map. The process involves inputting scene elements into a multi-layer map generation module to obtain the multi-layer raster map, including: generating a base raster map based on the multi-layer map generation module; finding the raster point in the base raster map that is closest to each center point in the center point location, setting `cond` to 1, and obtaining the center point map; finding the raster point in the base raster map that is closest to each element in the 3D dimension data, setting `cond` to 1, and simultaneously setting `cond` to 1 for raster points near the raster point that are inside the target, and obtaining the 3D dimension map; finding the raster point in the base raster map that is closest to each point cloud in the ground point cloud, setting `cond` to 1, and obtaining the ground point map; and finding the raster point in the base raster map that is closest to each point cloud in the single target point cloud data, setting `cond` to 1, and obtaining the non-ground point cloud map.

[0094] For example, the basic raster map is generated based on the multi-layer map generation module, and the specific steps are as follows:

[0095] (1) Obtain the range of ROI. To reduce computation, take x. min =-80, x max =80, y min =0, y max =80, z min =-2, z max =2;

[0096] (2) Divide the ROI range into ROIs with a step size of 1 mm to obtain ROI';

[0097] (3) Rasterize ROI' as follows:

[0098] ROI V =voxel(ROI')

[0099] (4) ROI V Expanding the dimension of (x, y, z) and filling it with 0s, yields the basic raster map. l0 (x,y,z,cond), in the basic raster map, cond is set to 0 for all values;

[0100] Further, a raster map of the detection results is generated, and the specific steps are as follows:

[0101] (1) Obtain the basic raster map. l0 (x,y,z,cond), center point location (CTR), 3D dimension data (TDS), ground point cloud (PTS) g Single target point cloud data PTS sg ;

[0102] (2) For each center point in the CTR, in the basic raster map map. l0 Find the nearest grid point in (x, y, z, cond), set cond to 1, and obtain the center point map. CTR (x,y,z,cond);

[0103] (3) For each element in the 3D dimension data TDS, in the basic raster map map. l0 Find the nearest grid point in (x, y, z, cond), set cond to 1, and also set the cond of all nearby grid points inside the target to 1, thus obtaining a 3D map. TDS The grid points inside the target satisfy the following conditions:

[0104]

[0105] Among them, (x o ,y o ,z o raster map based on ) l0 The grid point in (x,y,z,cond) that is closest to the corresponding element in TDS;

[0106] (4) Ground point cloud PTS g Each point cloud in the base raster map l0 Find the nearest grid point in (x, y, z, cond), set cond to 1, and obtain the ground point map. G ;

[0107] (5) PTS for single target point cloud data sg Each point cloud in the base raster map l0 Find the nearest grid point in (x, y, z, cond), set cond to 1, and obtain the non-ground point cloud map. NG .

[0108] In another implementation of the present invention, the grid points inside the target satisfy the following condition:

[0109]

[0110] Among them, (x o ,y o ,z o () is the grid point in the base grid map that is closest to the corresponding element in the 3D dimension data.

[0111] In another implementation of the present invention, a multi-layer raster map is input into a scene fast reconstruction module to obtain a reconstructed scene, including: the scene fast reconstruction module performs merging processing based on the center point map, the three-dimensional size map, the ground point map, and the non-ground point cloud map to generate map data for scene reconstruction.

[0112] For example, such as Figure 7 As shown, the reliability status of scene elements is obtained {con CTR ,con TDS ,con G ,con NG} and the generated scene raster map {map CTR map TDS map G map NG}, obtain the status flag of the ground point cloud extraction. If the extraction is successful, obtain the status flag of the non-ground point cloud clustering result; otherwise, re-obtain the reliability status of the scene elements and the generated scene raster map.

[0113] Preferably, such as Figure 6 As shown, the following steps are taken: First, obtain the status flag of the non-ground point cloud clustering result. If successful, obtain the generated non-ground point cloud map; otherwise, obtain the generated base raster map. Second, obtain the status flag of the image detection result. If successful, obtain the generated center point map; otherwise, obtain the generated base raster map. Third, obtain the status flag of the 3D detection result. If successful, obtain the generated 3D dimension map; otherwise, obtain the generated base raster map.

[0114] Furthermore, such as Figure 8 As shown, the generated multi-layer raster maps are merged to generate map data for scene reconstruction, and the reconstructed scene map is output.

[0115] Preferably, the generated multi-layer raster maps are merged, as follows:

[0116] (1) Obtain the basic raster map, center point map, 3D dimension map, and ground point cloud map based on the status flags, and denot them as {map}. CTR map TDS map G map NG};

[0117] (2) map CTR map TDS map G map NG In the}, the basic raster map is denoted as

[0118] (3) For {map CTR map TDS map G map NG The following steps will be taken to merge the elements:

[0119] map all =map l0 +Πmap li

[0120] Among them, map li ∈{map CTR map TDS map G map NG},and

[0121] By effectively processing and analyzing multi-layered raster maps, scenes can be reconstructed quickly and accurately, providing strong support for autonomous driving.

[0122] Specific embodiments of the invention have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0123] It should be noted that all directional indicators (such as up, down, left, right, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0124] In the description of this invention, the terms "first" and "second" are used only for convenience in describing different components or names, and should not be construed as indicating or implying a sequential relationship, relative importance, or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" and "second" may explicitly or implicitly include at least one of that feature.

[0125] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0126] It should be noted that although specific embodiments of the present invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of the present invention. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of the present invention.

[0127] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be an improper limitation of the embodiments of the present invention.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for fast reconstruction of a scene based on a multi-layer grid map, characterized in that, include: Collect lidar point cloud data and camera image data of the target scene; The lidar point cloud data and camera image data are preprocessed, including data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering. The preprocessed lidar point cloud data and camera image data are input into the scene element extraction module to obtain the scene elements of the target scene; the scene elements include the target category, center point position, three-dimensional size data, ground point cloud, ground model and single target point cloud data; Based on the scene elements, reliability analysis was performed on the detection results of camera image data, the results of ground point cloud extraction, and the results of non-ground point cloud clustering, and the reliability analysis results were obtained. The scene elements are input into the multi-layer map generation module to obtain a multi-layer raster map; The multi-layer raster map is input into the scene fast reconstruction module. The scene fast reconstruction module merges the center point map, the three-dimensional size map, the ground point map, and the non-ground point cloud map to generate map data for scene reconstruction, thus obtaining the reconstructed scene. The multi-layer raster map includes a center point map, a three-dimensional dimension map, a ground point map, and a non-ground point cloud map; The scene elements are input into the multi-layer map generation module to obtain a multi-layer raster map, including: A basic raster map is generated based on the multi-layer map generation module; finding the grid point closest to each of the center point locations in the base grid map, and cond setting to 1, obtaining a center point map; finding the grid point closest to each element in the three-dimensional size data in the basic grid map, setting the value of the grid point to 1, and setting the values of the grid points near the grid point to 0 to obtain a three-dimensional size map. cond setting to 1, while setting the values of the grid points near the grid point inside the target to 0 cond setting to 1, obtaining a three-dimensional size map; finding a grid point closest to each point cloud in the ground point cloud in the base grid map, and obtaining a ground point map by setting the grid point to 1 cond finding a grid point closest to each point cloud in the ground point cloud in the base grid map, and obtaining a ground point map by setting the grid point to 1 Finding the grid point closest to each point cloud in the single target point cloud data in the base grid map, and setting the value of the grid point to 1 to obtain a non-ground point cloud map. cond Setting to 1 to obtain a non-ground point cloud map.

2. The method of claim 1, wherein, The collected lidar point cloud data and camera image data of the target scene include: Acquire the LiDAR point cloud data of the target scene and the timestamp of the LiDAR point cloud; Acquire camera image data of the target scene and the timestamp of the camera image; The preprocessing of the lidar point cloud data and camera image data includes data parsing, time synchronization, camera calibration, point cloud filtering, and raster filtering, including: Determine if the time difference between the LiDAR and camera exceeds a threshold. If it does, the data is invalid, and the LiDAR point cloud data and camera image data of the target scene are re-acquired. If the time difference does not exceed a threshold, then... Perform external parameter calibration on the camera; Filter the lidar point cloud and retain the point cloud within the region of interest; The preserved lidar point cloud is subjected to dimensionality reduction by raster filtering.

3. The method of claim 1, wherein, The process of inputting preprocessed LiDAR point cloud data and camera image data into the scene element extraction module to obtain scene elements of the target scene includes: The pre-processed camera image data is processed by a trained deep learning model to obtain the target category, center point location, and three-dimensional size data in the camera image data. The preprocessed lidar point cloud data is subjected to ground point cloud extraction and non-ground point cloud clustering to obtain ground point cloud, ground model and single target point cloud data.

4. The method of claim 1, wherein, After obtaining the reliability analysis results, the following is also included: Based on the reliability analysis results, status labels are set for the detection results of the camera image data, the results of the ground point cloud extraction, and the results of the non-ground point cloud clustering, respectively. Based on the reliability analysis results, the reliability status of the camera image data detection, the ground point cloud extraction, and the non-ground point cloud clustering is determined.

5. The method of claim 1, wherein, The reliability analysis is performed on the detection results of the camera image data, the ground point cloud extraction results, and the non-ground point cloud clustering results based on the scene elements, respectively, to obtain the reliability analysis results, including: The reliability analysis results of the detection results of the camera image data are obtained by performing a reliability analysis on the target category, the center point position and the three-dimensional size; Based on the ground point cloud and the ground model, a reliability analysis is performed on the ground point cloud extraction results to obtain the reliability analysis results of the extraction results; The reliability analysis results of the clustering results of the non-ground point cloud are obtained by performing a reliability analysis on the clustering results based on the single target point cloud.

6. The method of claim 1, wherein, The grid points inside the target satisfy the following condition: wherein, is the grid point in the base grid map that is closest to the corresponding element in the three-dimensional dimension data.

Images