A dual-lidar-based map construction method, device, equipment, and medium

Abstract

The present application provides a dual-lidar-based map construction method, apparatus, device, and medium, relating to the field of autonomous driving technology. The method comprises: obtaining a final mapping pose based on first point cloud data and the initial pose of a first lidar; converting the first point cloud data into a map coordinate system based on the final mapping pose to obtain a first point cloud map; converting the second point cloud data collected by a second lidar into a map coordinate system based on the final mapping pose and calibrated external parameters to obtain a second point cloud map; generating an intensity map based on the first point cloud map and the second point cloud map; inputting the intensity map into a semantic segmentation model to obtain semantic labels for multiple pixels; clustering pixels with the same semantic label and spatially adjacent to each other into semantic elements, adding the semantic elements to the intensity map, and generating a semantic map. The dual-lidar on the roof and at the rear of the vehicle can collect information about the surrounding environment and road surface, which helps to generate a high-precision semantic map.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology and provides a map construction method, device, equipment and medium based on dual laser radars. Background Art

[0002] With the rapid development of autonomous driving technology, high-precision maps play a vital role in environmental perception, path planning, and vehicle positioning. Existing mapping solutions rely on lidar to collect point cloud data of the surrounding environment, calculate the mapping pose, and then generate a point cloud map based on the mapping pose. Finally, manual work is performed to create a high-precision semantic map based on the point cloud map.

[0003] However, existing mapping solutions mainly rely on a lidar installed on the roof for mapping, which can only obtain information about the vehicle's surrounding environment. There is a blind spot in the collection of information about the nearby ground, resulting in low accuracy of the final constructed map. Summary of the Invention

[0004] The present application provides a dual-lidar-based map construction method, apparatus, device, and medium, which are used to solve the problem of low map accuracy in existing mapping solutions due to the lack of road surface information.

[0005] In a first aspect, a dual-lidar-based map construction method is provided, which is applied to a processor of a target vehicle, wherein the target vehicle also includes a first lidar and a positioning device mounted on the roof, and a second lidar mounted diagonally behind the vehicle; the method comprises:

[0006] Obtaining a final mapping pose based on first point cloud data and an initial pose of the first laser radar; the first point cloud data is collected by the first laser radar; and the initial pose is collected by the positioning device;

[0007] According to the final mapping pose, converting the first point cloud data into a map coordinate system to obtain a first point cloud map;

[0008] According to the final mapping pose and the calibrated extrinsic parameters, convert the second point cloud data collected by the second lidar into a map coordinate system to obtain a second point cloud map;

[0009] Generate an intensity map based on the first point cloud map and the second point cloud map;

[0010] Input the intensity map into a semantic segmentation model to obtain semantic labels for multiple pixels;

[0011] Pixels with the same semantic label and spatially adjacent to each other are clustered into semantic elements, and the semantic elements are added to the intensity map to generate a semantic map.

[0012] Optionally, obtaining a final mapping pose based on the first point cloud data and the initial pose of the first lidar includes:

[0013] extracting line features and surface features from the first point cloud data;

[0014] Optimizing the initial pose of the first laser radar using the line feature and the surface feature as constraints to obtain an initial mapping pose;

[0015] The initial mapping pose is optimized through loop closure detection to obtain the final mapping pose.

[0016] Optionally, after obtaining a final mapping pose based on the first point cloud data and the initial pose of the first lidar, the method includes:

[0017] According to the final mapping pose, the line features and the surface features are respectively converted into a map coordinate system;

[0018] A grid map is generated according to the statistical information of the line features and the statistical information of the surface features.

[0019] Optionally, extracting line features and surface features from the first point cloud data includes:

[0020] Eliminating the point cloud of the dynamic target from the first point cloud data to obtain static point cloud data;

[0021] Performing line bundle processing on the static point cloud data, and calculating the curvature of each point cloud on each line bundle;

[0022] If the curvature of any point cloud is greater than the line feature threshold, the point is determined to belong to the line feature; if the curvature of any point cloud is less than the surface feature threshold, the point is determined to belong to the surface feature.

[0023] Optionally, generating an intensity map according to the first point cloud map and the second point cloud map includes:

[0024] Stitching and fusing the first point cloud map and the second point cloud map to obtain a stitched point cloud map;

[0025] An intensity map is generated according to the heights and reflectivities of all point clouds in the stitched point cloud map.

[0026] Optionally, generating an intensity map according to the heights and reflectivities of all point clouds in the spliced point cloud map includes:

[0027] For a first point cloud in the stitched point cloud map, determining a second point cloud having a minimum height difference with the first point cloud;

[0028] If the height difference between the first point cloud and the second point cloud is less than a height threshold, then calculating the average height and average reflectivity of the first point cloud and the second point cloud, setting the height of the first point cloud and the height of the second point cloud to the average height, and setting the reflectivity of the first point cloud and the reflectivity of the second point cloud to the average reflectivity, until all point clouds in the spliced point cloud map are traversed to obtain a final point cloud map;

[0029] Determining a target point cloud that meets a height threshold range and a reflectivity threshold range from the final point cloud map;

[0030] Generate an intensity map in HSV format based on the height range and reflectivity range corresponding to the target point cloud.

[0031] Optionally, generating an intensity map in HSV format according to the height range and reflectivity range corresponding to the target point cloud includes:

[0032] Normalized mapping is performed according to the height range corresponding to the target point cloud to obtain the H value;

[0033] Fix the S value of each point cloud in the target point cloud to 1;

[0034] Determine the V value of the point cloud with a reflectivity within the reflectivity range in the target point cloud as 1, and determine the V value of the point cloud with a reflectivity outside the reflectivity range as 0;

[0035] Generate an intensity map based on the HSV values of all point clouds in the target point cloud.

[0036] In a second aspect, a dual-lidar-based map construction device is provided, which is disposed in a processor of a target vehicle, wherein the target vehicle further includes a first lidar and a positioning device mounted on the roof, and a second lidar mounted diagonally behind the vehicle; the device includes:

[0037] A posture acquisition module, configured to obtain a final mapping posture based on first point cloud data and an initial posture of the first laser radar; the first point cloud data is collected by the first laser radar; and the initial posture is collected by the positioning device;

[0038] A first point cloud map acquisition module is configured to convert the first point cloud data into a map coordinate system according to the final mapping pose to obtain a first point cloud map;

[0039] A second point cloud map acquisition module is configured to convert the second point cloud data collected by the second lidar into a map coordinate system according to the final mapping pose and the calibration extrinsic parameters to obtain a second point cloud map;

[0040] An intensity map generation module, configured to generate an intensity map based on the first point cloud map and the second point cloud map;

[0041] A semantic segmentation module is used to input the intensity map into a semantic segmentation model to obtain semantic labels for multiple pixels;

[0042] The semantic map generation module is used to cluster pixels with the same semantic label and adjacent to each other in space into semantic elements, add the semantic elements to the intensity map, and generate a semantic map.

[0043] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the map construction method based on dual laser radars described in the first aspect.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and a processor executes the computer program to implement the map construction method based on dual laser radars described in the first aspect.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] This application utilizes dual laser radars, including a first laser radar mounted on the vehicle's roof and a second laser radar mounted diagonally behind the vehicle. The laser beam emitted by the first laser radar has a longer field of view, enabling it to collect first point cloud data of the vehicle's surroundings and generate a first point cloud map based on the first point cloud data. The second laser radar projects a denser laser beam onto the road surface near the vehicle, while maintaining a shorter viewing distance. This allows it to collect second point cloud data of road surface information. In the second point cloud map generated based on the second point cloud data, semantic features of road traffic, such as lane markings, turn arrows, and zebra crossings, are more clearly visible. Combining the first and second point cloud maps addresses the lack of road surface information in existing mapping solutions, enabling the subsequent generation of high-precision semantic maps. Furthermore, an intensity map is generated based on the first and second point cloud maps. By inputting the intensity map into a semantic segmentation model and processing it in milliseconds, semantic labels for each laser point are obtained. These semantic labels are then clustered to obtain semantic elements. This significantly reduces the time and labor costs associated with manually generating high-precision semantic maps based on intensity maps, improving the efficiency of semantic map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0048] Figure 1 A dual-lidar-based map construction system provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of an installation of a dual laser radar provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of a process for constructing a map based on dual laser radars provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the constraints of the point-to-line feature provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the constraints of the point-to-surface feature provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of the storage structure of a grid map provided in an embodiment of the present application;

[0054] Figure 7A schematic diagram of an intensity map provided in an embodiment of the present application;

[0055] Figure 8 A schematic diagram of a semantic map provided in an embodiment of the present application;

[0056] Figure 9 A schematic diagram of the structure of a dual-lidar-based map construction device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Unless there is a conflict, the embodiments in the present application and the features in the embodiments can be combined with each other in any way. In addition, although a logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in an order different from that here.

[0058] Existing mapping solutions primarily rely on a roof-mounted lidar (LiDAR). The LiDAR transmits a pulse that strikes an object and then returns. Based on the principle of laser ranging, the time difference between the transmission and reception times is calculated to determine the distance between the object and the LiDAR. Multi-line LiDARs (such as 32-line and 64-line LiDARs) use multiple laser emitters operating simultaneously. These emitters are typically arranged vertically and rotate horizontally together. Each rotation of the multi-line LiDAR within a certain period of time produces a frame of point cloud data. Mapping poses are calculated based on the point cloud data collected by the LiDAR. A point cloud map is then generated based on this pose. Finally, a high-precision semantic map is manually constructed from the point cloud map. However, a single roof-mounted LiDAR only captures information about the vehicle's surroundings and has a blind spot in collecting information about the nearby ground, resulting in low accuracy in the resulting semantic map.

[0059] In view of this, the embodiments of the present application provide a dual-lidar-based map construction method, which is applied to the processor of the target vehicle. The following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios described below are only for the purpose of illustrating the embodiments of the present application and are not limiting. During the specific implementation process, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0060] Please refer to Figure 1, a map construction system based on dual laser radars provided in an embodiment of the present application, the map construction system is set in the target vehicle, the map construction system includes a first laser radar, a second laser radar, a positioning device and a processor, and the processor is respectively communicated with the positioning device, the first laser radar and the second laser radar.

[0061] The first laser radar is installed on the roof of the target vehicle to collect first point cloud data of the vehicle's surroundings and send it to the processor. The second laser radar is installed at the diagonal rear of the target vehicle to collect second point cloud data of the road surface and send it to the processor.

[0062] The positioning device is used to collect the initial position of the first lidar and send it to the processor. The positioning device can be a global positioning system (GPS), an inertial measurement unit (IMU), or an integrated navigation system.

[0063] The processor is used to construct a map based on the initial pose of the first lidar, the first point cloud data and the second point cloud data. The specific map construction method will be introduced below.

[0064] Please refer to Figure 2 , which is a schematic diagram of the installation of a dual laser radar according to an embodiment of the present application. As can be seen, the first laser radar is mounted on the roof of the target vehicle, with an angle of 0 with respect to the vertical. The second laser radar is mounted diagonally behind the target vehicle, with an angle of θ with respect to the vertical. The value of θ ranges from 40 degrees to 55 degrees. At the same time, the rear of the vehicle must not block the laser beam emitted by the second laser radar.

[0065] In a possible embodiment, the first laser radar is a 64-line laser radar and the second laser radar is a 32-line laser radar. Compared with using 64-line laser radars for both laser radars, hardware costs can be relatively saved.

[0066] The following combination Figure 1 A dual-lidar-based map building system for Figure 3 The map construction method based on dual lidar is introduced as shown in FIG.

[0067] S301: Obtain a final mapping pose based on the first point cloud data and the initial pose of the first lidar.

[0068] Specifically, after collecting first point cloud data, the first lidar sends it to the processor. The first point cloud data is used to describe the target vehicle's surrounding environment and is in the first lidar coordinate system (i.e., the local coordinate system in which the first lidar resides). After collecting the initial pose of the first lidar, the positioning device sends it to the processor. The initial pose refers to the pose of the first lidar relative to the map coordinate system when the target vehicle is activated, as captured by the positioning device at the same moment. This is typically represented as a coordinate system transformation matrix, T0, which includes a rotation matrix (provided by an IMU, GPS, or manual calibration) and a translation vector (e.g., the GNSS coordinates at vehicle activation).

[0069] The processor can optimize the initial pose of the first laser radar based on the first point cloud data to obtain the final mapping pose. The final mapping pose refers to the real-time pose of the first laser radar relative to the map coordinate system during continuous motion, which is usually expressed as the coordinate system transformation matrix T m The point cloud map is composed of multiple keyframes. During the splicing process, each keyframe will undergo coordinate transformation based on its position and posture in the map coordinate system. This position and posture is the mapping pose of the keyframe.

[0070] In the embodiment of the present application, the first laser radar (64-line laser radar) has a large scanning range and extracts a rich variety of line and surface features, which can provide sufficient constraints. For example, line features such as tree trunks and light poles provide horizontal constraints, while surface features such as road surfaces and walls provide vertical constraints. The second laser radar (32-line laser radar) has a smaller scanning range, with the ground accounting for the majority. It extracts fewer line and surface features based on geometric relationships. Therefore, the mapping pose is calculated based solely on the first point cloud data collected by the first laser radar. Compared to existing calculation methods based on annularly distributed multi-lidar point cloud data, this reduces the amount of data and improves mapping efficiency.

[0071] In a possible embodiment, the specific steps of S301 include: extracting line features and surface features from the first point cloud data; optimizing the initial pose of the first lidar with the line features and surface features as constraints to obtain an initial mapping pose; and optimizing the initial mapping pose through loop closure detection to obtain a final mapping pose.

[0072] Line features are one-dimensional geometric structures composed of a series of continuous points in point cloud data, representing edges, corners, or slender objects (such as tree trunks and light poles) in the scene. Surface features are two-dimensional planar structures composed of a large number of points in point cloud data, representing flat surfaces (such as roads and walls) in the scene.

[0073] First, create constraints for point-to-line features, such as Figure 4As shown, assuming the current point is i, the two points on the line feature are j and l ， Then the distance from point i to line jl can be expressed as .

[0074] Secondly, establish point-to-surface feature constraints, such as Figure 5 As shown, assuming the current point is i ， The three points on the surface feature are j 、 l and m, then the distance from point i to surface jlm can be expressed as .

[0075] Then, based on the constraints of point-to-line features and point-to-surface features, an objective function is established, and a least squares problem is constructed in combination with the initial pose. The Gauss-Newton method or the Levenberg-Marquardt method is used for multiple iterations until the solution converges to the set threshold. The calculation is stopped and the initial mapping pose is obtained.

[0076] Finally, the initial mapping pose is further optimized through loop detection to obtain the final mapping pose. Specifically, when the target vehicle moves beyond a certain distance (such as 5m) or rotates by an angle (such as 30°), the current frame is marked as a keyframe and its features (including line features and surface features) are extracted. The features of the current keyframe are compared with those of the historical keyframes. If the feature matching degree exceeds a threshold (such as 80%), it is determined to be a loop, that is, the target vehicle has returned to the place it passed before. The relative pose of the two frames can be obtained through point cloud feature registration, which will deviate from the pose calculated by odometry data. In this case, the relative pose obtained by registration can be used as a constraint, and the deviation can be adjusted to each previous node for even distribution, thus achieving pose optimization.

[0077] In an embodiment of the present application, by optimizing the initial pose of the LiDAR using line and surface features as constraints, the impact of initial pose estimation errors on the mapping results can be effectively reduced. Since pose estimation is a recursive process, that is, the pose of the previous frame is used to calculate the pose of the current frame, cumulative errors will occur as the pose is passed from frame to frame. In this embodiment of the present application, the initial mapping pose is further optimized through loop detection, and the pose constraint information at the loop is used to optimize the pose, significantly reducing the impact of error accumulation and ensuring the accuracy of the mapping pose.

[0078] In a possible embodiment, the step of extracting line features and surface features from the first point cloud data includes: removing the point cloud of the dynamic target from the first point cloud data collected by the first lidar to obtain static point cloud data; performing line bundle processing on the static point cloud data, and calculating the curvature of each point cloud on each line bundle; if the curvature of any point cloud is greater than the line feature threshold, it is determined that the point belongs to the line feature; if the curvature of any point cloud is less than the surface feature threshold, it is determined that the point belongs to the surface feature.

[0079] Specifically, after acquiring the first point cloud data, the processor first uses a deep learning-based object detection model (such as PointPillar or CenterPoint) to detect dynamic targets, including pedestrians, vehicles, and temporary obstacles. Point clouds representing dynamic targets are then removed from the first point cloud data to obtain static point cloud data.

[0080] Then, according to the nth point cloud to the nth to last point cloud on the harness, the curvature of each point cloud is calculated using the following formula:

[0081]

[0082] in, c i,j Indicates the k The first i The curvature of the point cloud, Indicates the k The first i The horizontal coordinate of the point cloud, j= 1, 2, 3, ...,n , preferably, n =10.

[0083] Calculated c i,1 、 c i,2 ,..., c i,n , sort these n curvatures and take the median as the first k The first i The curvature of the point cloud.

[0084] Finally, each point cloud on the line bundle is divided into six segments, and line and surface features are selected in descending order of point cloud curvature. If the curvature of any point cloud is greater than the line feature threshold, the point is considered a line feature; if the curvature of any point cloud is less than the surface feature threshold, the point is considered a surface feature. If the curvature of any point cloud is greater than the surface feature threshold but less than the line feature threshold, the point cloud is discarded without further processing. The line feature threshold is generally set to 0.866, and the surface feature threshold is generally set to 0.5.

[0085] In the embodiment of the present application, by eliminating the point cloud data of dynamic targets and retaining the static point cloud data, it is helpful to eliminate the noise or interference caused by moving objects (such as pedestrians, vehicles, etc.), which can greatly improve the accuracy of subsequent analysis and mapping and avoid unnecessary errors. And by processing the line bundle and calculating the curvature of each point cloud, it is possible to distinguish line features and surface features based on the geometric features of different points. Line features and surface features have different geometric expressions. Curvature, as an important feature indicator, helps to accurately distinguish between these two features and improve the accuracy of subsequent analysis.

[0086] In a possible embodiment, after executing 301, the method includes: converting the line features and the surface features into a map coordinate system according to the final mapping pose, and generating a grid map according to the statistical information of the line features and the statistical information of the surface features.

[0087] In the specific implementation process, since the first point cloud data is in the first lidar coordinate system, the line features and surface features extracted from the first point cloud data are also in the first lidar coordinate system. Therefore, based on the final mapping pose, the line features and surface features can be converted from the first lidar coordinate system to the map coordinate system. The calculation formula is as follows:

[0088] P m =T ml × P l

[0089] in, P m is the coordinate value of the line feature or surface feature in the map coordinate system, P l is the coordinate value of the line feature or surface feature in the first laser radar coordinate system, T ml The final mapping pose.

[0090] Furthermore, the grid map is obtained by statistically analyzing the Gaussian distribution of point cloud data corresponding to line features and surface features in the grid. The storage structure of the grid map is as follows: Figure 6 As shown in the figure, a raster map consists of different layers, including line feature layers and area feature layers. Each layer consists of several tiles, and each tile consists of several grids.

[0091] In this embodiment, the tile side length for line features is 60 meters, the tile side length for surface features is 60 meters, the grid side length for line features is 1 meter, and the grid side length for surface features is 1 meter. Each grid contains relevant statistical information about the line feature or surface feature in the map coordinate system, such as mean, covariance, eigenvectors, eigenvalues, number of point clouds, and average reflectivity.

[0092] The formula for calculating the mean is as follows:

[0093]

[0094] in, Represents the three-dimensional coordinates of the point cloud in each grid, represents the mean of each grid, and n represents the number of point clouds in each grid.

[0095] The formula for calculating the covariance matrix is as follows:

[0096]

[0097] Among them, the covariance matrix represents Cov.

[0098] The calculation formulas for eigenvectors and eigenvalues are as follows:

[0099]

[0100] in, represents the covariance matrix, E represents the identity matrix, represents the eigenvalue, represents the feature vector.

[0101] The formula for calculating the average reflectivity is as follows:

[0102]

[0103] Among them, I k Represents the reflectivity of the point cloud in each grid, M I Represents the average reflectance value for each raster.

[0104] In the embodiments of this application, the grid map only stores the line and surface features extracted from each frame of the point cloud, rather than the entire point cloud data. This reduces the data size and reduces the memory and hardware storage requirements of the device. It also facilitates fast online positioning matching and reduces the computational complexity of the algorithm. Furthermore, the information stored in each grid in the grid map is statistical information about the feature point cloud, which provides high reliability and robustness when the surrounding environment changes.

[0105] The grid map can be used in the positioning module of unmanned driving. Based on the predicted position and posture of the target vehicle, the positioning module obtains the local map of nearby line and surface features from the grid map, extracts line and surface features from the laser frame scanned in real time by the target vehicle, and then aligns them to obtain the corrected position and posture, and predicts the position and posture at the next moment.

[0106] S302: According to the final mapping pose, convert the first point cloud data into a map coordinate system to obtain a first point cloud map.

[0107] Specifically, the point cloud of the dynamic target can be removed from the first point cloud data to obtain static point cloud data. Since the first point cloud data is in the first lidar coordinate system, the static point cloud data is also in the first lidar coordinate system. Based on the final mapping pose, the static point cloud data can be converted from the first lidar coordinate system to the map coordinate system. The calculation formula is as follows:

[0108] P' m =T ml × P' l1

[0109] in, P' m is the coordinate value of the static point cloud data in the map coordinate system, P' l1 is the coordinate value of the static point cloud data in the first laser radar coordinate system, T ml The final mapping pose.

[0110] S303: According to the final mapping pose and the calibrated extrinsic parameters, the second point cloud data collected by the second lidar is converted into a map coordinate system to generate a second point cloud map.

[0111] Specifically, after collecting the second point cloud data, the second laser radar sends it to the processor, where the second point cloud data is used to describe the road ground information near the target vehicle, and the second point cloud data is in the second laser radar coordinate system (that is, the local coordinate system where the second laser radar is located).

[0112] The processor can convert the second point cloud data from the second lidar coordinate system to the first lidar coordinate system based on the calibration extrinsic parameters, and then convert the second point cloud data from the first lidar coordinate system to the map coordinate system based on the final mapping pose to obtain a second point cloud map. The calibration extrinsic parameters refer to the transformation matrix from the second lidar coordinate system to the first lidar coordinate system, including the rotation matrix and the translation vector.

[0113] The calculation formula is as follows:

[0114] P" m =T ml ×T×P" l2

[0115] in, P" m is the coordinate value of the second point cloud data in the map coordinate system, P" l2 is the coordinate value of the second point cloud data in the second lidar coordinate system, T is Calibrate external parameters, T ml The final mapping pose.

[0116] S304: Generate an intensity map based on the first point cloud map and the second point cloud map.

[0117] In a possible embodiment, the first point cloud map and the second point cloud map are stitched and fused to obtain a stitched point cloud map; and an intensity map is generated according to the heights and reflectivities of all point clouds in the stitched point cloud map.

[0118] Specifically, the first and second point cloud maps are both in the map coordinate system. Feature points (such as corners and planar features) are extracted from the two point cloud maps. These feature points are then used for registration to ensure that the positions and orientations of the two point cloud maps are consistent. The two point cloud maps are then stitched together to remove duplicate point cloud data and reduce noise using a filtering algorithm to obtain the stitched point cloud map.

[0119] In a stitched point cloud map, each point has a corresponding height and reflectivity. An intensity map can be generated based on the heights and reflectivities of all the points in the stitched point cloud map. The height of each point refers to its vertical distance relative to the LiDAR installation location or the ground, and can be determined using the Z value in 3D coordinates. Reflectivity refers to the intensity of the laser signal reflected back to the LiDAR after hitting the surface of an object. An intensity map is a two-dimensional map created by clustering and classifying the point cloud data collected by the LiDAR.

[0120] In a possible embodiment, the step of generating an intensity map according to the heights and reflectivities of all point clouds in the spliced point cloud map includes:

[0121] For the first point cloud in the spliced point cloud map, determine the second point cloud with the smallest height difference with the first point cloud; if the height difference between the first point cloud and the second point cloud is less than the height threshold, calculate the average height and average reflectivity of the first point cloud and the second point cloud, set the height of the first point cloud and the height of the second point cloud to the average height, and set the reflectivity of the first point cloud and the reflectivity of the second point cloud to the average reflectivity, until each point cloud in the spliced point cloud map is traversed to obtain the final point cloud map; determine the target point cloud that meets the height threshold range and reflectivity threshold range from the final point cloud map; generate an intensity map in HSV format according to the height range and reflectivity range corresponding to the target point cloud, such as Figure 7 shown.

[0122] Specifically, the generation process of the Intensity map is as follows:

[0123] S1.1. Traverse and compare the height difference between each point cloud and the height of the rest of the point clouds in the spliced point cloud map, and obtain the minimum height difference d min And the corresponding two point clouds (such as P1 and P2).

[0124] S1.2. Set the height threshold to T d , if d min >T d , then jump out of the loop and traverse the next point cloud again. If d min <T d , then calculate the average height and average reflectivity of the two point clouds P1 and P2, set the heights of the two point clouds P1 and P2 to the average height of the two point clouds, and set the reflectivity of P1 and P2 to the average reflectivity of the two point clouds.

[0125] S1.3. Repeat steps S1.1 and S1.2 until all point clouds in the spliced point cloud map are traversed to obtain the final point cloud map.

[0126] S1.4. Set threshold range, including height threshold range and reflectivity threshold range , compare the height and reflectivity of each point in the final point cloud map in turn, and determine the height range of the target point cloud that meets the threshold range. and reflectivity range is ;

[0127] S1.5, according to the height range corresponding to the target point cloud and reflectivity range is , generate an Intensity map in HSV format.

[0128] In an embodiment of the present application, by calculating the average value of the height and reflectivity of adjacent point clouds, the point cloud data can be smoothed, noise and outliers can be reduced, and the processed point cloud data is more uniform, avoiding errors caused by high differences or abnormal reflectivity. By setting height thresholds and reflectivity thresholds, point clouds that do not meet expectations (such as noise, outliers, etc.) can be effectively screened out, reducing interference, thereby improving the accuracy of the final point cloud map.

[0129] In a possible embodiment, the step of generating an intensity map in HSV format according to the height range and reflectivity range corresponding to the target point cloud includes:

[0130] Perform normalized mapping according to the height range corresponding to the target point cloud to obtain the H value of each point cloud;

[0131] Fix the S value of each point cloud in the target point cloud to 1;

[0132] The V value of the point cloud with a reflectivity within the reflectivity range in the target point cloud is determined to be 1, and the V value of the point cloud with a reflectivity outside the reflectivity range is determined to be 0;

[0133] Generate an intensity map based on the HSV values of all point clouds in the target point cloud.

[0134] In the specific implementation process, the calculation formula of the HSV value of each point cloud is as follows:

[0135]

[0136] in, H k Indicates the first k Point cloud H value, S k Indicates the k Point cloud S value, V k Indicates the k Point cloud V value, Z max is the maximum value of the height range, Z min is the minimum value of the height range, Z k For the k The height value of the point cloud, I k For the k The reflectivity value of the point cloud, I max is the maximum value of the reflectivity range, I minis the minimum value of the reflectivity range, Represents logical OR.

[0137] In this embodiment, by fixing the S value to 1 and maintaining maximum color saturation, the calculation process is simplified and unnecessary complexity is avoided. By mapping the point cloud's height (H value), reflectivity (V value), and fixed saturation (S value) to color space, the characteristics of different points can be more intuitively viewed.

[0138] S305: Input the intensity map into the semantic segmentation model to obtain semantic labels for multiple pixels.

[0139] In the specific implementation process, a large number of labeled intensity maps are used to train semantic segmentation models based on deep learning (such as RangNet++). The performance of the model is continuously evaluated during training, and the model is adjusted according to the evaluation results to ensure the best segmentation effect.

[0140] For the newly generated Intensity map, each frame of local map data in the Intensity map is preprocessed and input into the trained semantic segmentation model to obtain the semantic label of each pixel, such as lane lines, stop lines, ground guide arrows, traffic lights, etc.

[0141] S306: Cluster pixels with the same semantic label and spatially adjacent to each other into semantic elements, add the semantic elements to the intensity map, and generate a semantic map.

[0142] In the specific implementation process, a regional clustering algorithm (such as DBSCAN, K-means, etc.) is used to cluster pixels with the same semantic label and spatial proximity into one category, which is used as a semantic element (such as lane lines, stop lines, ground guide arrows or traffic lights). Each semantic element is added to the corresponding position in the Intensity map to generate a high-precision semantic map, such as Figure 8 The semantic map includes traffic semantic elements such as lane lines, stop lines, ground guide arrows, and traffic lights.

[0143] High-precision semantic maps can be used in the perception, planning, and prediction modules of autonomous driving. For example, when the perception module detects an obstacle, it can map the obstacle onto the semantic map based on the target vehicle's current position. This determines whether the obstacle is in the target vehicle's path and, if so, avoids or stops the vehicle. The planning module uses the topological relationships of lanes in the semantic map to generate the optimal autonomous driving route. The prediction module can adopt different driving strategies based on the semantic map mapping of the position, speed, and driving direction of oncoming vehicles.

[0144] In summary, the present application provides a map construction method based on dual laser radars. The laser beam emitted by the 64-line laser radar on the roof has a long field of view, which can collect rich information about the vehicle's surrounding environment for the calculation of the map pose. The 32-line laser radar at the rear of the vehicle shoots relatively dense laser beams at the road surface near the vehicle. At the same time, the line of sight is relatively close, and the ground point cloud reflectivity is high, which can collect rich road surface information such as lane lines, turn arrows, and zebra crossings. Compared with the existing mapping solutions based on a single laser radar, the present application combines the point cloud data of the 64-line laser radar on the roof and the 32-line laser radar at an angle. The semantic features of road traffic such as lane lines, turn arrows, and zebra crossings in the generated point cloud map are clearer and more obvious, which facilitates the subsequent generation of high-precision semantic maps. In addition, the present application simultaneously realizes the construction of raster maps and semantic maps, wherein the raster map can be used for the positioning module in unmanned driving, and the semantic map can be used for the perception, planning, and prediction modules in unmanned driving.

[0145] Based on the same invention concept, please refer to Figure 9 The present application also provides a dual-lidar-based map construction device, which is provided in a processor of a target vehicle. The target vehicle also includes a first lidar and a positioning device mounted on the roof, and a second lidar mounted at an angle behind the vehicle. The device includes:

[0146] A mapping pose acquisition module, configured to obtain a final mapping pose based on first point cloud data and an initial pose of a first laser radar; the first point cloud data is collected by the first laser radar; and the initial pose is collected by a positioning device;

[0147] A first point cloud map acquisition module is used to convert the first point cloud data into a map coordinate system according to the final mapping pose to obtain a first point cloud map;

[0148] A second point cloud map acquisition module is used to convert the second point cloud data collected by the second lidar into a map coordinate system according to the final mapping pose and the calibration external parameters to obtain a second point cloud map;

[0149] An intensity map generation module, configured to generate an intensity map based on the first point cloud map and the second point cloud map;

[0150] The semantic segmentation module is used to input the intensity map into the semantic segmentation model to obtain semantic labels for multiple pixels;

[0151] The semantic map generation module is used to cluster pixels with the same semantic labels and spatially adjacent to each other into semantic elements, add the semantic elements to the intensity map, and generate a semantic map.

[0152] It should be noted that the processor in this embodiment corresponds one-to-one to each step in the map construction method based on dual laser radars in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned map construction method based on dual laser radars, and will not be repeated here.

[0153] Based on the same inventive concept, the present application also provides a computer device, which includes a processor, a memory, and a computer program stored in the memory. When the computer program is executed by the processor, it implements the aforementioned dual-lidar-based map construction method.

[0154] Based on the same inventive concept, the present application also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the aforementioned dual-lidar-based map construction method is implemented.

[0155] In some embodiments, the computer-readable storage medium may be a memory device such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface mount memory, optical disk, or CD-ROM; or various devices including any one or any combination of the above memories. The computer may be various computing devices including smart terminals and servers.

[0156] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0157] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions).

[0158] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0159] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0160] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0161] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a multimedia terminal device (which can be a mobile phone, a computer, a television receiver, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0162] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A map construction method based on dual laser radars, characterized in that: In a processor applied to a target vehicle, the target vehicle further includes a first laser radar and a positioning device mounted on the roof, and a second laser radar mounted at an angle behind the vehicle; the method includes: Obtaining a final mapping pose based on first point cloud data and an initial pose of the first laser radar; the first point cloud data is collected by the first laser radar; and the initial pose is collected by the positioning device; According to the final mapping pose, converting the first point cloud data into a map coordinate system to obtain a first point cloud map; According to the final mapping pose and the calibrated extrinsic parameters, convert the second point cloud data collected by the second lidar into a map coordinate system to obtain a second point cloud map; Stitching and fusing the first point cloud map and the second point cloud map to obtain a stitched point cloud map; For a first point cloud in the stitched point cloud map, determining a second point cloud having a minimum height difference with the first point cloud; If the height difference between the first point cloud and the second point cloud is less than a height threshold, then calculating the average height and average reflectivity of the first point cloud and the second point cloud, setting the height of the first point cloud and the height of the second point cloud to the average height, and setting the reflectivity of the first point cloud and the reflectivity of the second point cloud to the average reflectivity, until all point clouds in the spliced point cloud map are traversed to obtain a final point cloud map; Determining a target point cloud that meets a height threshold range and a reflectivity threshold range from the final point cloud map; Generate an intensity map in HSV format according to the height range and reflectivity range corresponding to the target point cloud; Input the intensity map into a semantic segmentation model to obtain semantic labels for multiple pixels; Pixels with the same semantic label and spatially adjacent to each other are clustered into semantic elements, and the semantic elements are added to the intensity map to generate a semantic map.

2. The method according to claim 1, wherein The obtaining of a final mapping pose according to the first point cloud data and the initial pose of the first laser radar includes: extracting line features and surface features from the first point cloud data; Optimizing the initial pose of the first laser radar using the line feature and the surface feature as constraints to obtain an initial mapping pose; The initial mapping pose is optimized through loop closure detection to obtain the final mapping pose.

3. The method according to claim 2, wherein After obtaining a final mapping pose based on the first point cloud data and the initial pose of the first laser radar, the method includes: According to the final mapping pose, the line features and the surface features are respectively converted into a map coordinate system; A grid map is generated according to the statistical information of the line features and the statistical information of the surface features.

4. The method according to claim 2, wherein The extracting of line features and surface features from the first point cloud data includes: Eliminating the point cloud of the dynamic target from the first point cloud data to obtain static point cloud data; Performing line bundle processing on the static point cloud data, and calculating the curvature of each point cloud on each line bundle; If the curvature of any point cloud is greater than the line feature threshold, the point is determined to belong to the line feature; if the curvature of any point cloud is less than the surface feature threshold, the point is determined to belong to the surface feature.

5. The method according to claim 1, wherein Generating an intensity map in HSV format according to the height range and reflectivity range corresponding to the target point cloud includes: Perform normalized mapping according to the height range corresponding to the target point cloud to obtain the H value of each point cloud; Fix the S value of each point cloud in the target point cloud to 1; Determine the V value of the point cloud with a reflectivity within the reflectivity range in the target point cloud as 1, and determine the V value of the point cloud with a reflectivity outside the reflectivity range as 0; Generate an intensity map based on the HSV values of all point clouds in the target point cloud.

6. A map construction device based on dual laser radars, characterized in that: In a processor applied to a target vehicle, the target vehicle further includes a first laser radar and a positioning device mounted on the roof, and a second laser radar mounted at an angle behind the vehicle; the device includes: a mapping pose acquisition module, configured to obtain a final mapping pose based on first point cloud data and an initial pose of the first laser radar; the first point cloud data is collected by the first laser radar; and the initial pose is collected by the positioning device; a conversion module, configured to convert the first point cloud data into a map coordinate system according to the final mapping pose to obtain a first point cloud map; A second point cloud map acquisition module is configured to convert the second point cloud data collected by the second lidar into a map coordinate system according to the final mapping pose and the calibration extrinsic parameters to obtain a second point cloud map; a stitching module, configured to stitch and fuse the first point cloud map and the second point cloud map to obtain a stitched point cloud map; A determination module, configured to determine, for the first point cloud in the stitched point cloud map, a second point cloud having a minimum height difference with the first point cloud; a calculation module, configured to calculate, if the height difference between the first point cloud and the second point cloud is less than a height threshold, an average height and an average reflectivity of the first point cloud and the second point cloud, set the height of the first point cloud and the height of the second point cloud to the average height, and set the reflectivity of the first point cloud and the reflectivity of the second point cloud to the average reflectivity, until each point cloud in the spliced point cloud map is traversed to obtain a final point cloud map; a target point cloud determination module, configured to determine, from the final point cloud map, a target point cloud that satisfies a height threshold range and a reflectivity threshold range; A map generation module is used to generate an intensity map in HSV format according to the height range and reflectivity range corresponding to the target point cloud; A label acquisition module is used to input the intensity map into a semantic segmentation model to obtain semantic labels for multiple pixels; The adding module is used to cluster pixels with the same semantic label and adjacent in space into semantic elements, add the semantic elements to the intensity map, and generate a semantic map.

7. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the dual-lidar-based map construction method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the processor executes the computer program to implement the dual-lidar-based map construction method according to any one of claims 1 to 5.

Images