Method and system for constructing a semantic map based on a three-dimensional laser radar

Abstract

The embodiment of the application provides a kind of method and system for constructing semantic map based on three-dimensional laser radar, belong to artificial intelligence technical field.The method and system include: the full convolutional neural network model of point cloud semantic segmentation is established;The full convolutional neural network model is trained using large-scale KITTI semantic dataset and field environment dataset;According to the full convolutional neural network of training completion, corresponding semantic information is generated according to acquisition information;According to the semantic information and acquisition information, semantic map is constructed using SLAM method.The method and system can efficiently generate semantic map.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically to a method and system for constructing semantic maps based on three-dimensional LiDAR. Background Technology

[0002] With the continuous development of artificial intelligence and the widespread adoption of 3D LiDAR, algorithms for semantic segmentation of laser point cloud data have received extensive attention and research. Currently, most laser SLAM systems only use the geometric information of laser data. Although they can obtain relatively accurate positioning and maps, the resulting maps are merely stacks of corresponding point clouds, making them difficult to understand and unsuitable for subsequent navigation applications. Furthermore, when the system operates in a highly dynamic environment, the presence of numerous moving objects makes generating a consistent map difficult. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for constructing semantic maps based on three-dimensional LiDAR, which can efficiently generate semantic maps.

[0004] To achieve the above objectives, embodiments of the present invention provide a method for constructing a semantic map based on three-dimensional LiDAR, the method comprising:

[0005] Establish a fully convolutional neural network model for point cloud semantic segmentation;

[0006] The fully convolutional neural network model was trained using a large-scale KITTI semantic dataset and a field environment dataset.

[0007] The trained fully convolutional neural network generates corresponding semantic information based on the collected information;

[0008] A semantic map is constructed using the SLAM method based on the semantic information and the collected information.

[0009] Optionally, the fully convolutional neural network model includes:

[0010] A downsampling encoder is used to receive input acquisition information, retain information in the vertical direction of the acquisition information, and perform downsampling operation in the horizontal direction;

[0011] The decoder module is used to upsample the features output by the downsampling encoder to the original image resolution;

[0012] The inference point cloud semantic module is used to classify the point cloud semantics based on the features output by the decoder module in order to obtain the semantic information.

[0013] Optionally, establishing a fully convolutional neural network model for point cloud semantic segmentation includes:

[0014] When the sensor of a 3D LiDAR can only scan and image on a vertical plane, the acquired information is de-skewed to remove moving objects from the acquired information in order to obtain distortion-free 3D point cloud data.

[0015] The 3D point cloud data is converted into two-dimensional image data according to formula (1).

[0016]

[0017] Where u and v are the coordinates in the two-dimensional image data, x1, y1, and z1 are the coordinates of the points in the acquired information, and fov is the coordinate of the point. up The vertical upward field of view of the 3D lidar is given by _col_, where _col_ is the width of the 2D image data, _row_ is the height of the 2D image data, and _fov_ is the vertical upward field of view. down The vertical downward field of view of the three-dimensional lidar is denoted as .

[0018] Optionally, the decoder module includes a Darknet53 network;

[0019] The reasoning point cloud semantic module is used for:

[0020] The point cloud semantics are classified according to formula (2).

[0021]

[0022] in, res represents the probability of normalized classification to class c. c This is the unbounded output for category c, where C is the number of categories.

[0023] Optionally, training the fully convolutional neural network model using a large-scale KITTI semantic dataset and a real-world environment dataset includes:

[0024] The fully convolutional neural network model was iteratively optimized using stochastic gradient descent and formula (3).

[0025]

[0026] Where L is the value of the weighted cross-entropy loss function, and C is the number of categories. f c Let ε be the probability of category c occurring, and ε be the bias. Let c be the probability of the normalized classification to class c. res c For the unbounded output result corresponding to category c, y c This represents the probability of classifying a class to category c before normalization.

[0027] Optionally, constructing a semantic map using the SLAM method based on the semantic information and the collected information includes:

[0028] Iterate through each current edge feature point in the current edge feature point set, and find the point that is closest to the current edge feature point in the local map edge feature point set;

[0029] Find the set of neighboring points using the closest point as the origin;

[0030] Calculate the mean coordinates of the centers of all points in the adjacent point set according to formula (4).

[0031]

[0032] Where cx, cy, and cz are the mean coordinates of the center, x k y k z k Let N be the coordinates of the k-th point, and N be the number of points in the adjacent point set.

[0033] The average error of the adjacent point set in each direction is calculated according to formula (5).

[0034]

[0035] Where ax, ay, and az are the average errors of the adjacent point set in each direction;

[0036] Calculate the covariance matrix of the adjacent point set according to formula (6).

[0037]

[0038] Calculate the distance residual from the current edge feature point to the corresponding edge line according to formula (7).

[0039]

[0040] in, Let P be the distance residual of the i-th current edge feature point, A and B be two arbitrarily chosen nearby points on the straight line passing through the central mean point of the adjacent point set, along the direction of the edge line corresponding to the current edge feature point, and let P be the current edge feature point. The direction of the edge line is matrix A. e The eigenvector corresponding to the largest eigenvalue;

[0041] The semantic inconsistency parameter is calculated according to formula (8).

[0042]

[0043] in, Let be the semantic inconsistency parameter of the i-th edge feature point. For the k-th point in the set of adjacent points, These represent the i-th current edge feature point, This indicates whether the semantic labels of the k-th point and the ith current edge feature point are consistent;

[0044] Calculate the total weight of the current distance residual according to formula (9).

[0045]

[0046] in, The total weight is given by α, where α and β are the geometric weight factor and the semantic weight factor, respectively.

[0047] Optionally, constructing a semantic map using the SLAM method based on the semantic information and the collected information includes:

[0048] Iterate through each current planar feature point in the current planar feature point set, and find the point in the local map planar feature point set that is closest to the current planar feature point;

[0049] Find the set of neighboring points using the closest point as the origin;

[0050] Calculate the covariance matrix of the adjacent point set.

[0051] Calculate the mean coordinates of the centers of all points in the adjacent point set according to formula (4).

[0052]

[0053] Where cx, cy, and cz are the mean coordinates of the center, x k y k z k Let N be the coordinates of the k-th point, and N be the number of points in the adjacent point set.

[0054] The average error of the adjacent point set in each direction is calculated according to formula (5).

[0055]

[0056] Where ax, ay, and az are the average errors of the adjacent point set in each direction;

[0057] Calculate the covariance matrix of the adjacent point set according to formula (6).

[0058]

[0059] The semantic inconsistency parameter is calculated according to formula (10).

[0060]

[0061] in, Let be the semantic inconsistency parameter of the i-th planar feature point. For the k-th point in the set of adjacent points, These represent the i-th current planar feature points, This indicates whether the semantic labels of the k-th point and the ith current edge feature point are consistent;

[0062] Calculate the total weight of the current distance residual according to formula (11).

[0063]

[0064] in, The total weight is given by α, where α and β are the geometric weight factor and the semantic weight factor, respectively.

[0065] Calculate the distance residual from the current plane feature point to the corresponding plane according to formula (12).

[0066]

[0067] in, Let Dx + Ey + Cz + G = 0 be the distance residual of the i-th current planar feature point, and let Dx + Ey + Cz + G = 0 be the plane containing the local map planar feature point set. The direction of this plane is the covariance matrix A. e The eigenvector corresponding to the smallest eigenvalue.

[0068] Optionally, constructing a semantic map using the SLAM method based on the semantic information and the collected information includes:

[0069] The objective function of the SLAM method is determined according to formula (13).

[0070]

[0071] Among them, e (T) Let F be the objective function. e p is the set of edge feature points of the current laser scan frame. i F represents the points in the set of feature points on the edge of the local map. p For the set of planar feature points of the current laser scan frame, The total weight of the i-th edge feature is... This represents the residual distance from the current i-th edge feature point to the corresponding edge line. The total weight of the i-th planar feature is... It represents the distance residual from the current i-th planar feature point to the corresponding plane.

[0072] On the other hand, the present invention also provides a system for constructing semantic maps based on three-dimensional LiDAR, the system including a processor for performing any of the methods described above.

[0073] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the methods described above.

[0074] Through the above technical solution, the method and system for constructing semantic maps based on 3D LiDAR provided by this invention constructs a fully convolutional neural network for point cloud semantic segmentation, trains this fully convolutional neural network by constructing a large-scale KITTI semantic dataset and a field environment dataset, and finally uses the semantic information output by the fully convolutional neural network and the information collected by the field LiDAR sensor to construct a semantic map using the SLAM method. Compared with the prior art, the method provided by this invention can realize the construction of semantic maps using only the hardware of a single LiDAR sensor, improving the efficiency of semantic map construction and reducing the complexity of the algorithm.

[0075] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0076] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0077] Figure 1 This is a flowchart of a method for constructing a semantic map based on a three-dimensional LiDAR according to an embodiment of the present invention;

[0078] Figure 2 This is a structural block diagram of a fully convolutional neural network model according to an embodiment of the present invention;

[0079] Figure 3 This is a preferred example diagram of a fully convolutional neural network model according to an embodiment of the present invention;

[0080] Figure 4 This is a schematic diagram of coordinate transformation according to one embodiment of the present invention;

[0081] Figure 5 This is a flowchart illustrating how, according to an embodiment of the present invention, a semantic map is constructed using the SLAM method based on semantic information and collected information for edge feature points.

[0082] Figure 6This is a flowchart of a process for constructing a semantic map based on semantic information and collected information using the SLAM method for planar feature points according to an embodiment of the present invention. Detailed Implementation

[0083] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0084] Figure 1 This is a flowchart of a method for constructing a semantic map based on a three-dimensional LiDAR according to an embodiment of the present invention. Figure 1 In this context, the method may include:

[0085] In step S10, a fully convolutional neural network model for point cloud semantic segmentation is established;

[0086] In step S11, a fully convolutional neural network model is trained using a large-scale KITTI semantic dataset and a field environment dataset;

[0087] In step S12, the trained fully convolutional neural network is used to generate corresponding semantic information based on the collected information;

[0088] In step S13, the SLAM method is used to construct a semantic map based on semantic information and collected information.

[0089] In Figure 1 The schematic diagram of the fully convolutional neural network model established in the method shown can be as follows: Figure 2 and Figure 3 As shown. Among them, Figure 2 This is a structural diagram of the overall structure. Figure 3 This is a preferred example diagram of the fully convolutional neural network model. Figure 2 In the middle, the downsampling encoder 01 can be connected to the decoder module 02, and the decoder module 02 can be connected to the inference point cloud semantic module 03.

[0090] Specifically, the downsampling encoder 01 can be used to receive input acquisition information, retain information in the vertical direction of the acquisition information, and perform a downsampling operation in the horizontal direction. The downsampling encoder 01 has a faster operating speed when performing downsampling operations. For LiDAR sensors, the information in the vertical direction of the image projected from their data is determined by the LiDAR beam, which contains relatively little information. Therefore, it is only necessary to retain the information in the vertical direction and instead perform a downsampling operation on the information in the horizontal direction.

[0091] The decoder module 02 can be used to upsample the features output by the downsampling encoder 01 to the original image resolution. In a preferred embodiment of the present invention, the decoder module can be constructed based on the DarkNet network backbone architecture (Darknet53 network) for tasks such as image classification and object detection, thereby upsampling the extracted features to the original image resolution.

[0092] The inference point cloud semantic module 03 can be used to classify point cloud semantics based on the features output by the decoder module 02 to obtain semantic information. While the classification function for this inference point cloud semantic module 03 can be any of the functions known to those skilled in the art, in a preferred embodiment of the present invention, the inference point cloud semantic module 03 can use formula (1) as the classification function.

[0093]

[0094] in, res represents the probability of normalized classification to class c. c This is the unbounded output for category c, where C is the number of categories.

[0095] Step S11 can be used to train a fully convolutional neural network model using a large-scale KITTI semantic dataset and a field environment dataset. The large-scale KITTI semantic dataset can be an existing KITTI dataset with extensive and dense point cloud semantic annotations. The field environment dataset can be a constructed semantic dataset of the actual environment corresponding to the large-scale KITTI semantic dataset. Training the model using the combination of the large-scale KITTI semantic dataset and the field environment dataset ensures that the trained fully convolutional neural network can achieve semantic segmentation performance in the actual test environment.

[0096] Before training this fully convolutional neural network model, mechanical LiDAR sensors such as Velodyne sensors scan and image on a vertical plane through mechanical rotation. This line-by-line exposure method, similar to a camera, can cause a rolling shutter effect. That is, when the carrier robot or the object being detected is moving, due to line-by-line exposure, different parts of the moving object are not exposed at the same time, leading to phenomena such as shaking and sloping images. Therefore, to obtain a more geometrically consistent representation of each LiDAR scanning environment, our first step is to convert each de-sloping 3D point cloud data into a two-dimensional image data representation, called a spherical projection image of the point cloud, such as... Figure 4 The diagram illustrates the projection coordinate transformation. This projection is not a simple unfolding of the LiDAR scanning surface, but rather a transformation of the base coordinate system. Within this system, a single pixel can contain multiple measurement values, arranged in descending order of the point cloud distance R from these measurements, ensuring that all points presented in the image are within the sensor's current field of view.

[0097] Specifically, when the sensor of the 3D lidar can only scan and image on the vertical plane, the acquired information is de-skewed to remove moving objects from the acquired information in order to obtain distortion-free 3D point cloud data; then, the 3D point cloud data is converted into 2D image data according to formula (2).

[0098]

[0099] Where u and v are the coordinates in the two-dimensional image data, x1, y1, and z1 are the coordinates of the points in the acquired information, and fov is the coordinate of the point. up Here, is the vertical upward field of view angle of the 3D LiDAR, col is the width of the 2D image data, row is the height of the 2D image data, and fov is the vertical upward field of view angle. down This refers to the vertical downward field of view angle of the 3D lidar.

[0100] During the training of this fully convolutional neural network model, stochastic gradient descent and formula (3) can be used to iteratively optimize the fully convolutional neural network model.

[0101]

[0102] Where L is the value of the weighted cross-entropy loss function, and C is the number of categories. f c Let ε be the probability of category c occurring, and ε be the bias. Let c be the probability of the normalized classification to class c. res c For the unbounded output result corresponding to category c, y c Let w be the probability of classifying a class to category c before normalization. In the optimization function shown in formula (3), the parameter w is added. c This results in a greater penalty for points in classes with low frequency compared to points in classes with high frequency, thereby improving the efficiency of iterative optimization.

[0103] Step S13 can be used to construct a semantic map based on semantic information and collected information using the SLAM method. Specifically, for edge feature points and planar feature points, step S13 can include, respectively, the following: Figure 5 and Figure 6 The steps shown are described. Figure 5 In this context, step S13 may include:

[0104] In step S20, each current edge feature point in the current edge feature point set is traversed, and the point closest to the current edge feature point is found in the local map edge feature point set. This local map edge feature point set can be any set of all currently generated feature points.

[0105] In step S21, the nearest point is used as the origin to find the set of neighboring points;

[0106] In step S22, the mean coordinates of the centers of each point in the adjacent point set are calculated according to formula (4).

[0107]

[0108] Where cx, cy, and cz are the central mean coordinates, x k y k z k Let N be the coordinates of the k-th point, and N be the number of points in the set of adjacent points.

[0109] In step S23, the average error of the adjacent point set in each direction is calculated according to formula (5).

[0110]

[0111] Where ax, ay, and az are the average errors of adjacent point sets in each direction;

[0112] In step S24, the covariance matrix of the adjacent point sets is calculated according to formula (6).

[0113]

[0114] In step S25, the distance residual from the current edge feature point to the corresponding edge line is calculated according to formula (7).

[0115]

[0116] in, Let P be the distance residual of the i-th current edge feature point, A and B be two arbitrarily chosen nearby points on the straight line along the direction of the edge line corresponding to the current edge feature point and passing through the central mean point of the adjacent point set, respectively, and let P be the current edge feature point. The direction of the edge line is represented by matrix A. e The eigenvector corresponding to the largest eigenvalue;

[0117] In step S26, the semantic inconsistency parameter is calculated according to formula (8).

[0118]

[0119] in, Let be the semantic inconsistency parameter of the i-th edge feature point. For the k-th point in the set of adjacent points, These represent the i-th current edge feature point, This indicates whether the semantic labels of the k-th point and the ith current edge feature point are consistent;

[0120] In step S27, the total weight of the current distance residual is calculated according to formula (9).

[0121]

[0122] in, The total weight is represented by α, and β is the geometric weight factor and semantic weight factor, respectively.

[0123] exist Figure 6 In this context, step S13 may include:

[0124] In step S30, each current planar feature point in the current planar feature point set is traversed, and the point closest to the current planar feature point is found in the local map planar feature point set;

[0125] In step S31, the nearest point is used as the origin to find the set of neighboring points;

[0126] In step S32, the covariance matrix of adjacent point sets is calculated.

[0127] In step S33, the mean coordinates of the centers of each point in the adjacent point set are calculated according to formula (4).

[0128]

[0129] Where cx, cy, and cz are the central mean coordinates, x k y k z k Let N be the coordinates of the k-th point, and N be the number of points in the set of adjacent points.

[0130] In step S34, the average error of the adjacent point set in each direction is calculated according to formula (5).

[0131]

[0132] Where ax, ay, and az are the average errors of adjacent point sets in each direction;

[0133] In step S35, the covariance matrix of the adjacent point sets is calculated according to formula (6).

[0134]

[0135] In step S36, the semantic inconsistency parameter is calculated according to formula (10).

[0136]

[0137] in, Let be the semantic inconsistency parameter of the i-th planar feature point. For the k-th point in the set of adjacent points, These represent the i-th current planar feature points, This indicates whether the semantic labels of the k-th point and the ith current edge feature point are consistent;

[0138] In step S37, the total weight of the current distance residual is calculated according to formula (11).

[0139]

[0140] in, Let α be the total weight, and β be the geometric weight factor and the semantic weight factor, respectively.

[0141] In step S38, the distance residual from the current plane feature point to the corresponding plane is calculated according to formula (12).

[0142]

[0143] in, Let Dx + Ey + Cz + G = 0 be the distance residual of the i-th current planar feature point, and let Dx + Ey + Cz + G = 0 be the plane containing the local map planar feature point set. The direction of this plane is the covariance matrix A. e The eigenvector corresponding to the smallest eigenvalue.

[0144] In this embodiment, the objective function of the SLAM method can take many forms known to those skilled in the art. However, in a preferred example of the present invention, the objective function can be formula (13).

[0145]

[0146] Among them, e (T) Let F be the objective function. e p is the set of edge feature points of the current laser scan frame. i F represents the points in the set of feature points on the edge of the local map. p For the set of planar feature points of the current laser scan frame, The total weight of the i-th edge feature is... This represents the residual distance from the current i-th edge feature point to the corresponding edge line. The total weight of the i-th planar feature is... It represents the distance residual from the current i-th planar feature point to the corresponding plane.

[0147] On the other hand, the present invention also provides a system for constructing semantic maps based on three-dimensional LiDAR, the system including a processor for performing any of the methods described above.

[0148] In another aspect, the present invention also provides a computer-readable storage medium storing instructions for being read by a machine to cause the machine to perform any of the methods described above.

[0149] Through the above technical solution, the method and system for constructing semantic maps based on 3D LiDAR provided by this invention constructs a fully convolutional neural network for point cloud semantic segmentation, trains this fully convolutional neural network by constructing a large-scale KITTI semantic dataset and a field environment dataset, and finally uses the semantic information output by the fully convolutional neural network and the information collected by the field LiDAR sensor to construct a semantic map using the SLAM method. Compared with the prior art, the method provided by this invention can realize the construction of semantic maps using only the hardware of a single LiDAR sensor, improving the efficiency of semantic map construction and reducing the complexity of the algorithm.

[0150] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0156] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0157] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0158] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for constructing a semantic map based on 3D LiDAR, characterized in that, The method includes: Establish a fully convolutional neural network model for point cloud semantic segmentation; The fully convolutional neural network model was trained using a large-scale KITTI semantic dataset and a field environment dataset. The trained fully convolutional neural network model generates corresponding semantic information based on the collected information; A semantic map is constructed using the SLAM method based on the semantic information and the collected information; The fully convolutional neural network model includes: A downsampling encoder is used to receive input acquisition information, retain information in the vertical direction of the acquisition information, and perform downsampling operation in the horizontal direction; The decoder module is used to upsample the features output by the downsampling encoder to the original image resolution; The inference point cloud semantic module is used to classify the point cloud semantics based on the features output by the decoder module in order to obtain the semantic information. The construction of a semantic map based on the semantic information and the collected information using the SLAM method includes: Iterate through each current edge feature point in the current edge feature point set, and find the point that is closest to the current edge feature point in the local map edge feature point set; Find the set of neighboring points using the closest point as the origin; Calculate the mean coordinates of the centers of all points in the adjacent point set according to formula (4). ，（4） in, , , The center mean coordinates, , , For the first The coordinates of the points The number of points in the adjacent point set; Calculate the average error of the adjacent point set in each direction according to formula (5). ，（5） in, , , The average error of the set of adjacent points in each direction; Calculate the covariance matrix of the adjacent point set according to formula (6). ，（6） Calculate the distance residual from the current edge feature point to the corresponding edge line according to formula (7). ，（7） in, For the first The distance residual of each current edge feature point , These are two points arbitrarily selected on the straight line along the direction of the edge line corresponding to the current edge feature point, and passing through the central mean point of the adjacent point set. The current edge feature point is defined as a matrix, and the direction of the edge line is a matrix. The eigenvector corresponding to the largest eigenvalue; The semantic inconsistency parameter is calculated according to formula (8). ，（8） in, For the first Semantic inconsistency parameters of edge feature points The first in the set of adjacent points One point, They represent the first The current edge feature points, Indicates the judgment of the first The point and the first Are the semantic labels of the current edge feature points consistent? Calculate the total weight of the current distance residual according to formula (9). ，（9） in, The total weight is... , These are the geometric weight factor and the semantic weight factor, respectively.

2. The method according to claim 1, characterized in that, The fully convolutional neural network model for point cloud semantic segmentation includes: When the sensor of a 3D LiDAR can only scan and image on a vertical plane, the acquired information is de-skewed to remove moving objects from the acquired information in order to obtain distortion-free 3D point cloud data. The 3D point cloud data is converted into two-dimensional image data according to formula (1). ，（1） in, , The coordinates in the two-dimensional image data, , , These are the coordinates of the points in the collected information. The vertically upward field of view of the three-dimensional lidar is denoted as . The width of the two-dimensional image data. The height of the two-dimensional image data. The vertical downward field of view of the three-dimensional lidar is denoted as .

3. The method according to claim 1, characterized in that, The decoder module includes a Darknet53 network; The reasoning point cloud semantic module is used for: The point cloud semantics are classified according to formula (2). ，（2） in, Normalized classification to category The probability, For category The corresponding unbounded output result, The number of categories.

4. The method according to claim 1, characterized in that, The fully convolutional neural network model was trained using a large-scale KITTI semantic dataset and a real-world environment dataset, including: The fully convolutional neural network model was iteratively optimized using stochastic gradient descent and formula (3). ，（3） in, The value of the weighted cross-entropy loss function. For the number of categories, , For category The probability of occurrence For bias, Normalized classification to category The probability, , For category The corresponding unbounded output result, Classification to Category before Normalization The probability of.

5. The method according to claim 1, characterized in that, The construction of a semantic map based on the semantic information and the collected information using the SLAM method includes: Iterate through each current planar feature point in the current planar feature point set, and find the point in the local map planar feature point set that is closest to the current planar feature point; Find the set of neighboring points using the closest point as the origin; Calculate the mean coordinates of the centers of all points in the adjacent point set according to formula (4). ，（4） in, , , The center mean coordinates, , , For the first The coordinates of the points The number of points in the adjacent point set; Calculate the average error of the adjacent point set in each direction according to formula (5). ，（5） in, , , The average error of the set of adjacent points in each direction; Calculate the covariance matrix of the adjacent point set according to formula (6). ，（6） The semantic inconsistency parameter is calculated according to formula (10). ，（10） in, For the first Semantic inconsistency parameters of planar feature points The first in the set of adjacent points One point, They represent the first Current planar feature points, Indicates the judgment of the first The point and the first Are the semantic labels of the current edge feature points consistent? Calculate the total weight of the current distance residual according to formula (11). ，（11） in, The total weight is... , These are geometric weighting factors and semantic weighting factors, respectively. This represents the distance residual from the current plane feature point to the corresponding plane. Calculate the distance residual from the current plane feature point to the corresponding plane according to formula (12). ，（12） in, For the first The distance residual from each current plane feature point to its corresponding plane. The plane containing the set of feature points on the local map.

6. The method according to claim 1, characterized in that, The construction of a semantic map based on the semantic information and the collected information using the SLAM method includes: The objective function of the SLAM method is determined according to formula (13). ，（13） in, Let the objective function be... The set of edge feature points for the current laser scan frame. These are the points in the edge feature point set of the current laser scan frame. These are points in the planar feature point set of the current laser scan frame. For the set of planar feature points of the current laser scan frame, For the first The total weight of each edge feature For the current number The distance residual from each edge feature point to the corresponding edge line For the first The total weight of each planar feature For the current number The distance residual from each planar feature point to its corresponding plane.

7. A system for constructing semantic maps based on 3D LiDAR, characterized in that, The system includes a processor for performing the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that are read by a machine to cause the machine to perform the method as described in any one of claims 1 to 6.

Images