A LiDAR-based navigation method for large-scale driving scenarios

Abstract

The present invention discloses a navigation method based on laser radar in large-scale driving scenarios, which belongs to the field of navigation technology. The method includes: acquiring and inputting laser point cloud signals; performing feature extraction and aggregation on the laser point cloud signals through the coding layer of the codec network to obtain local features, and then performing multi-resolution fusion of local features of different scales through the decoding layer of the codec network using an adaptive fusion mechanism combined with an attention interpolation module to output high-level features; inputting the high-level features output by the codec network into a semantic association mining module, and embedding the output features through two MLP layers; performing semantic segmentation on the final output features, and planning the optimal navigation route based on the semantic segmentation results. The network proposed in the present invention has achieved significant breakthroughs in the accuracy, efficiency and generalization ability of semantic segmentation of point clouds in large-scale scenarios through modular innovation and system optimization, providing a highly reliable environmental perception solution for autonomous driving navigation.

Description

Technical Field

[0001] The present invention belongs to the field of navigation technology, and in particular relates to a laser radar-based navigation method in large-scale driving scenarios. Background Art

[0002] In the field of autonomous driving, LiDAR technology is commonly used for positioning and navigation. LiDAR scans a target scene by emitting laser pulses, forming a laser point cloud. Each laser point cloud contains information such as the three-dimensional coordinates, color, and laser reflection intensity of that point in space. By processing and analyzing this point cloud data, the target scene environment can be perceived.

[0003] In recent years, with the increasing demand for understanding three-dimensional scenes, point cloud datasets have gradually expanded their scope, from small indoor scenes to large outdoor scenes and even city-scale scenes. Compared to the common problems of point cloud data in large-scale scenes, such as disorder, unstructuredness, high computational complexity, and low data utilization, the common characteristics of small-scale point cloud data are: relatively uniform density, a small number of points, a small scene area, and a relatively simple object type. Therefore, the laser point cloud processing network originally designed for small scenes is not suitable for direct scalability to large-scale scenes.

[0004] It can be seen from this that it is necessary to provide a lidar-based navigation method in large-scale driving scenarios to solve the problems raised in the above background technology. Summary of the Invention

[0005] The present invention discloses a lidar-based navigation method in large-scale driving scenarios, which provides a new point cloud semantic segmentation network to improve the accuracy of lidar navigation in large-scale driving scenarios, thereby effectively solving at least one technical problem involved in the background technology.

[0006] To achieve the above object, the technical solution of the present invention is:

[0007] A laser radar-based navigation method for large-scale driving scenarios includes the following steps:

[0008] Step S1, obtaining and inputting laser point cloud signals;

[0009] Step S2: extracting and aggregating features of the laser point cloud signal through the encoding layer of the encoding and decoding network to obtain local features, and then performing multi-resolution fusion of local features of different scales through the decoding layer of the encoding and decoding network using an adaptive fusion mechanism combined with an attention interpolation module to output high-level features;

[0010] Step S3: inputting the high-level features into a semantic association mining module, embedding the output high-level features through two MLP layers of the semantic association mining module to form final output features, wherein the semantic association mining module integrates spatial information and contextual information through cross compensation;

[0011] Step S4: semantically segment the final output features and plan the optimal navigation route based on the semantic segmentation results.

[0012] Optionally, in step S2, the specific implementation of the attention interpolation module includes:

[0013] Step S21: In the interpolation layer, the mth layer of the to-be-interpolated point set is used As the center, for small-scale point sets Feature map Perform a K-nearest neighbor search to obtain a local representation:

[0014] ;

[0015] ;

[0016] Where, KNN It is the K nearest neighbor classification algorithm; is the preliminary interpolation result; is the point cloud information of the γ module; is the parameter matrix; γ is the combination module of linear mapping and batch normalization layer; is the number of points in the mth layer to be interpolated; Small-scale point set Additional known features included; is the result feature of K value interpolation;

[0017] Step S22, through the linear mapping and batch normalization layer combination module γ Map and generate parameter matrix , and in Dimensions are normalized with softmax and weighted summation, and the interpolation results are output:

[0018] ;

[0019] Where, For Perform operations under different dimensions; softmax (·)for softmax operate.

[0020] Optionally, in step S3, the specific implementation of the semantic association mining module includes:

[0021] Step S31: Based on the neighborhood center context and coordinates ,in, is the number of points in the neighborhood center c, is the number of points; and the absolute point set context and coordinates , we can get the relative point set context and relative coordinates ;

[0022] Step S32: Encode the context information and geometric space information, and generate mixed features F′ and geometric code P′ through splicing operations and 1×1 convolutional layers;

[0023] Step S33: Combine the maximum pooling and attention pooling to aggregate the mixed features and generate the final context feature code .

[0024] Optionally, the encoding layer dynamically adjusts the feature extraction range through a receptive field reconstruction mechanism, and the attention interpolation module is used to improve the accuracy of high-level features.

[0025] Optionally, the number of points in the mth layer to be interpolated is , where δ m is the preset downsampling factor.

[0026] Optionally, in step S32, the concatenation operation of context coding and geometric coding satisfies:

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] Where, copy (·) indicates a shallow copy of the encoding; concat (·) indicates a concatenation operation; conv 1×1 Represents a 1×1 convolution.

[0032] Optionally, in step S33, the final context feature encoding It is expressed by the following formula:

[0033] .

[0034] Optionally, the semantic segmentation result is generated by a Softmax classifier, and the classification categories include roads, obstacles, vehicles and pedestrians.

[0035] The beneficial effects of the present invention are as follows:

[0036] 1. This method, through a semantic association mining module, achieves cross-compensation and parallel gradient propagation between spatial geometric and contextual information. This effectively addresses the semantic ambiguity caused by the sparsity and unstructured nature of point clouds in large-scale scenarios, ensuring the coordination of semantic association mining and enabling the network to learn relatively more accurate information. Experimental data show that in a six-fold cross-validation of the S3DIS dataset, this method achieved an overall accuracy (OA) of 89.4% and a mean intersection over union (mIoU) of 73.6%, representing an improvement of approximately 1.4%-2.0% over existing baseline models (such as BAAF-Net). On the SemanticKITTI validation set, this method achieved a 60% mIoU for the first time in a point-by-point MLP method without data augmentation, significantly outperforming similar methods in the recognition accuracy of key categories (such as people and vehicles).

[0037] 2. The attention interpolation module proposed in this paper uses a reverse scale interpolation mechanism (interpolating from small to large scales), combined with KNN local aggregation and adaptive weighting, to achieve inverse inference of contextual information from coarse-grained to fine-grained. This design effectively improves the resolution of sparse point clouds and addresses the information loss problem encountered during the encoding process in traditional methods. Ablation experiments show that the introduction of this module improves the model's mean Interpolation Over Union (MIOU) by 1.7%, increases inference time by only 0.1 seconds, and reduces the parameter size by 57% (from 4.97M to 2.15M).

[0038] 3. This invention replaces traditional dense KNN and downsampling strategies with a receptive field reconstruction (RFR) mechanism, reducing computational complexity while preserving key features. Combined with the hybrid pooling (max pooling and attention pooling) of the semantic context encoding (SCE) module, the model parameter count is reduced to less than 50% of the baseline model, and the inference speed is increased by 30% (from 0.8 seconds to 0.6 seconds), making it particularly suitable for autonomous driving scenarios with high real-time requirements.

[0039] 4. In complex driving scenarios (such as parking lots and areas with dense traffic signs), the method provided by this invention significantly improves the ability to distinguish easily confused categories (such as doors, debris, and pillars) through the deep fusion of geometric and contextual information. Visualization experiments show that in scenes such as corridors and conference rooms, the misjudgment rate of objects such as doors and bookshelves is reduced by over 40% compared to BAAF-Net, and the recognition accuracy of dynamic objects (such as pedestrians and vehicles) is improved by 15%-20%, ensuring the safety and reliability of the navigation system.

[0040] In summary, the network proposed in this paper has achieved significant breakthroughs in the accuracy, efficiency, and generalization ability of point cloud semantic segmentation in large-scale scenarios through modular innovation and system optimization, providing a highly reliable environmental perception solution for autonomous driving navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which:

[0042] Figure 1 A flowchart of a laser radar-based navigation method in a large-scale driving scenario provided by the present invention;

[0043] Figure 2 This is the semantic association mining module diagram provided by the present invention. DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0046] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referenced. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0047] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can refer to fixed connection, detachable connection, or integration; it can refer to mechanical connection; it can refer to direct connection or indirect connection through an intermediate medium; it can refer to internal communication between two elements or interaction between two elements, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0048] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0049] See also Figure 1 As shown, an embodiment of the present invention provides a laser radar-based navigation method in a large-scale driving scenario, comprising the following steps:

[0050] Step S1, obtaining and inputting laser point cloud signals;

[0051] Step S2: extracting and aggregating features of the laser point cloud signal through the encoding layer of the encoding and decoding network to obtain local features, and then performing multi-resolution fusion of local features of different scales through the decoding layer of the encoding and decoding network using an adaptive fusion mechanism combined with an attention interpolation module to output high-level features;

[0052] Step S3: input the high-level features into the semantic association mining module. Figure 2 The two MLP layers (shown in FIG) embed the output high-level features to form the final output features, wherein the semantic association mining module integrates spatial information and contextual information through cross compensation;

[0053] Step S4: semantically segment the final output features and plan the optimal navigation route based on the semantic segmentation results.

[0054] In step S2, the specific implementation of the attention interpolation module includes:

[0055] Step S21: In the interpolation layer, the mth layer of the to-be-interpolated point set is used As the center, for small-scale point sets Feature map Perform a K-nearest neighbor search to obtain a local representation:

[0056] ;

[0057] ;

[0058] Where, KNN It is the K nearest neighbor classification algorithm; is the preliminary interpolation result; is the point cloud information of the γ module; is the parameter matrix; γ is the combination module of linear mapping and batch normalization layer; is the number of points in the mth layer to be interpolated; Small-scale point set Additional known features included; is the result feature of K value interpolation;

[0059] Step S22, through the linear mapping and batch normalization layer combination module γ Map and generate parameter matrix , and in Dimensions are normalized with softmax and weighted summation, and the interpolation results are output. :

[0060] ;

[0061] Where, For Perform operations under different dimensions; softmax (·)for softmax operate.

[0062] In step S3, the specific implementation of the semantic association mining module includes:

[0063] Step S31: Based on the neighborhood center context and coordinates ,in, is the number of points in the neighborhood center c, is the number of points; and the absolute point set context and coordinates , we can get the relative point set context and relative coordinates ;

[0064] Step S32: Encode the context information and geometric space information, and generate mixed features F′ and geometric code P′ through splicing operations and 1×1 convolutional layers;

[0065] Step S33: Combine the maximum pooling and attention pooling to aggregate the mixed features and generate the final context feature code .

[0066] The encoding layer dynamically adjusts the feature extraction range through a receptive field reconstruction mechanism, and the attention interpolation module is used to improve the accuracy of high-level features.

[0067] The number of points in the mth layer to be interpolated is , where δ m is the preset downsampling factor.

[0068] In step S32, the concatenation operation of context coding and geometric coding satisfies:

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] Where, copy (·) indicates a shallow copy of the encoding; concat (·) indicates concatenation operation; conv 1×1 Indicates a 1×1 roll.

[0074] In step S33, the final context feature encoding It is expressed by the following formula:

[0075] .

[0076] To further illustrate, the hybrid local aggregation method mentioned above is used, that is, the combination of maximum pooling and attention pooling:

[0077] ;

[0078] Where, It is the encoding after the aggregation of maximum pooling and attention pooling; max (·) indicates the maximum value calculation process; attention (·) represents the calculation process of the three-dimensional attention mechanism.

[0079] The semantic segmentation result is generated by a Softmax classifier, and the classification categories include roads, obstacles, vehicles and pedestrians.

[0080] The following describes in detail the laser radar-based navigation method for large-scale driving scenarios provided by the present invention with specific embodiments.

[0081] Example 1

[0082] Table 1 shows the settings of some hyperparameters when using S3DIS as the experimental dataset. N is the scale of the input point cloud.

[0083] Table 1 Network hyperparameter settings

[0084]

[0085] Because S3DIS has a total of six regions, a six-fold crossover test was performed. This involves iterating through all six regions, extracting one of them at a time as the test set, and using the other five regions as the training set for training and testing. Finally, the test results for the six regions were aggregated on a point-by-point basis to calculate the mean and obtain the final performance. The data packaging strategy uses a probabilistic training sample selection mechanism, which maintains the randomness of training sample acquisition.

[0086] Table 2 shows the results of a six-fold cross-pollination experiment on the proposed model and other similar works in recent years. The experimental results show that the method using the proposed network is the best among the above methods in terms of overall accuracy and average cross-pollination ratio, surpassing BAAF-Net in 8 out of 13 categories.

[0087] Table 2 S3DIS six-fold crossover test results (%)

[0088]

[0089] Table 3 shows the verification results of S3DIS region 5. As shown in Table 3, the mIoU of our application increases by about 2.0% over the baseline, surpassing recent work.

[0090] Table 3 S3DIS test results in area 5 (%)

[0091]

[0092] Example 2

[0093] The results were obtained by conducting experiments on the SemanticKITTI validation set and compared with recent work based on pure lidar without data augmentation and large-scale parameter networks.

[0094] The experimental results show that the network of this application achieves advanced performance, reaching an average intersection-over-union ratio of 60% for the first time among pure point-by-point MLP-based methods without data augmentation and large parameter models, and surpassing all other listed methods in 8 of the 19 categories.

[0095] This advantage is even more evident in visualization experiments. It's well known that people and vehicles are the two most crucial categories in autonomous driving scene prediction, and accurate identification is crucial for autonomous driving safety. The proposed model significantly outperforms BAAF-Net in recognizing these two categories. Furthermore, the geometric structure of parking lots and roads is similar, and the proposed semantic association mining module better captures the connection between the sparse structure of the perceived area and contextual information, significantly improving recognition performance.

[0096] Example 3

[0097] Table 4 shows the results of module ablation experiments. K&D represents the dense KNN and layer-by-layer downsampling mechanism. BA stands for bilateral block. FP refers to the nearest neighbor interpolation decoder used by BAAF-Net. Inference time shows the time required to pass a batch of data through this application's model.

[0098] Table 4 Module ablation experiment (%)

[0099]

[0100] By ablating the module, different sub-models were constructed to evaluate the superiority of the proposed module, as shown in Table 4. Model I is the baseline model (BAAF-Net) without module replacement. Model II replaces the K&D mechanism with the RFR mechanism, achieving a 1.7% improvement in mIoU, thanks to the RFR mechanism's contribution in reducing information loss. Model III replaces the BA module with the proposed SCE module, achieving a 1.2% increase in mIoU compared to the baseline. The number of trainable parameters is significantly reduced to approximately half, and inference time is reduced by one-third, thanks to SCE's contribution in mining the association between spatial geometric features and contextual features. Model IV replaces the FP module with the AFP module, achieving a steady 1.6% improvement in mIoU, thanks to the module's enhanced adaptive decoding capability. However, due to the need for additional KNN operations, inference time is increased.

[0101] Models V through VII randomly replace two of the proposed modules, each outperforming a sub-model that replaces only one module. Model VIII, the complete model with all three modules replaced, achieves optimal mIoU and OA. While its parameter count is slightly improved compared to Model II, which has the fewest, it still significantly outperforms Model I. Inference time is slightly improved compared to Model I, due to the introduction of additional KNN operations. In summary, the more proposed modules replaced, the better the model's segmentation performance.

[0102] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are protected by the present invention.

Claims

1. A navigation method based on lidar in large-scale driving scenarios, characterized by: The following steps are involved: Step S1, obtaining and inputting laser point cloud signals; In step S2, the laser point cloud signal is subjected to feature extraction and aggregation by the encoding layer of the codec network to obtain local features. Then, the decoding layer of the codec network uses an adaptive fusion mechanism combined with an attention interpolation module to perform multi-resolution fusion on the local features of different scales to output high-level features. The specific implementation of the attention interpolation module includes: Step S21: In the interpolation layer, the mth layer of the to-be-interpolated point set is used As the center, for small-scale point sets Feature map Perform a K-nearest neighbor search to obtain a local representation: ； ； Where, KNN It is the K nearest neighbor classification algorithm; is the preliminary interpolation result; is the point cloud information of the γ module; is the parameter matrix; γ is the combination module of linear mapping and batch normalization layer; is the number of points in the mth layer to be interpolated; Small-scale point set Additional known features included; is the result feature of K value interpolation; Step S22, through the linear mapping and batch normalization layer combination module γ Map and generate parameter matrix , and in Dimensions are normalized with softmax and weighted summation, and the interpolation results are output. : ； Where, For Perform operations under different dimensions; softmax (·)for softmax operate; Step S3: inputting the high-level features into a semantic association mining module, embedding the output high-level features through two MLP layers of the semantic association mining module to form final output features, wherein the semantic association mining module integrates spatial information and contextual information through cross compensation; Step S4: semantically segment the final output features and plan the optimal navigation route based on the semantic segmentation results.

2. The method according to claim 1, characterized in that In step S3, the specific implementation of the semantic association mining module includes: Step S31: Based on the neighborhood center context and coordinates ,in, is the number of points in the neighborhood center c, is the number of points; and the absolute point set context and coordinates , we can get the relative point set context and relative coordinates ; Step S32: Encode the context information and geometric space information, and generate mixed features through splicing operation and 1×1 convolution layer F ′ and geometric coding P '; Step S33: Combine the maximum pooling and attention pooling to aggregate the mixed features and generate the final context feature code .

3. The method according to claim 1, characterized in that In step S2, the encoding layer dynamically adjusts the feature extraction range through the receptive field reconstruction mechanism, and the attention interpolation module is used to improve the accuracy of high-level features.

4. The method according to claim 1, wherein The number of points in the mth layer to be interpolated is , where δ m is the preset downsampling factor.

5. The method according to claim 2, characterized in that In step S32, the concatenation operation of context coding and geometric coding satisfies: ； ； ； ； Where, copy (·) indicates a shallow copy of the encoding; concat (·) indicates a concatenation operation; conv 1×1 Represents a 1×1 convolution.

6. The method according to claim 5, characterized in that In step S33, the final context feature encoding It is expressed by the following formula: 。 7. The method according to claim 5, characterized in that The semantic segmentation result is generated by a Softmax classifier, and the classification categories include roads, obstacles, vehicles and pedestrians.

Images