Multi-task detection method and system based on laser point cloud and storage medium

Abstract

The invention relates to a multi-task detection method and system based on laser point cloud and a storage medium. The method comprises the steps that M1, a vehicle runs on a road, point cloud data information of the road is obtained in real time based on a vehicle-mounted laser radar, point cloud features are extracted through a neighborhood attention mechanism based on Transform implementation, and data information of the point cloud features of the road is obtained; m2, performing down-sampling on the data information of the point cloud features of the road to extract high-level features to obtain a first high-level feature matrix of the point cloud of the road, and fusing the first high-level feature matrix with the data information of the point cloud features of the road to obtain first fused data information of the point cloud of the road; and M3, performing down-sampling on the first fusion data information of the point cloud of the road to extract high-level features. According to the method, hard parameter sharing is realized, the model parameter quantity and the calculation quantity are remarkably reduced, so that the complexity of the model is reduced, an efficient attention mechanism is adopted, and the calculation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving perception technology, and in particular to a multi-task detection method, system and storage medium based on laser point cloud. Background Art

[0002] In autonomous driving applications, simultaneously acquiring both the positional and semantic information of objects from point cloud data captured by LiDAR sensors is crucial for environmental perception and planning decisions. Existing multi-task network models based on LiDAR sensors mostly employ a multi-feature encoder architecture. Specifically, specialized point cloud representations (such as bird's-eye view projection or voxelization) are designed for different tasks (such as object detection and semantic segmentation), and corresponding feature encoders are trained separately. The features extracted by these encoders are then fed into the respective task decoders to achieve object detection and semantic segmentation. However, this multi-task LiDAR perception model, formed by combining single tasks, requires a separate feature encoder for each task. This results in a very large number of network parameters and slows down the model's training and inference speeds. Furthermore, deployment consumes significant resources, severely impacting real-time performance. Summary of the Invention

[0003] In view of the above problems, the present invention provides a multi-task detection method, system and storage medium based on laser point cloud, which not only realizes hard parameter sharing, significantly reduces the number of model parameters and calculation amount, thereby reducing the complexity of the model, but also has an efficient attention mechanism to improve computing efficiency.

[0004] In order to achieve the above-mentioned and other related purposes, the present invention provides the following technical solutions:

[0005] A multi-task detection method based on laser point cloud, the method comprising:

[0006] M1. A vehicle drives on a road, acquiring real-time point cloud data using its onboard LiDAR. This is then implemented using a Transformer-based algorithm, extracting point cloud features through a neighborhood attention mechanism to generate data on the road's point cloud features.

[0007] M2. Downsampling the data information of the road point cloud feature to extract high-level features to obtain a first high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud feature to obtain a first fused data information of the road point cloud;

[0008] M3. Downsampling the first fused data information of the road point cloud to extract high-level features to obtain a second high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a second fused data information of the road point cloud;

[0009] M4. Downsampling the second fused data information of the road point cloud to extract high-level features to obtain a third high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a third fused data information of the road point cloud;

[0010] M5. Upsample the third fused data information of the road point cloud to obtain first upsampled data information of the road point cloud, and combine it with the second fused data information of the road point cloud to construct a first depth fusion function Q1 of the road point cloud, perform fusion, and obtain first depth fused data information of the road point cloud.

[0011] Furthermore, the first depth fusion function Q1 of the road point cloud is:

[0012] ,

[0013] Among them, x is the first up-sampled data information of the road point cloud, y is the second fused data information of the road point cloud, ɑ1, ɑ2 and ɑ3 are the first depth fusion weight factors of the road point cloud.

[0014] Furthermore, the first depth fusion weight factors ɑ1, ɑ2 and ɑ3 of the road point cloud are,

[0015] ,

[0016] ,

[0017] ,

[0018] Among them, x is the first up-sampled data information of the road point cloud, and y is the second fused data information of the road point cloud.

[0019] Furthermore, the method further comprises:

[0020] M6. Upsampling the first depth-fused data information of the road point cloud to obtain second upsampled data information of the road point cloud, and fusing it with the data information of the road point cloud features to obtain fourth fused data information of the road point cloud;

[0021] M7. Based on the fourth fused data information of the road point cloud and in combination with the first fused data information of the road point cloud, construct a second depth fusion function Q2 of the road point cloud, perform fusion, and obtain second depth fused data information of the road point cloud.

[0022] Furthermore, the second depth fusion function Q2 of the road point cloud is:

[0023] ,

[0024] Among them, z1 is the fourth fusion data information of the road point cloud, z2 is the first fusion data information of the road point cloud, and β1, β2 and β3 are the second depth fusion weight factors of the road point cloud.

[0025] Furthermore, the second depth fusion weight factors β1, β2 and β3 of the road point cloud are,

[0026] ,

[0027] ,

[0028] ,

[0029] Among them, z1 is the fourth fused data information of the road point cloud, and z2 is the first fused data information of the road point cloud.

[0030] Furthermore, the method further comprises:

[0031] M8. Upsampling the second depth-fused data information of the road point cloud to obtain third upsampled data information of the road point cloud, and fusing it with the data information of the road point cloud features to obtain fifth fused data information of the road point cloud;

[0032] M9. Based on the fifth fused data information of the road point cloud and combined with the data information of the point cloud features of the road, a third depth fusion function Q3 of the road point cloud is constructed and fused to obtain the third depth fused data information of the road point cloud.

[0033] Furthermore, the third depth fusion function Q3 of the road point cloud is:

[0034] ,

[0035] ,

[0036] Among them, h1 is the fifth fusion data information of the road point cloud, h2 is the data information of the point cloud features of the road, and δ1, δ2 and δ3 are the third depth fusion weight factors of the road point cloud.

[0037] In order to achieve the above-mentioned and other related purposes, the present invention also provides a multi-task detection system based on laser point cloud, comprising a computer device programmed or configured to execute any one of the steps of the multi-task detection method based on laser point cloud.

[0038] In order to achieve the above-mentioned and other related purposes, the present invention also provides a computer-readable storage medium, which stores a computer program programmed or configured to execute any one of the multi-task detection methods based on laser point clouds.

[0039] The present invention has the following positive effects:

[0040] 1. The present invention builds a unified point cloud feature extraction framework through the first depth fusion function Q1 of the road point cloud, the second depth fusion function Q2 of the road point cloud, and the third depth fusion function Q3 of the road point cloud, realizes hard parameter sharing, significantly reduces the number of model parameters and calculation amount, and thus reduces the complexity of the model.

[0041] 2. This invention is based on a multi-task perception architecture of laser point clouds, and realizes semantic segmentation and target detection through parameter sharing and an efficient attention mechanism. It not only improves the computational efficiency of the system, but also has higher accuracy and enhances the perception capability of laser point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the overall framework of the multi-task network model of the present invention;

[0043] Figure 2 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0044] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0045] Example 1: Figure 1 or Figure 2 As shown, a multi-task detection method based on laser point cloud, the method includes:

[0046] M1. A vehicle drives on a road, acquiring real-time point cloud data using its onboard LiDAR. This is then implemented using a Transformer-based algorithm, extracting point cloud features through a neighborhood attention mechanism to generate data on the road's point cloud features.

[0047] M2. Downsampling the data information of the road point cloud feature to extract high-level features to obtain a first high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud feature to obtain a first fused data information of the road point cloud;

[0048] M3. Downsampling the first fused data information of the road point cloud to extract high-level features to obtain a second high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a second fused data information of the road point cloud;

[0049] M4. Downsampling the second fused data information of the road point cloud to extract high-level features to obtain a third high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a third fused data information of the road point cloud;

[0050] M5. Upsample the third fused data information of the road point cloud to obtain first upsampled data information of the road point cloud, and combine it with the second fused data information of the road point cloud to construct a first depth fusion function Q1 of the road point cloud, perform fusion, and obtain first depth fused data information of the road point cloud.

[0051] In this embodiment, the first depth fusion function Q1 of the road point cloud is:

[0052] ,

[0053] Among them, x is the first up-sampled data information of the road point cloud, y is the second fused data information of the road point cloud, ɑ1, ɑ2 and ɑ3 are the first depth fusion weight factors of the road point cloud.

[0054] In this embodiment, the first depth fusion weight factors ɑ1, ɑ2 and ɑ3 of the road point cloud are,

[0055] ,

[0056] ,

[0057] ,

[0058] Among them, x is the first up-sampled data information of the road point cloud, and y is the second fused data information of the road point cloud.

[0059] In this embodiment, the method further includes:

[0060] M6. Upsampling the first depth-fused data information of the road point cloud to obtain second upsampled data information of the road point cloud, and fusing it with the data information of the road point cloud features to obtain fourth fused data information of the road point cloud;

[0061] M7. Based on the fourth fused data information of the road point cloud and in combination with the first fused data information of the road point cloud, construct a second depth fusion function Q2 of the road point cloud, perform fusion, and obtain second depth fused data information of the road point cloud.

[0062] In this embodiment, the second depth fusion function Q2 of the road point cloud is:

[0063] ,

[0064] Among them, z1 is the fourth fusion data information of the road point cloud, z2 is the first fusion data information of the road point cloud, and β1, β2 and β3 are the second depth fusion weight factors of the road point cloud.

[0065] In this embodiment, the second depth fusion weight factors β1, β2 and β3 of the road point cloud are,

[0066] ,

[0067] ,

[0068] ,

[0069] Among them, z1 is the fourth fused data information of the road point cloud, and z2 is the first fused data information of the road point cloud.

[0070] In this embodiment, the method further includes:

[0071] M8. Upsampling the second depth-fused data information of the road point cloud to obtain third upsampled data information of the road point cloud, and fusing it with the data information of the road point cloud features to obtain fifth fused data information of the road point cloud;

[0072] M9. Based on the fifth fused data information of the road point cloud and combined with the data information of the point cloud features of the road, a third depth fusion function Q3 of the road point cloud is constructed and fused to obtain the third depth fused data information of the road point cloud.

[0073] In this embodiment, the third depth fusion function Q3 of the road point cloud is:

[0074] ,

[0075] ,

[0076] Among them, h1 is the fifth fusion data information of the road point cloud, h2 is the data information of the point cloud features of the road, and δ1, δ2 and δ3 are the third depth fusion weight factors of the road point cloud.

[0077] Example 2: Based on the multi-task detection method based on laser point cloud in Example 1, the present invention is further illustrated and described below.

[0078] like Figure 1 or Figure 2 As shown, a multi-task detection method based on laser point cloud, the method includes:

[0079] M1. A vehicle drives on a road, acquiring real-time point cloud data using its onboard LiDAR. This is then implemented using a Transformer-based algorithm, extracting point cloud features through a neighborhood attention mechanism to generate data on the road's point cloud features.

[0080] M2. Downsampling the data information of the road point cloud feature to extract high-level features to obtain a first high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud feature to obtain a first fused data information of the road point cloud;

[0081] M3. Downsampling the first fused data information of the road point cloud to extract high-level features to obtain a second high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a second fused data information of the road point cloud;

[0082] M4. Downsampling the second fused data information of the road point cloud to extract high-level features to obtain a third high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a third fused data information of the road point cloud;

[0083] M5. Upsample the third fused data information of the road point cloud to obtain first upsampled data information of the road point cloud, and combine it with the second fused data information of the road point cloud to construct a first depth fusion function Q1 of the road point cloud, perform fusion, and obtain first depth fused data information of the road point cloud.

[0084] The practical effects of the present invention were demonstrated using a specific autonomous driving test scenario as an example. The experimental environment consisted of a test vehicle equipped with a Velodyne HDL-64E lidar, driving on a section of urban road with various road conditions. The experiment was divided into three phases: Phase 1: Raw point cloud data was collected, recording for 10 minutes, resulting in approximately 10^7 point cloud data points. The initial feature matrix F0 was extracted using the Transformer architecture, taking approximately 2 seconds. Phase 2: Steps M2 to M4 were executed sequentially to generate the first, second, and third fused data information C1, C2, and C3, respectively. The feature extraction time for each step was 0.8 seconds, 0.6 seconds, and 0.5 seconds, respectively, for a total of 2.9 seconds. Phase 3: Step M5 was executed to upsample C3 and fuse it with C2 to generate the final deep fusion data information D1, taking approximately 1.2 seconds. Experimental results show that this method improves feature extraction efficiency by approximately 30% compared to traditional methods and significantly enhances its adaptability to complex scenarios. For example, on a test section with various road conditions such as potholes and cracks, D1's feature expression capability is improved by about 40% compared to a single feature matrix.

[0085] This method has been successfully applied to a production vehicle of an autonomous driving company, significantly improving the vehicle's ability to perceive complex road conditions. For example, on a test section containing potholes, cracks, and obstacles, a vehicle equipped with this method accurately identified and avoided all potential risks, with a false alarm rate of less than 5%. Furthermore, this method has also been used in highway inspection systems, enabling real-time monitoring and assessment of road conditions through multi-level processing of point cloud data.

[0086] This paper introduces the Transformer architecture and neighborhood attention mechanism to effectively address the shortcomings of traditional methods in feature extraction efficiency and multi-task fusion capabilities. Experimental results show that this method not only outperforms existing technologies in feature extraction speed but also demonstrates excellent adaptability in complex scenarios. Its innovations are mainly reflected in the following aspects:

[0087] 1. The Transformer architecture combined with the neighborhood attention mechanism significantly improves the efficiency and accuracy of feature extraction.

[0088] 2. Through multi-level feature fusion strategy, the perception ability of complex road conditions is enhanced.

[0089] 3. Reduced dependence on high-precision maps and improved system robustness.

[0090] In summary, the present invention has broad application prospects in the field of autonomous driving and has high practical value and promotion significance.

[0091] Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0092] In summary, the present invention not only realizes hard parameter sharing, significantly reduces the number of model parameters and computational complexity, thereby reducing the complexity of the model, but also has an efficient attention mechanism, which improves computational efficiency.

[0093] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A multi-task detection method based on laser point cloud, characterized in that: The method comprises: M1. A vehicle drives on a road, acquiring real-time point cloud data using its onboard LiDAR. This is then implemented using a Transformer-based algorithm, extracting point cloud features through a neighborhood attention mechanism to generate data on the road's point cloud features. M2. Downsampling the data information of the road point cloud feature to extract high-level features to obtain a first high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud feature to obtain a first fused data information of the road point cloud; M3. Downsampling the first fused data information of the road point cloud to extract high-level features to obtain a second high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a second fused data information of the road point cloud; M4. Downsampling the second fused data information of the road point cloud to extract high-level features to obtain a third high-level feature matrix of the road point cloud, and fusing it with the data information of the road point cloud features to obtain a third fused data information of the road point cloud; M5. Upsample the third fused data information of the road point cloud to obtain first upsampled data information of the road point cloud, and combine it with the second fused data information of the road point cloud to construct a first depth fusion function Q1 of the road point cloud, perform fusion, and obtain first depth fused data information of the road point cloud.

2. The multi-task detection method based on laser point cloud according to claim 1, characterized in that: The first depth fusion function Q1 of the road point cloud is: ， Among them, x is the first up-sampled data information of the road point cloud, y is the second fused data information of the road point cloud, ɑ1, ɑ2 and ɑ3 are the first depth fusion weight factors of the road point cloud.

3. The multi-task detection method based on laser point cloud according to claim 2, characterized in that: The first depth fusion weight factors ɑ1, ɑ2 and ɑ3 of the road point cloud are: ， ， ， Among them, x is the first up-sampled data information of the road point cloud, and y is the second fused data information of the road point cloud.

4. The multi-task detection method based on laser point cloud according to claim 1, characterized in that: The method further comprises: M6. Upsampling the first depth-fused data information of the road point cloud to obtain second upsampled data information of the road point cloud, and fusing it with the data information of the road point cloud features to obtain fourth fused data information of the road point cloud; M7. Based on the fourth fused data information of the road point cloud and in combination with the first fused data information of the road point cloud, construct a second depth fusion function Q2 of the road point cloud, perform fusion, and obtain second depth fused data information of the road point cloud.

5. The multi-task detection method based on laser point cloud according to claim 4, characterized in that: The second depth fusion function Q2 of the road point cloud is: ， Among them, z1 is the fourth fusion data information of the road point cloud, z2 is the first fusion data information of the road point cloud, and β1, β2 and β3 are the second depth fusion weight factors of the road point cloud.

6. The multi-task detection method based on laser point cloud according to claim 5, characterized in that: The second depth fusion weight factors β1, β2 and β3 of the road point cloud are, ， ， ， Among them, z1 is the fourth fused data information of the road point cloud, and z2 is the first fused data information of the road point cloud.

7. The multi-task detection method based on laser point cloud according to claim 4, characterized in that: The method further comprises: M8. Upsampling the second depth-fused data information of the road point cloud to obtain third upsampled data information of the road point cloud, and fusing it with the data information of the road point cloud features to obtain fifth fused data information of the road point cloud; M9. Based on the fifth fused data information of the road point cloud and combined with the data information of the point cloud features of the road, a third depth fusion function Q3 of the road point cloud is constructed and fused to obtain the third depth fused data information of the road point cloud.

8. The multi-task detection method based on laser point cloud according to claim 7, characterized in that: The third depth fusion function Q3 of the road point cloud is: ， ， Among them, h1 is the fifth fusion data information of the road point cloud, h2 is the data information of the point cloud features of the road, and δ1, δ2 and δ3 are the third depth fusion weight factors of the road point cloud.

9. A multi-task detection system based on laser point cloud, including computer equipment, characterized in that: The computer device is programmed or configured to execute the steps of the multi-task detection method based on laser point cloud as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program programmed or configured to execute the multi-task detection method based on laser point cloud according to any one of claims 1 to 8.

Images