A 3D object detection method, system, medium and terminal

Abstract

The application provides a 3D target detection method and system, comprising the following steps: extracting a 3D sparse feature column based on voxels in an original LiDAR point cloud, converting the 3D sparse feature column to a 2D feature map under a BEV perspective after densification, generating a multi-scale feature map by using a neck network, sending the multi-scale feature map into a detection head after cascading, and generating a 3D region candidate; collecting interest points near the 3D region candidate in the original point cloud, interpolating context features of the interest points from the 3D sparse feature column and the 2D feature map, and guiding feature learning of a backbone network by using semantic segmentation supervision; setting a position-enhanced RoI-corner pooling module to extract 3D RoI features of each 3D region candidate, refining the 3D RoI features, and generating a final 3D target detection result. The application uses accurate positioning information of original points, has a simple interpolation mode, has a small number of pooling points, and reduces corresponding overheads caused by local feature extraction of the interest points and representation of a large number of pooling points.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a 3D target detection method, system, medium, and terminal. Background Technology

[0002] Object detection is an important area in computer vision, with broad application prospects and market value. With the development of various sensor technologies and automotive technologies, the role of 3D object detection in autonomous driving is gradually becoming apparent. One of the commonly used sensors for 3D object detection tasks in autonomous driving is LiDAR, which corresponds to unordered and sparse LiDAR point clouds. Due to the ability of LiDAR point clouds to accurately acquire geometric information of the environment, and the decreasing cost of LiDAR, high-performance two-stage 3D object detection methods have gradually become a research focus both domestically and internationally.

[0003] Existing two-stage 3D object detection methods first use a voxel-based feature extraction backbone network to generate 3D region candidates in the BEV plane, and then refine the 3D region proposals in 3D space based on point representations. In previous high-performance two-stage detection methods, accurate point representations played a crucial role, which can be summarized into two main key steps: scene-to-keypoint feature encoding and using uniformly distributed pooled points to extract 3D region proposal features. 1. Scene-to-keypoint feature encoding: Encoding the entire scene features into a small number of keypoints, with the granularity of the keypoints closely related to the feature extraction backbone network; 2. 3D RoI feature extraction in 3D space: Representing the 3D region proposals as uniformly distributed points or voxels, and then using a point-based R-CNN process for point cloud region pooling.

[0004] Previous two-stage 3D object detection methods have relied on three key factors to improve performance: 1. Precise location information from keypoints preserves rich 3D structural information; 2. Each 3D region is proposed as a pooling point densely distributed in 3D space; 3. Local features of pooling points are accumulated based on keypoints. However, relying on a small number of keypoints affects the detection performance of small objects, while a large number of pooling points consumes significant computational and storage resources for the entire two-stage 3D object detection system. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a 3D target detection method, system, medium, and terminal.

[0006] According to one aspect of the present invention, a 3D target detection method is provided, comprising a first stage and a second stage;

[0007] The first stage includes:

[0008] Extract 3D sparse feature columns based on voxels from the original LiDAR point cloud, and after densification, transform it into a 2D feature map under the BEV view. Use the neck network to generate multi-scale feature maps, cascade them and send them into the detection head to generate 3D region candidates.

[0009] Interest points near the candidate 3D region are collected in the original LiDAR point cloud, and the context features of the interest points are interpolated from the 3D sparse feature column and the 2D feature map. Semantic segmentation is used to supervise the feature learning in the first stage.

[0010] The second stage includes:

[0011] Based on the interest points containing the context features and the 3D region candidates, a location-enhanced RoI-corner pooling module is set up to extract 3D RoI features from the 3D region candidates, refine the 3D region candidates, and generate the final 3D object detection result.

[0012] Preferably, the step of extracting 3D sparse feature columns based on voxels from the original LiDAR point cloud, densifying them, and converting them to a BEV viewpoint to obtain a 2D feature map, using a neck network to generate multi-scale feature maps, cascading them, and then feeding them into the detection head to generate 3D region candidates includes:

[0013] Acquire raw airborne LiDAR point cloud (L), point cloud voxel converter (VE) L 3D Sparse Backbone Network Feature Extractor Bone V and a 2D CNN-based neck network feature extractor p ;

[0014] The LiDAR point cloud L is input into the point cloud voxel converter VE. L The 3D sparse voxel feature representation V of the point cloud L is obtained;

[0015] The 3D sparse voxel feature representation V is input into the 3D sparse backbone network feature extractor Bone. V A multi-scale sparse feature column {V1,V2,V3,V4} is generated, and after densification and flattening, a 2D feature map B4 is obtained under the BEV (Body Equivalent Value). The neck network feature extractor of the 2DCNN is then used. p Multi-scale features are obtained, cascaded, and then fed into the detection head to generate 3D region candidates.

[0016] Preferably, the step of acquiring interest points near the candidate 3D region in the original LiDAR point cloud and interpolating the context features of the interest points from the 3D sparse feature column and the 2D feature map includes:

[0017] Based on the 3D sparse feature column and the 2D feature map B4, a 3D sparse interpolation layer and a bilinear interpolation layer are constructed respectively.

[0018] Collect the original points near the candidate 3D region as points of interest;

[0019] The 3D sparse interpolation layer is used to extract voxel-based context features of the interest points, and the bilinear interpolation layer is used to extract context features that expand the receptive field.

[0020] The features that have passed through the 3D sparse interpolation layer and the bilinear interpolation layer are fed into the multilayer perceptron (MLP) to reduce the number of channels and obtain the final context features of the interest points.

[0021] Preferably, supervising feature learning in the first stage using semantic segmentation includes:

[0022] The detection head is a center-based detection head, and its loss function is constructed. in, For heatmap loss, L off L z L size and L ori , respectively, are the center position deviation, height position offset, target size offset, and target orientation loss functions relative to their respective target ground truth values, and λ is the weighting coefficient.

[0023] Preferably, the step of setting a location-enhanced RoI-corner pooling module based on interest points with context features and the 3D region candidates to extract 3D RoI features for each 3D region candidate, refining the 3D region candidates, and generating the final 3D object detection result includes:

[0024] Eight corner points and 19 intermediate transition points are extracted from the 3D region candidates as pooling points, which serve as key points for each 3D region candidate.

[0025] At the points of interest, local features of 27 pooling points are obtained using a point-based local feature extractor.

[0026] The location information of the candidate center points in the 3D region is used to enhance the local features of the 27 pooling points and reduce the impact of the sparse distribution of pooling points on the detection performance.

[0027] The enhanced local features of the pooling points are fed into a two-layer MLP to extract 3D RoI features, and then bounding box regression and confidence prediction are performed.

[0028] Preferably, the enhanced key point information includes: concatenating the relative position information of the 3D region candidate center point and the pooling point to the feature channel of the pooling point.

[0029] Preferably, regression and confidence estimation are performed on the 3D region candidates, and the bounding box regression loss L' is calculated. reg And category confidence loss L' cls Construct the loss function L refinement =L' cls +L' reg .

[0030] According to a second aspect of the present invention, a 3D target detection system is provided, comprising:

[0031] The 3D region candidate generation module extracts 3D sparse feature columns based on voxels from the original LiDAR point cloud, densifies them, and transforms them into 2D feature maps under the BEV view. It then uses a neck network to generate multi-scale feature maps, which are cascaded and fed into the detection head to generate 3D region candidates.

[0032] The Context Feature Enhancement Module collects interest points near the 3D region candidate in the original LiDAR point cloud, interpolates the context features of the interest points from the 3D sparse feature column and the 2D feature map, and supervises the feature learning of the 3D region candidate generation module with semantic segmentation.

[0033] The target detection module, based on interest points containing the context features and the 3D region candidates, sets up a location-enhanced RoI-corner pooling module to extract 3D RoI features from the 3D region candidates, refines the 3D region candidates, and generates the final 3D target detection result.

[0034] According to a third aspect of the present invention, a terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the 3D target detection method described above, or to run the 3D target detection system described above.

[0035] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, can be used to perform the 3D target detection method described above, or to run the 3D target detection system described above.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] The 3D target detection method and system in this invention achieve high-performance and real-time 3D target detection technology by designing an efficient 3D sparse interpolation module and a RoI-corner pooling module. Specifically, in the first stage, the 3D sparse interpolation module is used to efficiently extract context features from interest points near 3D region candidates. In the second stage, a location-enhanced RoI-corner pooling module is used to extract 3D RoI features to refine 3D region candidates. This invention efficiently utilizes the precise localization information of the original points during the detection process. Compared with existing technologies, the interpolation method is simpler, and the number of pooling points is less, reducing the overhead associated with local feature extraction of interest points and the representation of numerous pooling points.

[0038] The 3D target detection method and system in this embodiment of the invention are designed with a pipeline of sequential voxel-wise context feature extraction, point-wise feature enhancement and point-wise feature accumulation, thereby completing point-voxel-based 3D target detection more efficiently.

[0039] The 3D target detection method and system in this embodiment of the invention, by designing a simple 3D sparse interpolation module, extracts fine-grained point features from coarse-grained voxel features, thereby efficiently completing the transformation between point-voxel features.

[0040] The 3D target detection method and system in this embodiment of the invention uses a RoI-corner pooling module that uses sparsely distributed corner points and their transition points to represent each 3D region candidate proposal, and uses position enhancement to reduce the impact of non-uniform distribution, thereby efficiently completing accurate 3D target detection. Attached Figure Description

[0041] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0042] Figure 1 This is a flowchart of a 3D target detection method provided in one embodiment of the present invention;

[0043] Figure 2 This is a flowchart of a 3D target detection method provided in a preferred embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the composition structure of a 3D target detection system provided in one embodiment of the present invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0046] See Figure 1 and Figure 2 The present invention provides an embodiment of a 3D target detection method, the process of which includes:

[0047] S100 extracts the voxel-based multi-scale sparse feature column {V1,V2,V3,V4} from the original LiDAR point cloud L, densifies it, and transforms it into BEV to obtain a 2D feature map B4. After concatenation, it is sent to a BEV-based detection head such as centerhead to generate high-quality 3D region candidates.

[0048] S200, based on the 3D sparse voxel feature column, 2D dense feature map B4 and 3D region candidates in S100, collects interest points near the 3D region candidates in the original point cloud L, and interpolates the context features of the interest points from the feature column and feature map, guiding the network features to learn more efficiently in a semantic segmentation supervision manner.

[0049] S300, based on the interest points and 3D region candidates in S200, sets up a location-enhanced RoI-corner pooling module to extract 3D RoI features for each 3D region candidate, and then refines the 3D region candidates to generate the final 3D object detection result.

[0050] This embodiment fully leverages the efficient representation of geometric structures using point-voxel point cloud representation. By designing an efficient 3D sparse interpolation layer and RoI-corner pooling module, 3D target detection is achieved, improving detection quality.

[0051] In a preferred embodiment of the present invention, S100 is implemented as follows:

[0052] S101, acquire raw airborne LiDAR point cloud L, point cloud voxel converter VE L 3D Sparse Backbone Network Feature Extractor Bone V and a 2D CNN-based neck network feature extractor p ;

[0053] S102, input the LiDAR point cloud L obtained in S101 into the point cloud voxel converter VE. LThe sparse voxel feature representation V of the point cloud L is obtained;

[0054] S103, input the 3D sparse voxel feature representation V obtained in S102 into the feature extractor Bone in S101. V A multi-scale sparse feature column {V1,V2,V3,V4} is generated and then densified (sparse to dense transformation, i.e., dense process, empty positions are padded with 0s) and flattened under BEV to obtain feature map B4. This is then further processed using a top-down neck network structure, i.e., a 2D CNN neck network feature extractor. p Multi-scale features are obtained, cascaded (i.e., feature channel stacking), and then fed into a commonly used BEV-based detection head to generate 3D region candidates.

[0055] This embodiment proposes a simple and elegant method for efficiently representing geometric features based on voxels, which extracts 3D region candidates on the BEV plane through point-to-voxel feature extraction.

[0056] In a preferred embodiment of the present invention, reusing the voxel-based multi-scale sparse feature column {V1,V2,V3,V4} extracted in S100 and the densed 2D feature map B4, and by designing a corresponding interpolation method, thereby assigning multi-scale context features to the interest points, may include the following steps:

[0057] S201, in 3D space, i.e. the original airborne LiDAR point cloud L, collect candidate neighboring original points in the 3D region as points of interest;

[0058] S202, based on the voxel-based sparse feature column {V1,V2,V3,V4} of S100, constructs a 3D sparse interpolation module, and extracts the voxel-based multi-scale context features {P1,P2,P3,P4} of the interest point based on the relationship between the position of the interest point and the corresponding voxel center of the voxel sparse feature column in S201.

[0059] S203, Based on the denser feature map B4, bilinear interpolation is used on the projected BEV plane to extract context features P with a larger receptive field. BEV ;

[0060] S204, combine the multi-scale context features {P1, P2, P3, P4} from S202 with the context features P from S203. BEV After cascading, the data are fed into a multilayer perceptron (MLP) to reduce the number of channels and use them as the final context features for the points of interest. Semantic supervision is used to guide efficient feature learning based on a 3D voxel backbone network.

[0061] In this embodiment, the coarse-grained voxel-based context features are transferred to the fine-grained interest points through the designed 3D sparse interpolation layer and the commonly used bilinear interpolation layer. This is used to guide the network to learn more distinctive features and generate high-quality 3D region candidates.

[0062] In a preferred embodiment, taking a center-based detection head as an example, a loss function is constructed. The network parameters are updated by minimizing this loss function until the network converges. Generally, λ is set to 2.

[0063] In a preferred embodiment of the present invention, step S300 is implemented, which constructs a location-enhanced RoI-corner pooling module based on the interest points and their context features in step S200. A small number of pooling points are used to refine the 3D region candidates in 3D space to generate the final 3D target detection result. This includes the following steps:

[0064] S301, using the method described in S200 above, obtain interest points rich in context features for RoI-corner pooling; obtain 3D region candidate Pro points from S200. 3D Pooling points in 3D space include 8 corner points and 19 intermediate transition points.

[0065] S302, Obtain the 3D region candidate Pro from S301 above. 3D In 3D space, pooling points, including 8 corner points and 19 intermediate transition points, totaling 27 points, are used as pooling points. Based on interest points rich in context features, PointNet is used to accumulate the local features of each pooling point. The local features of each pooling point and its relative position information with respect to the corresponding 3D region candidate are concatenated. An MLP with two fully connected layers is applied to extract 3D RoI features and generate the final 3D target bounding box.

[0066] This embodiment can construct a location-enhanced RoI-corner pooling module that generates high-quality target detection using only a small number of pooling points.

[0067] In a preferred embodiment, a 3D region proposal is made. 3D Perform regression and confidence prediction, and calculate the regression loss L' reg And classification loss L' cls Construct the loss function L refinement =L' cls +L' regThe network parameters are updated by minimizing this loss function until the network converges.

[0068] Based on the same inventive concept, other embodiments of the present invention also provide a 3D target detection system, see [link to relevant documentation]. Figure 3 The module consists of an initial point-voxel feature extraction module, a 3D region proposal generation module, and a point-based object detection module.

[0069] The initial point-voxel feature extraction module extracts multi-scale sparse 3D voxel feature columns from the original airborne LiDAR point cloud based on point cloud voxels. After densification transformation to BEV, a top-down neck network is used to generate 3D region proposals, and neighboring interest points are collected from the original point cloud. Simultaneously, a 3D sparse interpolation layer is designed and combined with bilinear interpolation to assign coarse-grained context features to fine-grained interest points, guiding the aforementioned backbone network to generate high-quality 3D region candidates during backpropagation. The target detection module, based on interest points rich in context information, designs a location-enhanced RoI-corner pooling module, refining the 3D region candidates using only a small number of pooling points to generate the final 3D target detection result.

[0070] The specific implementation techniques of each module / unit in the above examples of the present invention can be referred to the steps of a 3D target detection method in the above embodiments, and will not be repeated here.

[0071] Based on the same inventive concept, other embodiments of the present invention also provide a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to perform the method in any of the embodiments, or to run the system in the embodiments.

[0072] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDRSDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0073] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0074] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0075] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0076] Based on the same inventive concept, other embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the method in any of the embodiments, or to run the system in the embodiments.

[0077] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as separate components in a communication device.

[0078] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] Based on the same inventive concept as the above embodiments, this invention provides an application example. Tests were conducted on the publicly available and well-known KITTI dataset and the large-scale 3D object detection dataset Waymo Open Dataset. Taking the KITTI dataset as an example, this dataset was divided into training, validation, and test sets, with 3712, 3769, and 7518 samples respectively. The officially provided metric tool used mAP with 40 recall points as the evaluation metric at three difficulty levels. Specifically, targets were categorized into three levels—easy, moderate, and hard—based on their distance and occlusion degree. The evaluation system tested the algorithm's detection accuracy on a class of targets at these three levels. Table 1 compares the performance of this invention on the publicly available KITTI dataset with existing 3D object detection methods on the validation set. As shown in Table 1, it can be seen that the method provided in the above embodiments of this invention achieves a better balance between accuracy and speed compared to other methods.

[0083] The Waymo dataset has two difficulty levels, LEVEL1 and LEVEL2, with LEVEL2 encompassing LEVEL1. The difficulty level is determined by the annotator and the quantized data of the object. Specifically, all point clouds without 3D annotations are first ignored. Points with 3D annotations are classified as LEVEL2 if they are marked as "hard" by the annotator or if the number of points within the bounding box is less than 5. The remaining points are classified as LEVEL1. Table 2 compares the performance of this invention with existing 3D object detection methods on the publicly available Waymo Open Dataset. As shown in Table 2, the method provided by the above embodiments of this invention significantly outperforms other methods in terms of performance improvement on the benchmark model.

[0084] Table 1

[0085]

[0086]

[0087] Table 2

[0088]

[0089] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The preferred features described above can be used in any combination without conflict.

Claims

1. A 3D target detection method, characterized in that, Including Phase 1 and Phase 2; The first stage includes: Extract 3D sparse feature columns based on voxels from the original LiDAR point cloud, and after densification, transform it into a 2D feature map under the BEV view. Use the neck network to generate multi-scale feature maps, cascade them and send them into the detection head to generate 3D region candidates. Interest points near the candidate 3D region are collected in the original LiDAR point cloud, and the context features of the interest points are interpolated from the 3D sparse feature column and the 2D feature map. Semantic segmentation is used to supervise the feature learning in the first stage. The second stage includes: Based on interest points containing the context features and the 3D region candidates, a location-enhanced RoI-corner pooling module is set up to extract 3D RoI features from the 3D region candidates, refine the 3D region candidates, and generate the final 3D object detection result, including: Eight corner points and 19 intermediate transition points are extracted from the 3D region candidates as pooling points, which serve as key points for each 3D region candidate. At the points of interest, local features of 27 pooling points are obtained using a point-based local feature extractor. The location information of the candidate center points in the 3D region is used to enhance the local features of the 27 pooling points and reduce the impact of the sparse distribution of pooling points on the detection performance. The enhanced local features of the pooling points are fed into a two-layer MLP to extract 3D RoI features, and then bounding box regression and confidence prediction are performed.

2. The 3D target detection method according to claim 1, characterized in that, The process involves extracting 3D sparse feature columns based on voxels from the original LiDAR point cloud, densifying them, and converting them to a BEV viewpoint to obtain a 2D feature map. A neck network is then used to generate multi-scale feature maps, which are cascaded and fed into the detection head to generate 3D region candidates, including: Acquire raw airborne LiDAR point cloud (L), point cloud voxel converter (VE) L 3D Sparse Backbone Network Feature Extractor Bone V and a 2D CNN-based neck network feature extractor p ; The LiDAR point cloud L is input into the point cloud voxel converter VE. L The 3D sparse voxel feature representation V of the point cloud L is obtained; The 3D sparse voxel feature representation V is input into the 3D sparse backbone network feature extractor Bone. V A multi-scale sparse feature column {V1,V2,V3,V4} is generated, and after densification and flattening, a 2D feature map B4 is obtained under the BEV (Body Equivalent Value). The neck network feature extractor of the 2D CNN is then used. p Multi-scale features are obtained, cascaded, and then fed into the detection head to generate 3D region candidates.

3. The 3D target detection method according to claim 2, characterized in that, The process of acquiring interest points near the candidate 3D region in the original LiDAR point cloud and interpolating the context features of the interest points from the 3D sparse feature column and the 2D feature map includes: Based on the 3D sparse feature column and the 2D feature map B4, a 3D sparse interpolation layer and a bilinear interpolation layer are constructed respectively. Collect the original points near the candidate 3D region as points of interest; The 3D sparse interpolation layer is used to extract voxel-based context features of the interest points, and the bilinear interpolation layer is used to extract context features that expand the receptive field. The features that have passed through the 3D sparse interpolation layer and the bilinear interpolation layer are fed into the multilayer perceptron (MLP) to reduce the number of channels and obtain the final context features of the interest points.

4. The 3D target detection method according to claim 3, characterized in that, The step of supervising feature learning in the first stage using semantic segmentation includes: The detection head is a center-based detection head, and its loss function is constructed. in, For heatmap loss, L off L z L size and L ori , respectively, are the center position deviation, height position offset, target size offset, and target orientation loss functions relative to their respective target ground truth values, and λ is the weighting coefficient.

5. The 3D target detection method according to claim 1, characterized in that, The enhancement of the local features of the 27 pooling points includes: concatenating the relative position information of the 3D region candidate center point and the pooling point to the feature channel of the pooling point.

6. The 3D target detection method according to claim 1, characterized in that, Regression and confidence estimation are performed on the 3D region candidates, and the bounding box regression loss L' is calculated. reg And category confidence loss L' cls Construct the loss function L refinement =L' cls +L' reg .

7. A 3D target detection system, characterized in that, include: The 3D region candidate generation module extracts 3D sparse feature columns based on voxels from the original LiDAR point cloud, densifies them, and transforms them into 2D feature maps under the BEV view. It then uses a neck network to generate multi-scale feature maps, which are cascaded and fed into the detection head to generate 3D region candidates. The Context Feature Enhancement Module collects interest points near the 3D region candidate in the original LiDAR point cloud, interpolates the context features of the interest points from the 3D sparse feature column and the 2D feature map, and supervises the feature learning of the 3D region candidate generation module with semantic segmentation. The target detection module, based on interest points containing the context features and the 3D region candidates, uses a location-enhanced RoI-corner pooling module to extract 3D RoI features from the 3D region candidates, refines the 3D region candidates, and generates the final 3D target detection result, including: Eight corner points and 19 intermediate transition points are extracted from the 3D region candidates as pooling points, which serve as key points for each 3D region candidate. At the points of interest, local features of 27 pooling points are obtained using a point-based local feature extractor. The location information of the candidate center points in the 3D region is used to enhance the local features of the 27 pooling points and reduce the impact of the sparse distribution of pooling points on the detection performance. The enhanced local features of the pooling points are fed into a two-layer MLP to extract 3D RoI features, and then bounding box regression and confidence prediction are performed.

8. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it can be used to perform the method of any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, this program can be used to perform the method of any one of claims 1-6.

Images