Three-dimensional target detection method based on voxel-key point fusion and hybrid anchor frame optimization

Abstract

The invention discloses a three-dimensional target detection method based on voxel-key point fusion and hybrid anchor frame optimization. The method comprises the following steps: generating a candidate region based on a multi-head hybrid anchor frame based on an automatic driving scene; obtaining global voxel-key point multi-source features according to an SDF-PS sampling method based on the candidate region; the global voxel-key point multi-source features are transmitted to uniformly distributed RoI grid points, and a generated candidate region is refined to obtain a candidate frame refining module; and for a candidate frame refining module and global voxel-key point multi-source feature aggregation, carrying out further regression correction on the candidate region to obtain a three-dimensional target detection result.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving and machine vision technology, specifically to a three-dimensional target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization. Background Technology

[0002] 3D object detection relies on the rich spatial information in point cloud data and is a core technology in the fields of autonomous driving and machine vision. Directly utilizing raw point clouds leads to a waste of computational resources, while voxelization methods, although efficient, are prone to losing local details. In recent years, methods that fuse points and voxels, such as PV-RCNN, have gradually emerged, but they still have shortcomings in areas such as balanced multi-object detection and appropriate feature fusion. Traditional anchor-box methods can capture large targets well, but have a low matching rate for small targets; anchor-box-less methods, while improving sensitivity to small targets, suffer from errors in direction prediction for large targets due to the lack of geometric constraints. Summary of the Invention

[0003] The purpose of this invention is to provide a 3D target detection method based on voxel-keypoint fusion and hybrid anchor box optimization, in order to solve the problems in existing 3D target detection methods, such as the loss of local details in voxelization methods, the waste of computational resources due to direct use of point clouds, and the imbalance of single anchor box strategies for target detection at different scales. This invention can achieve balanced detection of multi-class and multi-scale targets, improve the robustness of target detection in complex driving scenarios, and meet the needs of accurate detection of multiple types of targets such as cars, pedestrians, and cyclists in different road conditions such as urban roads, rural roads, and park roads in the core scenario of autonomous driving environment perception. It is also adapted to the perception requirements of various operating modes of autonomous driving, such as low-speed driving, high-speed cruising, and congested following.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: A 3D target detection method based on voxel-keypoint fusion and hybrid anchor box optimization includes the following steps: Step 1: Generate candidate regions based on multi-head hybrid anchor boxes for autonomous driving scenarios; Step 2: Based on the candidate regions generated in Step 1, obtain global voxel-keypoint multi-source features using the SDF-PS sampling method; Step 3: Transfer the global voxel-keypoint multi-source features obtained in Step 2 to the uniformly distributed RoI grid points to refine the generated candidate regions, thus obtaining the candidate box refinement module; Step 4: Based on the candidate box refinement module generated in Step 3 and the global voxel-key point multi-source feature aggregation obtained in Step 2, the candidate regions are further regressed and corrected to obtain the 3D target detection results.

[0005] Furthermore, step 1, which generates candidate regions based on multi-head hybrid anchor boxes for the autonomous driving scenario, includes the following steps: Step 1.1: Obtain 3D features based on the autonomous driving scenario, compress the 3D features to a 2D BEV plane, and obtain 2D features; Step 1.2: Based on two-dimensional features, generate candidate regions using multi-head hybrid anchor boxes.

[0006] Further, in step 1.1, three-dimensional features are obtained based on the autonomous driving scenario, and these three-dimensional features are compressed onto a two-dimensional BEV plane to obtain two-dimensional features, specifically as follows: The collected 3D point cloud data of real autonomous driving scenarios is processed through multi-level feature extraction and dimensionality reduction using a 3D sparse convolutional network to output 3D features, which are represented as follows:

[0007] in, L 3. W 3 represents the resolution after downsampling in the X and Y directions using a 3D sparse convolutional network, respectively; H 3 indicates the downsampling resolution in the Z direction; C 3 represents the number of feature channels output by the last layer of the 3D sparse convolutional network; F 3D R represents the output 3D features; R is the resolution of the 3D point cloud data in the spatial dimension. The three-dimensional features are compressed along the Z-axis to form two-dimensional features, as follows:

[0008] in, f compress This indicates a compression operation along the Z-axis. C BEV F represents the number of channels in the compressed two-dimensional feature map; BEV It is a two-dimensional feature; In step 1.2, candidate regions are generated based on two-dimensional features using multi-head hybrid anchor boxes, specifically as follows: A multi-head hybrid anchor box method combining anchor box detection head and anchor box-free detection head is adopted. First, the two-dimensional features are classified to determine the existence of the target and obtain positive sample regions. Then, target category recognition and bounding box refinement are performed on the positive sample regions to generate output candidate boxes of anchor box detection head and anchor box-free detection head. The output candidate boxes of anchor box detection head and anchor box-free detection head are merged to obtain a candidate box set. The candidate boxes with the highest scores are selected, redundant boxes are removed, and finally the candidate regions are output.

[0009] Furthermore, in step 2, based on the candidate regions generated in step 1, global voxel-keypoint multi-source features are obtained using the SDF-PS method, including the following steps: Step 2.1: Process the point cloud and sample set of the candidate region, update the SDF-PS comprehensive distance, and obtain the sample set after fusing the SDF-PS samples. S ; Step 2.2: Based on the sample set S And multi-scale voxel features of candidate regions, construct a voxel-keypoint feature association mechanism, and obtain voxel fusion features corresponding to all keypoints; Step 2.3: Extend the voxel fusion features in Step 2.2 to obtain global voxel-keypoint multi-source features.

[0010] Furthermore, step 2.1 specifically includes: First, the point cloud is preliminarily processed to extract semantic feature representations for each point. , and Let these be the minimum and maximum values ​​of the semantic features of all points, respectively, and then normalize the semantic features to the range [0,1], as follows:

[0011] Then initialize the sampling set from the input point cloud. Randomly select a point Add to sample set S In, as a sampling set S The initial point is as follows:

[0012] in, j It is the first j There are N sampling points, where N is the number of sampling points; Then, the input point cloud For each remaining point, initialize it with the sample set. S The distance between points in the middle; Then perform SDF-PS integrated distance update for the input point cloud. midpoint Its relationship with the sampling set S midpoint The overall distance is updated as follows:

[0013] Then iterative sampling is performed. In each iteration, points that have not been sampled are selected from the current sampling set. S The point with the largest overall distance will be used as the next sampling point. Add this point to the sampling set S The details are as follows:

[0014] Then update the remaining unsampled points and the sample set. S The combined distance is used to repeat the above process until the number of points in the sampling set reaches the target number. M Finally, the sample set after fusion sampling SDF-PS is obtained. S .

[0015] Furthermore, in step 2.2, the sampling set S The key points and multi-scale voxel features of the candidate regions are linked to each other, enabling the allocation of multi-scale voxel features to the key points obtained through fusion sampling, thereby obtaining the voxel fusion features corresponding to all key points; specifically: First, for each scale voxel feature, find the voxel corresponding to the key point within the preset range and record its voxel feature. Then, stitch together the voxel features of each key point at different scales to obtain the voxel fusion feature corresponding to all key points. Step 2.3 expands the voxel fusion features to obtain global voxel-keypoint multi-source features. , is represented as:

[0016] in, Features derived from point clouds; Key point features obtained from BEV; This is a characteristic of voxel fusion; It is a global voxel-keypoint multi-source feature.

[0017] Further, step 3 transfers the global voxel-keypoint multi-source features to the uniformly distributed RoI grid points to refine the generated candidate regions, resulting in a candidate box refinement module, which includes the following steps: Step 3.1: Transfer the global voxel-keypoint multi-source features to the uniformly distributed RoI grid points, perform local neighborhood feature aggregation, and obtain the candidate box feature representation; Step 3.2: Use the real bounding boxes in the autonomous driving scenario to filter the feature representation of candidate bounding boxes and obtain the candidate bounding box refinement module.

[0018] Furthermore, step 3.1 specifically includes: First, key points within the neighborhood of the RoI grid points are retrieved, forming a local neighborhood feature set, represented as follows:

[0019] in, r g The search radius; g k For each RoI grid point; S1 represents the global set of key points; N kg It is a set of local neighborhood features; r g The size of the candidate region is dynamically adjusted. Then, key point features within the neighborhood are collected, and a set of neighborhood features at each grid point is constructed. The specific representation is as follows:

[0020] in, Represents the multi-source features of each global voxel-keypoint within the neighborhood; Max pooling is used to globally aggregate the neighborhood feature sets of grid points to generate the final feature, as shown below:

[0021] The final features of all grid points within the candidate region are concatenated to obtain the candidate bounding box feature representation.

[0022] Further, step 3.2 uses the feature representations of real bounding boxes in the autonomous driving scenario to filter candidate bounding boxes, resulting in a candidate bounding box refinement module, specifically: Let the feature representation of the k-th candidate box be... b k Represented as:

[0023] in, The center coordinates of the candidate bounding box feature representation in three-dimensional space; The size parameter representing the feature expression of the candidate box; The orientation angle of the candidate bounding box feature representation; Realistic frames in autonomous driving scenarios b GT Represented as:

[0024] in, The coordinates of the center position of the true bounding box in 3D space. The width, height, and length of the actual frame. The orientation angle of the actual bounding box; The residual between the candidate bounding box feature representation and the ground truth bounding box is defined as follows:

[0025]

[0026]

[0027]

[0028]

[0029]

[0030]

[0031]

[0032] in, The residual of the candidate box feature representation on the center coordinates. The residuals in size of the candidate box feature representation. The residual of the candidate box feature representation in the orientation angle; The candidate box refinement module is obtained by filtering the candidate box feature representation based on the residual between the candidate box feature representation and the ground truth box.

[0033] Furthermore, in step 4, during the further regression correction of the candidate regions, the total loss function used includes the anchor box branch loss function, the anchorless branch loss function, and the candidate box fusion and refinement branch loss function. The anchor frame branch loss function is calculated as follows:

[0034] in, It is the anchor box branch loss function. It is a classification loss function. It is the regression loss function in the anchor box branch;

[0035] In the formula, p t For positive samples, α To balance the hyperparameters of positive and negative samples, c For focusing parameters;

[0036] In the formula, t For the parameters of the prediction box, t* For the true bounding box parameters, Smooth-L1 represents the weighted summation; The specific calculation for the anchorless branch loss function is as follows:

[0037] in, It is an anchorless branch loss function. It's heatmap loss. It is the regression loss function in the anchorless branch;

[0038] In the formula, N f This refers to the number of pixels in the heatmap. H j To predict heatmaps, H * j This is a real heat map;

[0039] In the formula, To predict residuals, For true residuals, x、y、z The coordinates of the center position, w , h , l The width, height, and length are respectively. i The orientation angle is usually expressed in radians; The loss function for candidate box fusion and refinement branches is calculated as follows:

[0040] In the formula, To fuse candidate boxes and refine branch loss functions, It is a regression loss function. It is the direction angle loss function. It is a classification loss function. l 1. l 2 and l 3 represents the weighting coefficients of each part of the loss function; The total loss function is calculated as follows:

[0041] In the formula, It is the total loss function. l a and l f These are the loss weights for the anchor frame detection head and the non-anchor frame detection head, respectively; By minimizing the total loss function, 3D object detection is achieved, and the 3D object detection result is obtained.

[0042] Compared with the prior art, the present invention has the following beneficial technical effects: This invention proposes a two-stage 3D target detection network, VKP-HADNet, based on voxel-keypoint fusion and hybrid anchor box detection head optimization. It fully leverages the advantages of voxels and raw point clouds to achieve feature complementarity, and designs a hybrid anchor box strategy to achieve balanced detection of targets at different scales. This significantly enhances the network's robustness in target detection in complex road scenarios for autonomous driving (such as urban intersections, residential roads, city streets, and pedestrian streets). It provides reliable technical support for core application scenarios such as autonomous driving environmental perception and machine vision intelligent monitoring, helping to improve the environmental perception accuracy and operational safety of autonomous driving systems. In particular, it improves the accuracy of detecting small pedestrian targets, effectively adapting to the perception needs of complex road environments in autonomous driving systems. It provides accurate target location, size, and attitude information to the autonomous driving decision-making and planning module, validating its effectiveness. Attached Figure Description

[0043] The accompanying drawings are provided to further understand the invention and constitute a part of this invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0044] Figure 1 This is a schematic diagram of the VKP-HADNet framework for the three-dimensional target detection network of this invention; Figure 2 This is a schematic diagram of step 1.2 based on a hybrid detection head with anchors and no anchors; Figure 3 This is a flowchart of the SDF-PS module in step 2; Figure 4 This is a schematic diagram of the RoI mesh pooling module in step 3; Figure 5(a) is an image of a real-world scene at a complex traffic intersection; Figure 5(b) shows the SECOND detection results for images of complex traffic intersections; Figure 5(c) shows the Part-A2 detection results for complex traffic intersections; Figure 5(d) shows the PV-RCNN detection results at complex traffic intersections; Figure 5(e) shows the detection results of the algorithm of this invention at complex traffic intersections; Figure 6(a) is an image of a real-world scenario in a complex residential environment; Figure 6(b) shows the SECOND detection results for images of complex residential environments; Figure 6(c) shows the Part-A2 test results for complex residential environments; Figure 6(d) shows the PV-RCNN detection results in a complex residential environment; Figure 6(e) shows the detection results of the algorithm of this invention in complex residential environments; Figure 7(a) is an image of a real-world scene in a complex street environment; Figure 7(b) shows the SECOND detection results for complex street environment images; Figure 7(c) shows the Part-A2 detection results for complex street environments; Figure 7(d) shows the PV-RCNN detection results in complex street environments; Figure 7(e) shows the detection results of the algorithm of this invention in complex street environments; Figure 8(a) is an image of a complex pedestrian street scene; Figure 8(b) shows the SECOND detection results for complex pedestrian street images; Figure 8(c) shows the Part-A2 detection results for complex pedestrian streets; Figure 8(d) shows the PV-RCNN detection results for complex pedestrian streets; Figure 8(e) shows the detection results of the algorithm of this invention for complex pedestrian streets. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0047] This invention provides a 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization, comprising the following steps: Step 1: Generate candidate regions based on multi-head hybrid anchor boxes for autonomous driving scenarios; specifically including the following steps: Step 1.1: Obtain 3D features based on the autonomous driving scenario, compress the 3D features to a 2D bird's eye view (BEV) plane, and obtain 2D features; The collected 3D point cloud data of real autonomous driving scenarios is processed through multi-level feature extraction and dimensionality reduction using a 3D sparse convolutional network to output a 3D feature volume, which is represented as follows:

[0048] in, L 3. W 3 represents the resolution after downsampling in the X and Y directions using a 3D sparse convolutional network, respectively; H 3 indicates the downsampling resolution in the Z direction; C 3 represents the number of feature channels output by the last layer of the 3D sparse convolutional network; F 3D R represents the output 3D feature volume, where R is the resolution of the point cloud data in the spatial dimension.

[0049] The three-dimensional feature volume is compressed along the Z-axis to form a two-dimensional feature, as follows:

[0050] in, f compress This indicates a compression operation along the Z-axis. C BEV F represents the number of channels in the compressed two-dimensional feature map; BEV It is a two-dimensional feature; Step 1.2: Based on two-dimensional features, generate candidate regions using multi-head hybrid anchor boxes; A multi-head hybrid anchor box method combining anchor box detection head and anchor box-free detection head is adopted. First, the two-dimensional features are classified to determine the existence of the target and obtain positive sample regions. Then, target category recognition and bounding box refinement are performed on the positive sample regions to generate output candidate boxes of anchor box detection head and anchor box-free detection head. The output candidate boxes of anchor box detection head and anchor box-free detection head are merged to obtain a candidate box set. The candidate boxes with the highest scores are selected, redundant boxes are removed, and finally the candidate regions are output.

[0051] Step 2: Based on the candidate regions generated in Step 1, obtain global voxel-keypoint multi-source features using the Semantic-Distance Fusion Point Sampling (SDF-PS) method; specifically, this includes the following steps: Step 2.1: Process the point cloud and sampling set of the candidate region, update the comprehensive distance of the semantic-distance fusion keypoint sampling method (Semantic-Distance Fusion Point Sampling, SDF-PS), and obtain the sampling set after fusing the sampling semantic-distance fusion keypoint sampling method (Semantic-Distance Fusion Point Sampling, SDF-PS). S ; First, the point cloud is preliminarily processed to extract semantic feature representations for each point. , and These are the minimum and maximum values ​​of the original semantic features for all points, respectively. The semantic features are then normalized to the range [0,1], as follows:

[0052] Then initialize the sampling set from the input point cloud. Randomly select a point Add to sample set S In, as a sampling set S The initial point is as follows:

[0053] Where j is the j-th sampling point, and N is the number of sampling points.

[0054] Then, to For each remaining point, initialize it with the sample set. S The distance between points in the equation.

[0055] Then, semantic-distance fusion point sampling (SDF-PS) is performed to comprehensively update the distance for the point cloud. midpoint Its relationship with the sampling set S midpoint s j The minimum comprehensive distance is updated as follows:

[0056] Then iterative sampling is performed. In each iteration, points that have not been sampled are selected from the current sampling set. S The point with the largest overall distance will be used as the next sampling point. Add this point to the sampling set S The details are as follows:

[0057] Then update the remaining unsampled points and S Find the minimum comprehensive distance, and repeat the above process until the number of points in the sampling set reaches the target number. M Finally, the sample set obtained by the keypoint sampling method based on semantic-distance fusion (SDF-PS) is obtained. S, M Take 2048.

[0058] Step 2.2: Based on the sample set S And multi-scale voxel features of candidate regions, construct a voxel-keypoint feature association mechanism, and obtain voxel fusion features corresponding to all keypoints; Sample set S The key points and multi-scale voxel features of the candidate regions are linked to each other, enabling the allocation of multi-scale voxel features to the key points obtained through fusion sampling, thereby obtaining the voxel fusion features corresponding to all key points. Specifically: First, for each scale voxel feature, find the voxel corresponding to the key point within the preset range and record its voxel feature. Then, stitch together the voxel features of each key point at different scales to obtain the voxel fusion feature corresponding to all key points. Step 2.3: Extend the voxel fusion features in Step 2.2 to obtain global voxel-keypoint multi-source features; The voxel fusion features are extended to obtain global voxel-keypoint multi-source features. ,

[0059]

[0060] in, Features derived from point clouds; Key point features obtained from the Bird's Eye View (BEV) image; This is a characteristic of voxel fusion; It is a global voxel-keypoint multi-source feature.

[0061] Step 3: Transfer the global voxel-keypoint multi-source features obtained in Step 2 to the uniformly distributed set of abstract global keypoint-mesh Region of Interest (RoI) grid points, refine the generated candidate regions, and obtain the candidate box refinement module; The global voxel-keypoint multi-source features are transferred to a uniformly distributed set of abstract global keypoint-mesh Region of Interest (RoI) grid points. The generated candidate regions are then refined to obtain the candidate box refinement module, which includes the following steps: Step 3.1: Transfer the global voxel-keypoint multi-source features to the uniformly distributed set of abstract global keypoint-grid Region of Interest (RoI) grid points, perform local neighborhood feature aggregation, and obtain the candidate box feature representation; First, key points within the neighborhood of grid points in the global keypoint-grid region of interest (RoI) based on set abstraction are retrieved, forming a local neighborhood feature set, represented as follows:

[0062] in, The search radius; For each grid point; A global set of key points; It is the set of local neighborhood features formed by the region.

[0063] r g The size of the candidate bounding box is dynamically adjusted. Then, key point features within the neighborhood are collected, and a set of neighborhood features for each grid point is constructed. Specifically, this is represented as follows:

[0064] in, f j global This represents the multi-source features of each global voxel-keypoint within the neighborhood.

[0065] The neighborhood features are globally aggregated using max pooling, resulting in the following final feature representation:

[0066] The features of all grid points within the candidate box are concatenated to obtain the candidate box feature representation.

[0067] Step 3.2: Use the real bounding boxes in the autonomous driving scenario to filter the feature representation of candidate bounding boxes and obtain the candidate bounding box refinement module.

[0068] By using ground truth bounding boxes in autonomous driving scenarios to filter candidate bounding box feature representations, a candidate bounding box refinement module is obtained, specifically: Let the feature representation of the k-th candidate box be... b k Represented as:

[0069] in, The coordinates of the center of the candidate box in 3D space; The size parameter of the candidate box; The orientation angle of the candidate box.

[0070] Realistic frames in autonomous driving scenarios b GT Represented as:

[0071] in, The coordinates of the center position of the true bounding box in 3D space. The width, height, and length of the actual frame. The orientation angle of the actual bounding box; The residual between the candidate bounding box and the ground truth bounding box is defined as follows:

[0072]

[0073]

[0074]

[0075]

[0076]

[0077]

[0078]

[0079] in, The residual of the candidate box at the center coordinates. The residual in size of the candidate box. This represents the residual of the candidate box in the orientation angle.

[0080] The candidate box refinement module is obtained by filtering the candidate box feature representation based on the residual between the candidate box feature representation and the ground truth box.

[0081] Step 4: Based on the candidate box refinement module generated in Step 3 and the global voxel-key point multi-source feature aggregation obtained in Step 2, the candidate regions are further regressed and corrected to obtain the 3D target detection results.

[0082] In the process of further regression correction of candidate regions, the total loss function used includes anchor box branch loss function, anchorless branch loss function and candidate box fusion and refinement branch loss function; The anchor frame branch loss function is calculated as follows:

[0083] in, It is the anchor box branch loss function. It is a classification loss function. It is the regression loss function in the anchor box branch;

[0084] In the formula, p t For positive samples, α To balance the hyperparameters of positive and negative samples, c For focusing parameters;

[0085] In the formula, t For the parameters of the prediction box, t* For the true bounding box parameters, Smooth-L1 represents the weighted summation; The specific calculation for the anchorless branch loss function is as follows:

[0086] in, It is an anchorless branch loss function. It's heatmap loss. It is the regression loss function in the anchorless branch;

[0087] In the formula, N f This refers to the number of pixels in the heatmap. H j To predict heatmaps, H * j This is a real heat map;

[0088] In the formula, To predict residuals, For true residuals, x、y、z The coordinates of the center position, w , h , l The width, height, and length are respectively. i The orientation angle is usually expressed in radians; The loss function for candidate box fusion and refinement branches is calculated as follows:

[0089] In the formula, To fuse candidate boxes and refine branch loss functions, It is a regression loss function. It is the direction angle loss function. It is a classification loss function. l 1. l 2 and l 3 represents the weighting coefficients of each part of the loss function; The total loss function is calculated as follows:

[0090] In the formula, It is the total loss function. l a and l f These are the loss weights for the anchor frame detection head and the non-anchor frame detection head, respectively; By minimizing the total loss function, 3D object detection is achieved, and the 3D object detection result is obtained.

[0091] Figure 2 This is a hybrid anchor detection head that combines the advantages of anchor frame detection and anchor frameless detection. The detection head consists of three parts: an anchor frame detection head, an anchor frameless detection head, and a hybrid detection result fusion unit.

[0092] Figure 3 This is a flowchart of the SDF-PS module. The module consists of three parts: a sampling layer, a combination layer, and a feature extraction layer. It captures local and global features of the point cloud using farthest point sampling, neighborhood construction, and feature aggregation.

[0093] Figure 4 The RoI grid pooling module generates rich candidate box feature vectors by generating uniform grid points within the candidate boxes, aggregating neighborhood features, performing SDF-PS encoding, and fusing features, thus providing support for candidate box refinement.

[0094] Figures 5(a) to 5(e), 6(a) to 6(e), 7(a) to 7(e), and 8(a) to 9(e) show the detection performance of the algorithm of this invention in different scenarios, and a visual comparison with the classic detection algorithm SECOND and two other two-stage algorithms (PV-RCNN and Part-A2). In the detection results, the green 3D boxes generated represent vehicle targets, the blue 3D boxes represent pedestrian targets, and the yellow 3D boxes represent cyclist targets. Experiments show that the algorithm of this invention has better detection performance for distant targets, occluded targets, and small targets in complex traffic intersections, residential environments, street environments, and pedestrian streets. The number of positive detection boxes and accuracy are both outstanding, and the false negative rate and missed detection rate are significantly reduced.

[0095] This invention was developed on a computer with an Intel Core i7-12700H 2.60GHz CPU, an RTX 3060 GPU, and 32GB of RAM. The operating system was Ubuntu 20.04. The algorithm was built using the PyTorch 1.13.0 deep learning framework, with Adam as the optimizer, an initial learning rate of 0.003, a weight decay of 0.01, a batch size of 4, and an epoch of 80.

[0096] To verify the performance of the algorithm of this invention for pedestrian target detection, it was compared with other publicly available pedestrian detection accuracy algorithms on the KITTI official test set, as shown in Table 1.

[0097] Table 1. Pedestrian Target Detection Accuracy (AP) on the KITTI Test Set 3D (%)contrast

[0098] As shown in the table above, the total mAP of the algorithm presented in this invention outperforms other comparative methods when detecting pedestrian targets on the KITTI test set. Specifically, in easy difficulty scenarios, the AP value of the algorithm presented in this invention is 52.93%, which is 0.76% higher than the two-stage algorithm PV-RCNN and 1.01% higher than Point-GNN. In medium difficulty scenarios, the AP value of the algorithm presented in this invention is 43.77%, on par with Point-GNN and 0.48% higher than PV-RCNN. In difficult difficulty scenarios, the AP value of the algorithm presented in this invention is 40.42%, which is 0.13% higher than PV-RCNN and 0.28% higher than Point-GNN. In summary, the algorithm presented in this invention maintains good detection performance across all difficulty levels, and is particularly advantageous in medium and difficult scenarios, verifying its effectiveness in handling pedestrian target detection tasks with sparse local point clouds and varied poses.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation of the invention, but these changes, modifications or equivalent substitutions are all within the scope of protection of the pending claims of the invention.

Claims

1. A 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization, characterized in that, Includes the following steps: Step 1: Generate candidate regions based on multi-head hybrid anchor boxes for autonomous driving scenarios; Step 2: Based on the candidate regions generated in Step 1, obtain global voxel-keypoint multi-source features using the SDF-PS sampling method; Step 3: Transfer the global voxel-keypoint multi-source features obtained in Step 2 to the uniformly distributed RoI grid points to refine the generated candidate regions, thus obtaining the candidate box refinement module; Step 4: Based on the candidate box refinement module generated in Step 3 and the global voxel-key point multi-source feature aggregation obtained in Step 2, the candidate regions are further regressed and corrected to obtain the 3D target detection results.

2. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 1, characterized in that, Step 1, which generates candidate regions based on multi-head hybrid anchor boxes for autonomous driving scenarios, includes the following steps: Step 1.1: Obtain 3D features based on the autonomous driving scenario, compress the 3D features to a 2D BEV plane, and obtain 2D features; Step 1.2: Based on two-dimensional features, generate candidate regions using multi-head hybrid anchor boxes.

3. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 2, characterized in that, In step 1.1, three-dimensional features are obtained based on the autonomous driving scenario. These three-dimensional features are then compressed onto a two-dimensional BEV plane to obtain two-dimensional features, specifically: The collected 3D point cloud data of real autonomous driving scenarios is processed through multi-level feature extraction and dimensionality reduction using a 3D sparse convolutional network to output 3D features, which are represented as follows: in, L 3. W 3 represents the resolution after downsampling in the X and Y directions using a 3D sparse convolutional network, respectively; H 3 indicates the downsampling resolution in the Z direction; C 3 represents the number of feature channels output by the last layer of the 3D sparse convolutional network; F 3D R represents the output 3D features; R is the resolution of the 3D point cloud data in the spatial dimension. The three-dimensional features are compressed along the Z-axis to form two-dimensional features, as follows: in, f compress This indicates a compression operation along the Z-axis. C BEV F represents the number of channels in the compressed two-dimensional feature map; BEV It is a two-dimensional feature; In step 1.2, candidate regions are generated based on two-dimensional features using multi-head hybrid anchor boxes, specifically as follows: A multi-head hybrid anchor box method combining anchor box detection head and anchor box-free detection head is adopted. First, the two-dimensional features are classified to determine the existence of the target and obtain positive sample regions. Then, target category recognition and bounding box refinement are performed on the positive sample regions to generate output candidate boxes of anchor box detection head and anchor box-free detection head. The output candidate boxes of anchor box detection head and anchor box-free detection head are merged to obtain a candidate box set. The candidate boxes with the highest scores are selected, redundant boxes are removed, and finally the candidate regions are output.

4. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 1, characterized in that, Step 2, based on the candidate regions generated in Step 1, obtains global voxel-keypoint multi-source features using the SDF-PS method, including the following steps: Step 2.1: Process the point cloud and sample set of the candidate region, update the SDF-PS comprehensive distance, and obtain the sample set after fusing the SDF-PS samples. S ; Step 2.2: Based on the sample set S And multi-scale voxel features of candidate regions, construct a voxel-keypoint feature association mechanism, and obtain voxel fusion features corresponding to all keypoints; Step 2.3: Extend the voxel fusion features in Step 2.2 to obtain global voxel-keypoint multi-source features.

5. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 4, characterized in that, Step 2.1 specifically involves: First, the point cloud is preliminarily processed to extract semantic feature representations for each point. , and Let these be the minimum and maximum values ​​of the semantic features of all points, respectively, and then normalize the semantic features to the range [0,1], as follows: Then initialize the sampling set from the input point cloud. Randomly select a point Add to sample set S In, as a sampling set S The initial point is as follows: in, j It is the first j There are N sampling points, where N is the number of sampling points; Then, the input point cloud For each remaining point, initialize it with the sample set. S The distance between points in the middle; Then perform SDF-PS integrated distance update for the input point cloud. midpoint Its relationship with the sampling set S midpoint The overall distance is updated as follows: Then iterative sampling is performed. In each iteration, points that have not been sampled are selected from the current sampling set. S The point with the largest overall distance will be used as the next sampling point. Add this point to the sampling set S The details are as follows: Then update the remaining unsampled points and the sample set. S The combined distance is used to repeat the above process until the number of points in the sampling set reaches the target number. M Finally, the sample set after fusion sampling SDF-PS is obtained. S .

6. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 4, characterized in that, In step 2.2, the sampling set will be... S The key points and multi-scale voxel features of the candidate region are linked to each other, so as to assign the multi-scale voxel features to the key points obtained by fusion sampling, thereby obtaining the voxel fusion features corresponding to all key points. Specifically: First, for each scale voxel feature, find the voxel corresponding to the key point within the preset range and record its voxel feature. Then, stitch together the voxel features of each key point at different scales to obtain the voxel fusion feature corresponding to all key points. Step 2.3 expands the voxel fusion features to obtain global voxel-keypoint multi-source features. , is represented as: in, Features derived from point clouds; Key point features obtained from BEV; This is a characteristic of voxel fusion; It is a global voxel-keypoint multi-source feature.

7. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 1, characterized in that, Step 3 transfers the global voxel-keypoint multi-source features to uniformly distributed RoI grid points, refining the generated candidate regions to obtain the candidate box refinement module, which includes the following steps: Step 3.1: Transfer the global voxel-keypoint multi-source features to the uniformly distributed RoI grid points, perform local neighborhood feature aggregation, and obtain the candidate box feature representation; Step 3.2: Use the real bounding boxes in the autonomous driving scenario to filter the feature representation of candidate bounding boxes and obtain the candidate bounding box refinement module.

8. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 7, characterized in that, Step 3.1 specifically involves: First, key points within the neighborhood of the RoI grid points are retrieved, forming a local neighborhood feature set, represented as follows: in, r g The search radius; g k For each RoI grid point; S 1 represents the global set of key points; N kg It is a set of local neighborhood features. r g The size of the candidate region is dynamically adjusted. Then, key point features within the neighborhood are collected, and a set of neighborhood features at each grid point is constructed. The specific representation is as follows: in, Represents the multi-source features of each global voxel-keypoint within the neighborhood; Max pooling is used to globally aggregate the neighborhood feature sets of grid points to generate the final feature, as shown below: The final features of all grid points within the candidate region are concatenated to obtain the candidate bounding box feature representation.

9. The 3D target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 7, characterized in that, Step 3.2 uses the feature representations of real bounding boxes in the autonomous driving scenario to filter candidate bounding box features, resulting in the candidate bounding box refinement module, specifically: Let the feature representation of the k-th candidate box be... b k Represented as: in, The center coordinates of the candidate bounding box feature representation in three-dimensional space; The size parameter representing the feature expression of the candidate box; The orientation angle of the candidate bounding box feature representation; Realistic frames in autonomous driving scenarios b GT Represented as: in, The coordinates of the center position of the true bounding box in 3D space. The width, height, and length of the actual frame. The orientation angle of the actual bounding box; The residual between the candidate bounding box feature representation and the ground truth bounding box is defined as follows: in, The residual of the candidate box feature representation on the center coordinates. The residuals in size of the candidate box feature representation. The residual of the candidate box feature representation in the orientation angle; The candidate box refinement module is obtained by filtering the candidate box feature representation based on the residual between the candidate box feature representation and the ground truth box.

10. The three-dimensional target detection method based on voxel-keypoint fusion and hybrid anchor frame optimization according to claim 1, characterized in that, In step 4, during the further regression correction of the candidate regions, the total loss function used includes the anchor box branch loss function, the anchorless branch loss function, and the candidate box fusion and refinement branch loss function. The anchor frame branch loss function is calculated as follows: in, It is the anchor box branch loss function. It is a classification loss function. It is the regression loss function in the anchor box branch; In the formula, p t For positive samples, α To balance the hyperparameters of positive and negative samples, γ For focusing parameters; In the formula, t For the predicted bounding box parameters, t* For the true bounding box parameters, Smooth-L1 represents the weighted summation; The specific calculation for the anchorless branch loss function is as follows: in, It is an anchorless branch loss function. It's heatmap loss. It is the regression loss function in the anchorless branch; In the formula, N f This refers to the number of pixels in the heatmap. H j To predict heatmaps, H * j This is a real heat map; In the formula, To predict residuals, For true residuals, x, y, z The coordinates of the center position, w , h , l The width, height, and length are respectively. θ The orientation angle is usually expressed in radians; The loss function for candidate box fusion and refinement branches is calculated as follows: In the formula, To fuse candidate boxes and refine branch loss functions, It is a regression loss function. It is the direction angle loss function. It is a classification loss function. λ 1. λ 2 and λ 3 represents the weighting coefficients of each part of the loss function; The total loss function is calculated as follows: In the formula, It is the total loss function. λ a and λ f These are the loss weights for the anchor frame detection head and the non-anchor frame detection head, respectively; By minimizing the total loss function, 3D object detection is achieved, and the 3D object detection result is obtained.

Images