A three-dimensional target detection method and system based on a progressive fine-grained refinement network

Abstract

The application discloses a three-dimensional target detection method and system based on a progressive fine-grained refinement network, and relates to the technical field of three-dimensional point cloud target detection. The method comprises the following steps: extracting voxel features through point cloud voxelization; compressing the features to obtain bird's eye view features; generating initial candidate boxes through an RPN; training a guide backbone through a mask module; constructing an L-level progressive refinement network; extracting grid point features; dynamically propagating and enhancing the features; fusing internal and external attention; performing single-level prediction; and performing box pooling fusion and output. The application strengthens the foreground through a mask guide, dynamically propagates and enhances fine granularity of features, fuses multi-level contexts through internal and external attention, and stabilizes the result through box pooling, thereby solving the problems of missing detection, inaccurate positioning and strong background interference of small targets in the prior art and being applicable to scenarios such as automatic driving and robot perception.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud target detection technology, and in particular to a 3D target detection method and system based on a progressive fine-grained refinement network. Background Technology

[0002] With the rapid development of technologies such as autonomous driving and intelligent robots, 3D target detection based on LiDAR has become a core task in environmental perception. LiDAR can provide accurate depth information, but the point cloud data it collects is characterized by sparsity, disorder, and uneven distribution.

[0003] Existing 3D object detection methods are mainly divided into single-stage and two-stage methods. Two-stage methods, such as PV-RCNN and Voxel RCNN, typically generate candidate bounding boxes first, and then refine the features of the candidate regions using grid points. However, existing technologies have the following drawbacks: 1. Feature loss problem: In the process of sparse point cloud feature extraction, as the number of network layers increases and downsampling occurs, fine-grained geometric information of distant small targets or occluded targets is easily lost, resulting in a decrease in detection accuracy; 2. Insufficient utilization of context: The existing refinement module only focuses on the current candidate region, ignores the historical context information in the multi-stage refinement process, and does not fully explore the topological dependencies within the grid points, thus failing to effectively utilize global features; 3. Background noise interference: Existing networks lack explicit supervisory signals to distinguish foreground targets from background noise. In complex scenarios (such as rainy days, foggy days, and dense crowds), background noise is easily misdetected as a target, leading to a significant decrease in detection accuracy.

[0004] Therefore, there is an urgent need to design a 3D target detection method that can effectively utilize multi-stage contextual information, enhance fine-grained feature representation, and suppress background noise. Summary of the Invention

[0005] The purpose of this invention is to overcome at least one technical problem existing in the prior art and to provide a three-dimensional target detection method and system based on a progressive fine-grained refinement network.

[0006] On one hand, embodiments of the present invention provide a three-dimensional target detection method based on a progressively fine-grained network. The method includes: step S1, acquiring three-dimensional point cloud data of the scene to be detected, performing voxelization on the three-dimensional point cloud data, and extracting voxel features through a three-dimensional sparse convolutional backbone network; step S2, compressing the voxel features along the height direction to obtain bird's-eye view features; step S3, using a region generation network to generate initial three-dimensional candidate boxes based on the bird's-eye view features; and step S4, constructing a mask segmentation module, which uses the progressively fine-grained network to extract voxel features. The bird's-eye view features are used as input, and the output is a foreground mask probability map. During the training phase, the foreground mask probability map is compared with the real foreground mask, and the mask segmentation loss is calculated using a focus loss function. The mask segmentation loss is backpropagated to the 3D sparse convolutional backbone network to guide the backbone network to focus on the foreground region and suppress background noise. During the inference phase, the mask segmentation module is pruned and removed, and does not participate in detection inference. Step S5: Construct a progressive fine-grained refinement network containing L cascaded refinement subnetworks. Each refinement subnetwork contains a RoI-grid. The system comprises a pooling unit, a feature dynamic propagation module, a cascaded perceptual attention mechanism, and a detection head; Step S6: In each refined sub-network, based on the initial 3D candidate box, grid points within the candidate box are sampled using the RoI-grid Pooling unit, and grid point features are extracted; Step S7: The grid point features are enhanced using the feature dynamic propagation module, and enhanced features are output; Step S8: The cascaded perceptual attention mechanism includes an external attention mechanism and an internal attention mechanism; the external attention mechanism generates external attention output features, and the internal attention mechanism generates internal attention output features; Step S9: The external attention output features, internal attention output features, and current-stage enhanced features are fused, and the input to the detection head predicts the current-stage 3D bounding box and confidence score; Step S10: A box pooling strategy is used to weight and fuse all 3D bounding boxes and corresponding confidence scores output by the L refined sub-networks to obtain the final 3D target detection result.

[0007] Furthermore, the method is executed by a pre-installed detection model, which includes a 3D sparse convolutional backbone network, a region generation network, a mask segmentation module, a progressive fine-grained refinement network, and a detection head. The detection model employs an end-to-end joint training approach for parameter optimization, and the total loss function used during training is defined as: ; ; in, Generate network loss for the region; For mask segmentation loss; This is the balance coefficient; Total loss; To refine network loss; Indicates the first In the refined subnetwork, the first The prediction confidence score of each candidate box; Indicates the first In the refined subnetwork, the first The true confidence scores corresponding to each candidate box; Indicates the first In the refined subnetwork, the first The regression residuals of the predicted bounding boxes of each candidate box; Indicates the first In the refined subnetwork, the first The regression residuals of the true bounding boxes corresponding to each candidate box; For confidence loss; For regression loss; For indicator functions, only when the first The first subnetwork Candidate boxes and true values Greater than the threshold Calculate the regression loss at that time.

[0008] Furthermore, the three-dimensional point cloud data in step S1 includes spatial coordinate information and reflection intensity information. The voxelization process divides the disordered three-dimensional point cloud into a regular three-dimensional grid according to a preset size. The three-dimensional sparse convolutional backbone network only performs convolution operations on non-empty voxels containing point clouds to extract voxel features.

[0009] Furthermore, in step S4, the mask segmentation module consists of two layers of multilayer perceptrons and a sigmoid activation function; the output expression of the foreground mask probability map is: ; in, Features of a bird's-eye view ( It is a two-layer, multi-layer perceptron. Let S be the Sigmoid activation function, and S be the foreground mask probability map. The expression for the focus loss function is: ; in, The label probability corresponds to the true foreground region. The actual background area corresponds to α1 and γ1 are hyperparameters used to balance the weights of positive and negative samples and reduce the weights of simple background samples, respectively.

[0010] Furthermore, in step S5, L is a positive integer greater than or equal to 2, and the L refinement subnetworks are cascaded and connected in series. The first refinement subnetwork takes the initial 3D candidate box as input, and each subsequent refinement subnetwork takes the optimized candidate box output by the previous refinement subnetwork as input, and corrects the candidate box accuracy step by step.

[0011] Furthermore, the feature dynamic propagation module in step S7 employs a multi-head self-attention mechanism to map grid point features into query, key, and value features, capturing long-distance dependencies between grid points; the output feature calculation expression of the i-th attention head is: ; The enhanced features are obtained by concatenating the output features of all attention heads, processing them through residual connections and a multilayer perceptron, and the expression is as follows: ; In the formula, grid point features , For the number of grid points, For feature channel dimension, For the query features of the i-th attention head, For the key features of the i-th attention head, The value feature of the i-th attention head, For the first The output features of each attention head For the enhanced features, It is a multilayer perceptron. Indicates the number of attention heads, Concat( ) represents the splicing operation.

[0012] Furthermore, the external attention mechanism in step S8 includes a key memory unit. AND-value memory unit The key and value feature information from the preceding stages are stored respectively. Within the current i-th level refinement subnetwork, an attention weight map is calculated for a single attention head, and the corresponding context features are retrieved using the memory unit. The calculation formula includes: ; ; in, This is the attention weight map corresponding to the j-th attention head within the i-th level refined subnetwork; This refers to the single-head contextual features obtained by relying on memory units for retrieval; and They represent the first Each attention head's memory unit key and value; This indicates a compound normalization operation; For linear transformation layer; For location embedding; For the front Each refined subnetwork outputs enhanced features; The single-head context features obtained from solving all attention heads within the same level of the refined subnetwork are concatenated, and feature mapping and integration are completed through a unified weight matrix to obtain the complete external attention output features of this level, expressed as: ; in, Indicates the number of external attention heads. Indicates the external projection layer. This is the integrated external attention output feature of the final output of the i-th level refined subnetwork.

[0013] Furthermore, the internal attention mechanism in step S8 includes: The enhanced features output by the current i-th level refined subnetwork Through three independent linear transformation layers, each mapped to a query feature. Key features Sum value characteristics ; Based on the query features Key features Sum value characteristics pass The function computes the output features of the j-th internal attention head: ; In the formula, The output feature of the j-th internal attention head within the i-th level refined subnetwork; The outputs of all internal attention heads are concatenated and projected to obtain the internal attention features: ; In the formula, The number of internal attention heads. The internal attention output features generated by the i-th level refined subnetwork. This is the internal projection layer.

[0014] Furthermore, the weighted fusion expression in step S10 is: ; ; In the formula, L represents the number of refined subnetworks. The 3D bounding box output by the i-th refined subnetwork. Let be the confidence level of the output of the i-th refined subnetwork. To determine the final confidence level, This is the final 3D bounding box.

[0015] Secondly, embodiments of the present invention provide a 3D target detection system based on a progressive fine-grained network, implemented using the aforementioned 3D target detection method based on a progressive fine-grained network. The system includes: an initial 3D candidate box construction unit, suitable for acquiring 3D point cloud data of the scene to be detected; performing voxelization processing on the 3D point cloud data; extracting voxel features through a 3D sparse convolutional backbone network; compressing the voxel features along the height direction to obtain bird's-eye view features; generating initial 3D candidate boxes based on the bird's-eye view features using a region generation network; and a mask segmentation module construction unit, suitable for the mask segmentation module. The segmentation module takes the bird's-eye view features as input and outputs a foreground mask probability map. During the training phase, the foreground mask probability map is compared with the real foreground mask, and the mask segmentation loss is calculated using a focus loss function. The mask segmentation loss is backpropagated to the 3D sparse convolutional backbone network to guide the backbone network to focus on the foreground region and suppress background noise. During the inference phase, the mask segmentation module is pruned and removed, and does not participate in detection inference. The current 3D bounding box and confidence prediction unit is suitable for constructing a progressively fine-grained refinement network containing L cascaded refinement subnetworks. Each refinement subnetwork contains a RoI-grid. The system comprises a pooling unit, a dynamic feature propagation module, a cascaded perceptual attention mechanism, and a detection head. In each refined sub-network, based on the initial 3D candidate bounding boxes, grid points within the candidate boxes are sampled and their features extracted using the RoI-gridPooling unit. The dynamic feature propagation module enhances these grid point features, outputting enhanced features. The cascaded perceptual attention mechanism includes an external attention mechanism and an internal attention mechanism. The external attention mechanism generates external attention output features, and the internal attention mechanism generates internal attention output features. These external attention output features, internal attention output features, and the current-stage enhanced features are fused and input to the detection head to predict the current-stage 3D bounding box and confidence score. A 3D object detection result output unit is used, suitable for employing a box pooling strategy to weightedly fuse all 3D bounding boxes and corresponding confidence scores output by L refined sub-networks to obtain the final 3D object detection result.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-described three-dimensional target detection method based on a progressive fine-grained network.

[0017] Fourthly, embodiments of the present invention also provide a readable storage medium, which, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to execute the above-described three-dimensional target detection method based on a progressive fine-grained refinement network.

[0018] The advantages of this invention compared to the prior art are as follows: 1. This invention constructs a mask segmentation module that only takes effect during the training phase. The network learns to distinguish between foreground objects and background noise by supervising the network through a focus loss function. The module is removed during the inference phase by pruning. Without increasing the computational load of inference, the backbone network is guided to focus on the foreground region, thereby enhancing the feature discrimination power and effectively suppressing background noise interference. 2. This invention introduces a feature dynamic propagation module, which uses a multi-head self-attention mechanism to capture long-distance dependencies between grid points, effectively recovering the loss of fine-grained geometric features caused by occlusion, sparseness or long distance, and improving the detection accuracy of small targets and occluded targets; 3. This invention designs a cascaded perception attention mechanism, which aggregates the historical feature context of all previous refinement stages through an external attention mechanism and refines the features of the current stage through an internal attention mechanism. This breaks the limitation of each stage being independent in traditional cascaded networks, makes full use of multi-stage context information, and significantly improves the target localization accuracy. 4. This invention employs a box pooling strategy to weightedly fuse the 3D bounding boxes and confidence scores output by multiple refined sub-networks. It uses the results from high-confidence stages to correct low-confidence results, smoothing out prediction jitter in a single stage and improving the stability and robustness of the detection results. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Figure 1 This is a flowchart of a three-dimensional target detection method based on a progressive fine-grained network, provided in Embodiment 1 of this application.

[0021] Figure 2 This is a schematic diagram of a detection model structure provided in Embodiment 1 of the present invention.

[0022] Figure 3 This is a schematic diagram of a progressive fine-grained network structure provided in Embodiment 1 of the present invention.

[0023] Figure 4 This is a schematic diagram of a feature dynamic propagation module structure provided in Embodiment 1 of the present invention.

[0024] Figure 5 This is a block diagram of a cascaded perception attention mechanism provided in Embodiment 1 of the present invention.

[0025] Figure 6 This is a block diagram of a multi-head external attention mechanism provided in Embodiment 1 of the present invention.

[0026] Figure 7 This is a block diagram of a multi-head internal attention mechanism provided in Embodiment 1 of the present invention.

[0027] Figure 8 This is a structural block diagram of a mask segmentation module provided in Embodiment 1 of the present invention.

[0028] Figure 9 This is a schematic diagram of a three-dimensional target detection system based on a progressive fine-grained network, provided in Embodiment 2 of this application.

[0029] Figure 10 This is a partial block diagram of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1

[0032] like Figure 1 The flowchart shown is a three-dimensional target detection method based on a progressive fine-grained network provided in this application embodiment.

[0033] As an example, the method includes: Step S1, acquiring 3D point cloud data of the scene to be detected, performing voxelization on the 3D point cloud data, and extracting voxel features through a 3D sparse convolutional backbone network; Step S2, compressing the voxel features along the height direction to obtain bird's-eye view features; Step S3, using a region generation network to generate initial 3D candidate boxes based on the bird's-eye view features; Step S4, constructing a mask segmentation module, which takes the bird's-eye view features as input and outputs a foreground mask probability map; during the training phase, comparing the foreground mask probability map with the real foreground mask, and calculating the mask segmentation loss using a focus loss function; backpropagating the mask segmentation loss to the 3D sparse convolutional backbone network to guide the backbone network to focus on the foreground region and suppress background noise; during the inference phase, pruning and removing the mask segmentation module, which does not participate in detection inference; Step S5, constructing a progressive fine-grained refinement network containing L cascaded refinement sub-networks; each refinement sub-network contains a RoI-grid. The system comprises a pooling unit, a feature dynamic propagation module, a cascaded perceptual attention mechanism, and a detection head; Step S6: In each refined sub-network, based on the initial 3D candidate box, grid points within the candidate box are sampled using the RoI-grid Pooling unit, and grid point features are extracted; Step S7: The grid point features are enhanced using the feature dynamic propagation module, and enhanced features are output; Step S8: The cascaded perceptual attention mechanism includes an external attention mechanism and an internal attention mechanism; the external attention mechanism generates external attention output features, and the internal attention mechanism generates internal attention output features; Step S9: The external attention output features, internal attention output features, and current-stage enhanced features are fused, and the input to the detection head predicts the current-stage 3D bounding box and confidence score; Step S10: A box pooling strategy is used to weight and fuse all 3D bounding boxes and corresponding confidence scores output by the L refined sub-networks to obtain the final 3D target detection result.

[0034] In some feasible implementations, combined with Figure 2 As shown, the method is executed by a pre-installed detection model, which includes a 3D sparse convolutional backbone network, a region generation network, a mask segmentation module, a progressive fine-grained refinement network, and a detection head. The detection model employs an end-to-end joint training approach for parameter optimization, and the total loss function used during training is defined as: ; ; in, Generate network loss for the region; For mask segmentation loss; This is the balance coefficient; Total loss; To refine network loss; Indicates the first In the refined subnetwork, the first The prediction confidence score of each candidate box; Indicates the first In the refined subnetwork, the first The true confidence scores corresponding to each candidate box; Indicates the first In the refined subnetwork, the first The regression residuals of the predicted bounding boxes of each candidate box; Indicates the first In the refined subnetwork, the first The regression residuals of the true bounding boxes corresponding to each candidate box; The confidence loss is calculated using binary cross-entropy loss. The regression loss is calculated using Smooth-L1 loss. For indicator functions, only when the first The first subnetwork Candidate boxes and true values Greater than the threshold Calculate the regression loss at that time.

[0035] Specifically, the training parameters include: optimizer: Adam; learning rate: , attenuation; batch: 8; number of rounds: 80 rounds.

[0036] In some feasible implementations, the three-dimensional point cloud data in step S1 includes spatial coordinate information and reflection intensity information. The voxelization process divides the disordered three-dimensional point cloud into a regular three-dimensional grid according to a preset size. The three-dimensional sparse convolutional backbone network only performs convolution operations on non-empty voxels containing point clouds to extract voxel features.

[0037] Specifically, the input is the original 3D point cloud: ;in, In three-dimensional space coordinates, The reflection intensity is 0~255, and N is the total number of points.

[0038] Specifically, the purpose of voxelization is to map disordered, sparse point clouds into regular 3D meshes, facilitating subsequent convolutional processing. Its implementation methods include, but are not limited to: setting voxel sizes; for example... The point cloud space is divided into cubic grids (voxels) according to the size; each point falls into the corresponding voxel according to its coordinates; for each non-empty voxel: the mean / feature of the internal points is calculated to form voxel features; output: sparse 3D voxel feature tensor.

[0039] Specifically, the purpose of a 3D sparse convolutional backbone network is to extract high-dimensional, structured 3D features from voxel features. Its implementation includes, but is not limited to: using a 3D Sparse CNN for multi-layer convolution and pooling: low layers: edges, geometric contours; high layers: parts, overall target features; output: 3D feature map (sparse).

[0040] In some feasible implementations, step S2, which compresses the voxel features along the height direction to obtain the bird's-eye view features, aims to compress the 3D features into a 2D plane, facilitating 2D convolution / RPN processing. Specifically, max pooling / average pooling is performed along the Z-axis (height); at each (x,y) position: features from all heights are merged; output: BEV feature map. ,shape: (Height × Width × Channel).

[0041] In some feasible implementations, step S3 utilizes a region generation network to generate initial 3D candidate boxes based on the bird's-eye view features, with the aim of generating multiple 3D candidate boxes (thick boxes) of "potential targets" on the BE.

[0042] Specific implementation methods include, but are not limited to: The above uses 2D convolution to extract candidate features; multiple anchor boxes are preset at each location, including different sizes / aspect ratios; two branches are included: a classification branch: predicting whether the anchor box is foreground / background; a regression branch: predicting the offset of the anchor box from the ground truth box; NMS (non-maximum suppression): removing overlapping boxes and retaining high-scoring boxes; output: initial 3D candidate box set. These boxes are rough, inaccurate, and lack details, and need to be refined later.

[0043] In some feasible implementations, combined with Figure 8 As shown, in step S4, the mask segmentation module consists of two layers of multilayer perceptrons and a sigmoid activation function; the output expression of the foreground mask probability map is: ; in, Features of a bird's-eye view ( It is a two-layer, multi-layer perceptron. Let S be the Sigmoid activation function, and S be the foreground mask probability map. The expression for the focus loss function is: ; in, The label probability corresponds to the true foreground region. The actual background area corresponds to α1 and γ1 are hyperparameters used to balance the weights of positive and negative samples and reduce the weights of simple background samples, respectively.

[0044] Preferably, in step S4, to address the issue of background noise interference in the point cloud, this embodiment constructs a mask segmentation (MS) module to assist training. This module utilizes a foreground mask supervision network to distinguish foreground objects from the background, and is only effective during the training phase, being pruned during the inference phase to reduce computational load. Specifically, the calculation process of the mask segmentation module is as follows: The bird's-eye view features output by the 3D feature backbone network Input to mask network In order to accurately extract point features, the masking network... It consists of a two-layer multilayer perceptron (MLP) network. Features are processed... After processing, it is activated by the Sigmoid function. Generate foreground mask prediction map : ; During the training phase, a focal loss function is used to address the problem of extreme imbalance in the number of foreground and background pixels. Supervise the foreground mask: .

[0045] The MS module is pruned (deleted) directly during the inference phase without increasing the computational load.

[0046] By incorporating the total loss, the weights of the 3D sparse convolutional backbone network are updated in reverse, making the features extracted by S1 more focused on the foreground and suppressing noise.

[0047] In some feasible implementations, combined with Figure 3 As shown, in step S5, L is a positive integer greater than or equal to 2, and the L refinement subnetworks are cascaded and connected in series. The first refinement subnetwork takes the initial 3D candidate box as input, and each subsequent refinement subnetwork takes the optimized candidate box output by the previous refinement subnetwork as input, and corrects the candidate box accuracy step by step.

[0048] In some feasible implementations, step S6 includes the objective of dividing the interior of each 3D candidate box into a uniform grid and extracting features from each grid point. Implementation methods include, but are not limited to: for each candidate box... Within the bounding box, uniformly sample G×G×G grid points (e.g., 6×6×6), interpolate from 3D / BEV features to obtain the feature of each grid point, and output: ,in, This represents the total number of grid points (e.g., 216). This represents the number of feature channels (e.g., 128 / 256).

[0049] In some feasible implementations, combined with Figure 4 As shown, the feature dynamic propagation module in step S7 employs a multi-head self-attention mechanism to map grid point features into query, key, and value features, capturing long-distance dependencies between grid points; the output feature calculation expression of the i-th attention head is: ; The enhanced features are obtained by concatenating the output features of all attention heads, processing them through residual connections and a multilayer perceptron, and the expression is as follows: ; In the formula, grid point features , For the number of grid points, For feature channel dimension, For the query features of the i-th attention head, For the key features of the i-th attention head, The value feature of the i-th attention head, For the first The output features of each attention head For the enhanced features, It is a multilayer perceptron. Indicates the number of attention heads, Concat( ) represents the splicing operation.

[0050] Specifically, the purpose of step S7 is to capture long-distance dependencies between grid points and recover fine-grained features lost due to occlusion or sparsity.

[0051] In some feasible implementations, combined with Figure 5 As shown, in order to make full use of the information of the preceding stages in the cascade structure during the refinement process, a cascaded perceptual attention mechanism (CPA) is introduced, which includes an external attention mechanism (EA) and an internal attention mechanism (IA).

[0052] In some feasible implementations, combined with Figure 6 As shown, the external attention mechanism in step S8 includes a key memory unit. AND-value memory unit The key and value feature information from the preceding stages are stored respectively. Within the current i-th level refinement subnetwork, an attention weight map is calculated for a single attention head, and the corresponding context features are retrieved using the memory unit. The calculation formula includes: ; ; in, This is the attention weight map corresponding to the j-th attention head within the i-th level refined subnetwork; This refers to the single-head contextual features obtained by relying on memory units for retrieval; and They represent the first Each attention head's memory unit key and value; This indicates a compound normalization operation; For linear transformation layer; For location embedding; For the front Each refined subnetwork outputs enhanced features; The single-head context features obtained from solving all attention heads within the same level of the refined subnetwork are concatenated, and feature mapping and integration are completed through a unified weight matrix to obtain the complete external attention output features of this level, expressed as: ; in, Indicates the number of external attention heads. Indicates the external projection layer. This is the integrated external attention output feature of the final output of the i-th level refined subnetwork.

[0053] Specifically, External Attention (EA) aggregates the historical stage context, aiming to utilize all the information learned from previous refinement layers without losing the historical context.

[0054] In some feasible implementations, combined with Figure 7 As shown, the internal attention mechanism in step S8 includes: The enhanced features output by the current i-th level refined subnetwork Through three independent linear transformation layers, each mapped to a query feature. Key features Sum value characteristics ; Based on the query features Key features Sum value characteristics pass The function computes the output features of the j-th internal attention head: ; In the formula, The output feature of the j-th internal attention head within the i-th level refined subnetwork; The outputs of all internal attention heads are concatenated and projected to obtain the internal attention features: ; In the formula, The number of internal attention heads. The internal attention output features generated by the i-th level refined subnetwork. This is the internal projection layer.

[0055] Specifically, the internal attention mechanism is used to optimize the feature representation of the current stage. The self-attention mechanism is used to refine the features of the current stage. The aim is to further refine the current features output by the FDP, reducing noise and enhancing relevance.

[0056] In some feasible implementations, the weighted fusion expression in step S10 is: ; ; In the formula, L represents the number of refined subnetworks. The 3D bounding box output by the i-th refined subnetwork. Let be the confidence level of the output of the i-th refined subnetwork. To determine the final confidence level, This is the final 3D bounding box.

[0057] Specifically, to obtain more robust results, this invention employs a box pooling strategy to weightedly fuse the bounding boxes output by multiple refined sub-networks. Let there be a total of... The refined subnetwork, the first The bounding box of the output of each refined subnetwork is The confidence score is The final merged bounding box is obtained. and final confidence score This strategy can smooth out prediction jitter in a single stage, use high-confidence stage results to correct low-confidence results, and improve the overall detection stability.

[0058] In summary, the 3D target detection method based on progressive fine-grained refinement networks proposed in this invention effectively addresses practical pain points in existing technologies, such as strong background interference, loss of fine-grained features, fragmentation of multi-level context, and unstable prediction, through five core designs: mask guidance, progressive multi-level refinement, dynamic feature enhancement, cascaded perception dual attention, and multi-level bounding box fusion. It balances detection accuracy with operational efficiency, making it well-suited for various complex real-world scenarios such as autonomous driving and robot perception.

[0059] Example 2

[0060] Please see Figure 9 This embodiment provides a structural diagram of a three-dimensional target detection system based on a progressive fine-grained refinement network.

[0061] As an example, the system is implemented using the three-dimensional target detection method based on progressive fine-grained network described in Example 1, and the system includes: The initial 3D candidate box construction unit 910 is suitable for acquiring 3D point cloud data of the scene to be detected, performing voxelization processing on the 3D point cloud data, extracting voxel features through a 3D sparse convolutional backbone network, compressing the voxel features along the height direction to obtain bird's-eye view features, and generating initial 3D candidate boxes based on the bird's-eye view features using a region generation network. The mask segmentation module construction unit 920 is suitable for the mask segmentation module to take the bird's-eye view features as input and output a foreground mask probability map; during the training phase, the foreground mask probability map is compared with the real foreground mask, and the mask segmentation loss is calculated using the focus loss function; the mask segmentation loss is backpropagated to the three-dimensional sparse convolutional backbone network to guide the backbone network to focus on the foreground region and suppress background noise; during the inference phase, the mask segmentation module is pruned and removed, and does not participate in detection inference; The current stage's 3D bounding box and confidence prediction unit 930 is suitable for constructing a progressive fine-grained refinement network containing L cascaded refinement sub-networks. Each refinement sub-network includes a RoI-grid Pooling unit, a feature dynamic propagation module, a cascaded perceptual attention mechanism, and a detection head. In each refinement sub-network, based on the initial 3D candidate box, grid points within the candidate box are sampled and grid point features are extracted using the RoI-grid Pooling unit. The feature dynamic propagation module is used to enhance the grid point features and output enhanced features. The cascaded perceptual attention mechanism includes an external attention mechanism and an internal attention mechanism. The external attention mechanism generates external attention output features, and the internal attention mechanism generates internal attention output features. The external attention output features, internal attention output features, and current stage enhanced features are fused and input into the detection head to predict the current stage's 3D bounding box and confidence. The 3D object detection result output unit 940 is suitable for using a box pooling strategy to perform weighted fusion of all 3D bounding boxes and their corresponding confidence scores output by L refined sub-networks to obtain the final 3D object detection result.

[0062] It is not difficult to see that this embodiment is a system implementation corresponding to the first embodiment, and this embodiment can be implemented in conjunction with the first embodiment. The relevant technical details mentioned in the first embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the first embodiment.

[0063] It is worth mentioning that all modules involved in this embodiment are logical units. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by this invention; however, this does not mean that other units are absent from this embodiment.

[0064] Example 3

[0065] Please see Figure 10 The present invention also provides an electronic device, including: a memory and a processor; the memory stores at least one program instruction; the processor loads and executes the at least one program instruction to implement the three-dimensional target detection method based on progressive fine-grained network provided in Embodiment 1.

[0066] The memory 702 and processor 701 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 701 and memory 702 together. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 701 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 701.

[0067] Processor 701 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 702 can be used to store data used by processor 701 during operation.

[0068] Example 4

[0069] This invention also proposes a storage medium storing a 3D target detection method based on a progressive fine-grained network. When the 3D target detection program based on the progressive fine-grained network is executed by a processor, it implements the steps of the 3D target detection method based on the progressive fine-grained network described above. Since this storage medium adopts all the technical solutions of all the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated further here.

[0070] The above descriptions are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, based on the guidance provided in this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A three-dimensional target detection method based on progressive fine-grained network refinement, characterized in that, The method includes: Step S1: Obtain the 3D point cloud data of the scene to be detected, perform voxelization processing on the 3D point cloud data, and extract voxel features through a 3D sparse convolutional backbone network. Step S2: Compress the voxel features along the height direction to obtain the bird's-eye view features; Step S3: Using a region generation network, generate initial 3D candidate boxes based on the bird's-eye view features; Step S4: Construct a mask segmentation module. This module takes the bird's-eye view features as input and outputs a foreground mask probability map. During the training phase, the foreground mask probability map is compared with the real foreground mask, and the mask segmentation loss is calculated using a focus loss function. The mask segmentation loss is then backpropagated to the 3D sparse convolutional backbone network to guide the backbone network to focus on the foreground region and suppress background noise. During the inference phase, the mask segmentation module is pruned and removed, and it does not participate in detection inference. Step S5: Construct a progressive fine-grained refinement network containing L cascaded refinement subnetworks; each refinement subnetwork contains a RoI-grid Pooling unit, a feature dynamic propagation module, a cascaded perceptual attention mechanism, and a detection head; Step S6: In each refined sub-network, based on the initial 3D candidate box, the grid points within the candidate box are sampled through the RoI-grid Pooling unit and the grid point features are extracted; Step S7: Enhance the features of the grid points using the feature dynamic propagation module and output the enhanced features; Step S8: The cascaded perception attention mechanism includes an external attention mechanism and an internal attention mechanism; the external attention mechanism generates external attention output features, and the internal attention mechanism generates internal attention output features. Step S9: Fuse external attention output features, internal attention output features, and current stage enhancement features, and input them into the detection head to predict the 3D bounding box and confidence level of the current stage; Step S10: Using a box pooling strategy, weighted fusion is performed on all 3D bounding boxes and their corresponding confidence scores output by the L refined sub-networks to obtain the final 3D target detection result.

2. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, The method is executed by a pre-installed detection model, which includes a 3D sparse convolutional backbone network, a region generation network, a mask segmentation module, a progressive fine-grained refinement network, and a detection head. The detection model employs an end-to-end joint training approach for parameter optimization, and the total loss function used during training is defined as: ； ； in, Generate network loss for the region; For mask segmentation loss; This is the balance coefficient; Total loss; To refine network loss; Indicates the first In the refined subnetwork, the first The prediction confidence score of each candidate box; Indicates the first In the refined subnetwork, the first The true confidence scores corresponding to each candidate box; Indicates the first In the refined subnetwork, the first The regression residuals of the predicted bounding boxes of each candidate box; Indicates the first In the refined subnetwork, the first The regression residuals of the true bounding boxes corresponding to each candidate box; For confidence loss; For regression loss; For indicator functions, only when the first The first subnetwork Each candidate box and the true value Greater than the threshold Calculate the regression loss at that time.

3. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, The three-dimensional point cloud data in step S1 includes spatial coordinate information and reflection intensity information. The voxelization process divides the disordered three-dimensional point cloud into a regular three-dimensional grid according to a preset size. The three-dimensional sparse convolutional backbone network only performs convolution operations on non-empty voxels containing point clouds to extract voxel features.

4. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, In step S4, the mask segmentation module consists of two layers of multilayer perceptron and a sigmoid activation function; the output expression of the foreground mask probability map is: ； in, Features of a bird's-eye view ( It is a two-layer, multi-layer perceptron. Let S be the Sigmoid activation function, and S be the foreground mask probability map. The expression for the focus loss function is: ； in, The label probability corresponds to the true foreground region. The actual background area corresponds to α1 and γ1 are hyperparameters used to balance the weights of positive and negative samples and reduce the weights of simple background samples, respectively.

5. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, In step S5, L is a positive integer greater than or equal to 2, and the L thinning subnetworks are cascaded and connected in series. The first thinning subnetwork takes the initial 3D candidate box as input, and each subsequent thinning subnetwork takes the optimized candidate box output by the previous thinning subnetwork as input to correct the candidate box accuracy step by step.

6. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, The feature dynamic propagation module in step S7 employs a multi-head self-attention mechanism to map grid point features to query, key, and value features, capturing long-distance dependencies between grid points; the output feature calculation expression of the i-th attention head is: ； The enhanced features are obtained by concatenating the output features of all attention heads, processing them through residual connections and a multilayer perceptron, and the expression is as follows: ； In the formula, grid point features , For the number of grid points, For feature channel dimension, For the query features of the i-th attention head, For the key features of the i-th attention head, The value feature of the i-th attention head, For the first The output features of each attention head For the enhanced features, It is a multilayer perceptron. Indicates the number of attention heads, Concat( ) represents the splicing operation.

7. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, The external attention mechanism in step S8 includes a key memory unit. AND-value memory unit They respectively store the key feature information and value feature information of the preceding stages; Within the current i-th level refined subnetwork, an attention weight map is calculated for a single attention head, and the corresponding contextual features are retrieved using memory units. The calculation formula is as follows: include: ； ； in, This is the attention weight map corresponding to the j-th attention head within the i-th level refined subnetwork; This refers to the single-head contextual features obtained by relying on memory units for retrieval; and They represent the first Each attention head's memory unit key and value; This indicates a compound normalization operation; For linear transformation layer; For location embedding; For the front Each refined subnetwork outputs enhanced features; The single-head context features obtained from solving all attention heads within the same level of the refined subnetwork are concatenated, and feature mapping and integration are completed through a unified weight matrix to obtain the complete external attention output features of this level, expressed as: ； in, Indicates the number of external attention heads. Indicates the external projection layer. This is the integrated external attention output feature of the final output of the i-th level refined subnetwork.

8. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, The internal attention mechanism in step S8 includes: The enhanced features output by the current i-th level refined subnetwork Through three independent linear transformation layers, each mapped to a query feature. Key features Sum value characteristics ; Based on the query features Key features Sum value characteristics pass The function computes the output features of the j-th internal attention head: ； In the formula, The output feature of the j-th internal attention head within the i-th level refined subnetwork; The outputs of all internal attention heads are concatenated and projected to obtain the internal attention features: ； In the formula, The number of internal attention heads. The internal attention output features generated by the i-th level refined subnetwork. This is the internal projection layer.

9. The three-dimensional target detection method based on progressive fine-grained network refinement according to claim 1, characterized in that, The weighted fusion expression in step S10 is: ； ； In the formula, L represents the number of refined subnetworks. The 3D bounding box output by the i-th refined subnetwork. Let be the confidence level of the output of the i-th refined subnetwork. To determine the final confidence level, This is the final 3D bounding box.

10. A three-dimensional target detection system based on a progressive fine-grained refinement network, implemented using the three-dimensional target detection method based on a progressive fine-grained refinement network as described in any one of claims 1-9, characterized in that, The system includes: The initial 3D candidate box construction unit is suitable for acquiring 3D point cloud data of the scene to be detected, performing voxelization on the 3D point cloud data, extracting voxel features through a 3D sparse convolutional backbone network, compressing the voxel features along the height direction to obtain bird's-eye view features, and generating initial 3D candidate boxes based on the bird's-eye view features using a region generation network. The mask segmentation module construction unit is suitable for the mask segmentation module to take the bird's-eye view features as input and output a foreground mask probability map; during the training phase, the foreground mask probability map is compared with the real foreground mask, and the mask segmentation loss is calculated using the focus loss function; the mask segmentation loss is backpropagated to the three-dimensional sparse convolutional backbone network to guide the backbone network to focus on the foreground region and suppress background noise; during the inference phase, the mask segmentation module is pruned and removed, and does not participate in detection inference; The current stage's 3D bounding box and confidence prediction unit is suitable for constructing a progressive fine-grained refinement network containing L cascaded refinement sub-networks. Each refinement sub-network includes a RoI-grid Pooling unit, a feature dynamic propagation module, a cascaded perceptual attention mechanism, and a detection head. In each refinement sub-network, based on the initial 3D candidate box, grid points within the candidate box are sampled and grid point features are extracted using the RoI-grid Pooling unit. The feature dynamic propagation module enhances the grid point features and outputs enhanced features. The cascaded perceptual attention mechanism includes an external attention mechanism and an internal attention mechanism. The external attention mechanism generates external attention output features, and the internal attention mechanism generates internal attention output features. The external attention output features, internal attention output features, and current stage enhanced features are fused and input into the detection head to predict the current stage's 3D bounding box and confidence. The 3D object detection result output unit is suitable for using a box pooling strategy to perform weighted fusion of all 3D bounding boxes and their corresponding confidence scores output by L refined sub-networks to obtain the final 3D object detection result.

Images