Three-dimensional target detection method and device based on RoI conditional multi-head attention

Abstract

The invention discloses a three-dimensional target detection method and device based on RoI conditional multi-head attention, and relates to the technical field of target detection. The method comprises the following steps: acquiring a three-dimensional candidate frame and a corresponding backbone feature map; linearly mapping the seven-dimensional geometric parameters into an attention query vector; constructing a regular grid in the three-dimensional candidate frame, and sampling in the backbone feature map to obtain a grid feature matrix; respectively and linearly mapping the grid characteristic matrix into a key matrix and a value matrix; by taking the attention query vector as a query, executing multi-head attention aggregation on the key matrix and the value matrix to obtain an aggregation feature of the three-dimensional candidate box; and based on the aggregation feature, outputting the category and position regression result of the three-dimensional candidate box through a shared full connection layer. According to the method, the geometric identity of the RoI is inquired, geometric conditional adaptive weighting is carried out among internal grid points through multi-head attention, and compared with a traditional flattening-MLP mode, the detection precision is greatly improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a method and apparatus for three-dimensional target detection based on RoI conditional multi-head attention. Background Technology

[0002] With the rapid development of autonomous driving, mobile robots, and intelligent transportation, 3D target detection technology based on LiDAR, millimeter-wave radar, or vision-LiDAR fusion has become a core component of environmental perception. Current mainstream 3D detection frameworks generally employ the following approach: Backbone / Neck (e.g., sparse convolution or BEV) extracts voxel / BEV features; RPN generates 3D candidate boxes (RoIs); RoI grid pooling samples features (G×C) from regular grid points within the candidate boxes; the grid features are flattened and passed through a shared fully connected layer (Shared FC) to obtain RoI-level features, which are then connected to a classification / regression branch.

[0003] In the currently disclosed technical solutions, RoI feature aggregation usually adopts the implementation method of "regular grid sampling + flattening + shared multilayer perceptron (Shared MLP)": a regular three-dimensional grid is arranged inside the RoI, a feature matrix is ​​sampled from the backbone feature map, the matrix is ​​flattened into a vector, and mapped to a fixed dimension through 1 to 2 layers of shared MLP as the final representation of the RoI.

[0004] The above scheme has a simple structure and is easy to embed into mainstream 3D detectors, but it has revealed the following common problems in practice: 1. Lack of RoI geometric conditional guidance. The flattened-MLP approach does not explicitly utilize the geometric priors of RoI itself. Candidate boxes of different shapes, orientations, or scales are treated as vectors of the same length without distinction, resulting in the network being insensitive to geometric changes and making it difficult to adaptively assign weights to grid point features.

[0005] 2. Insufficient long-range dependencies and interaction capabilities. Simple flattening operations disrupt the spatial adjacency relationships of grid points, making it impossible to dynamically capture long-range dependencies across grid points, thus limiting the ability to express fine geometric features such as boundaries and corners.

[0006] 3. Poor robustness and interpretability. Since the weights rely entirely on data-driven learning, it is difficult to generate a visual attention / weight graph from within the module for diagnosis or constraint purposes.

[0007] Chinese patent applications with publication numbers CN202210894983.2, CN202411836575.7, and CN202310463029.2 all attempt to introduce self-attention into the backbone network, but their attention mechanisms only act on global voxels or BEV features and do not perform conditional design for geometric parameters at the key granularity of RoI-internal mesh. Summary of the Invention

[0008] In view of the shortcomings of existing technologies, this invention provides a 3D target detection method and apparatus based on RoI conditional multi-head attention. This invention queries the geometric identity of the RoI itself and performs geometrically conditional adaptive weighting among internal grid points through multi-head attention, significantly improving detection accuracy compared to the traditional flattening-MLP method.

[0009] The technical means employed in this invention are as follows: A 3D object detection method based on RoI conditional multi-head attention includes the following steps: Obtain a three-dimensional candidate box and its corresponding backbone feature map, wherein the three-dimensional candidate box has seven-dimensional geometric parameters; The seven-dimensional geometric parameters are linearly mapped to an attention query vector; A regular grid is constructed within the three-dimensional candidate box, and samples are taken from the backbone feature map to obtain the grid feature matrix; The grid feature matrix is ​​linearly mapped to a key matrix and a value matrix, respectively. Using the attention query vector as the query, multi-head attention aggregation is performed on the key matrix and the value matrix to obtain the aggregated features of the 3D candidate box; Based on the aggregated features, the category and location regression results of the 3D candidate boxes are output through a shared fully connected layer.

[0010] Further, obtain the 3D candidate bounding boxes and their corresponding backbone feature maps, including: The process involves acquiring 3D point cloud data of the scene to be detected, performing preprocessing on the 3D point cloud data, inputting the preprocessed data into the backbone network of a pre-trained 3D target detection model, extracting multi-scale backbone feature maps using the backbone network, and generating several 3D candidate boxes based on the backbone feature maps through a region proposal network, thereby obtaining a backbone feature map region corresponding to each 3D candidate box.

[0011] Furthermore, the geometric parameter vector of the three-dimensional candidate box is:

[0012] in, This represents the center coordinates of the candidate bounding box along the x-axis in a 3D coordinate system. This represents the center coordinates of the candidate bounding box along the y-axis in a 3D coordinate system. This represents the center coordinates of the candidate bounding box along the z-axis in the 3D coordinate system. This indicates the length of the candidate box along the x-axis. This indicates the length of the candidate box along the y-axis. This represents the height of the candidate bounding box along the z-axis. This represents the rotation angle of the candidate box relative to the z-axis.

[0013] Furthermore, the seven-dimensional geometric parameters are linearly mapped to the attention query vector according to the following formula:

[0014] in, Let represent the weight matrix used to map the geometric parameter vector p to the attention query space. The representation is related to the weight matrix. The corresponding bias vector, d, represents the dimension of the feature space where the attention query vector q resides.

[0015] Furthermore, a side length of [missing information] is constructed within the three-dimensional candidate box. The regular grid is obtained A grid sampling point; and sampling is performed in the backbone feature map to obtain the following grid feature matrix:

[0016] in This represents the number of backbone feature channels.

[0017] Furthermore, the mesh feature matrix is ​​linearly mapped to the key matrix according to the following formula:

[0018] in, Let represent the weight matrix used to map each grid feature vector from the channel dimension C to the attention key space dimension d. This represents a column vector of size G×1 with all elements having a value of 1, used to copy the bias term in the row direction. The representation is related to the weight matrix. The transpose of the corresponding bias vector is used to add the same bias to each row of the key matrix; The grid feature matrix is ​​linearly mapped to a value matrix according to the following formula:

[0019] in, Let represent the weight matrix used to map each grid feature vector from the channel dimension C to the attention value space dimension d. The representation is related to the weight matrix. The transpose of the corresponding bias vector is used to add the same bias to each row of the value matrix.

[0020] Further, using the attention query vector as the query, multi-head attention aggregation is performed on the key matrix and the value matrix, including: query vector Expanded into a single query matrix Obtain the key matrix Sum matrix ; Will They got separately ,in, Let represent the query matrix corresponding to the i-th attention head. Let represent the key matrix corresponding to the i-th attention head. Let represent the value matrix corresponding to the i-th attention head; calculate

[0021] in, Let represent the attention weight matrix corresponding to the i-th attention head, used to characterize the weight distribution of a single query across G grid points. This represents the feature dimension of a single attention head. This indicates that the i-th attention head is in the value matrix. according to The output matrix obtained after weighted summation; The outputs from each header are concatenated and then subjected to linear transformation and residual normalization: .

[0022] in, The expression represents the RoI-level output feature vector obtained after multi-head attention aggregation, linear transformation, and residual and normalization processing. Let represent the weight matrix used to map the concatenated multi-head outputs back to the attention feature dimension d. The representation is related to the weight matrix. The corresponding bias vector, This represents a short residual term, which is directly introduced from the input features of the attention module after dimensional alignment and added element-wise to the multi-head output. It is used to preserve input information and improve the stability of the training process. This is the layer normalization function.

[0023] Furthermore, the method also includes: after obtaining the grid feature matrix, flattening the grid feature matrix, performing two-layer MLP aggregation processing, and outputting the category and location regression results of the three-dimensional candidate boxes.

[0024] This invention also discloses a 3D target detection device based on RoI conditional multi-head attention, used to implement the above method, comprising: The candidate box acquisition module is used to acquire three-dimensional candidate boxes and their corresponding backbone feature maps. The three-dimensional candidate boxes have seven-dimensional geometric parameters. The first mapping module is used to linearly map the seven-dimensional geometric parameters into an attention query vector; A mesh construction module is used to construct a regular mesh within the 3D candidate box and sample it in the backbone feature map to obtain a mesh feature matrix. The second mapping module is used to linearly map the grid feature matrix into a key matrix and a value matrix, respectively. A multi-head attention aggregation module is used to perform multi-head attention aggregation on the key matrix and the value matrix with the attention query vector as the query, so as to obtain the aggregated features of the three-dimensional candidate box; The output module is used to output the category and location regression results of the 3D candidate boxes through a shared fully connected layer based on the aggregated features.

[0025] Compared with the prior art, the present invention has the following advantages: 1. This invention explicitly utilizes the 7D geometric prior of RoI to generate queries, guides the adaptive weighting of mesh features, enhances the adaptability to changes in shape, scale, and orientation, and realizes RoI conditionalization.

[0026] 2. This invention achieves global interaction through multi-head attention within the grid, outperforming the expressive power of local convolution or simple MLPs. This invention performs multi-head self-attention calculation on the features of all G grid points within each RoI, ensuring that any two grid points interact through an attention weight matrix. Output the weighted sum Generating explicit weighted interaction terms is equivalent to constructing an adaptive kernel function in the feature space that depends on global content and geometric location. Because the attention weights adaptively change with the input features and the multi-head structure divides the feature dimension d into multiple subspaces to model different relational substructures, compared to relying solely on a fixed convolutional kernel's local receptive field or... By performing linear transformations on a single MLP, this invention can theoretically characterize higher-order and more complex inter-point correlations and long-range dependencies, thereby significantly improving the expressive power of feature aggregation within RoIs.

[0027] 3. The residual and normalization design of this invention improves training stability, and the module provides a switch, resulting in low risk of engineering implementation.

[0028] 4. This invention can seamlessly collaborate with existing 3D detection backbones, RPNs, and loss functions, facilitating rapid migration and reuse in scenarios such as autonomous driving and robotics, and has strong versatility.

[0029] In summary, the technical solution of this invention enables RoIs to actively perceive and utilize their own geometric information such as shape, orientation, and scale when aggregating internal mesh features, achieving more accurate, interpretable, and easily deployable 3D target detection. This invention can be widely applied in fields such as autonomous driving and assisted driving (BEV / surround view fusion), mobile robot environmental perception, intelligent transportation roadside perception, security and logistics park monitoring, indoor and outdoor digital twins and surveying. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating the execution of a 3D target detection method based on RoI conditional multi-head attention in an embodiment of the present invention.

[0032] Figure 2 This is the conditional multi-head attention aggregation module architecture in this embodiment of the invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] like Figure 1 As shown, this invention provides a 3D target detection method based on RoI conditional multi-head attention, comprising the following steps: S1. Obtain a three-dimensional candidate box and its corresponding backbone feature map. The three-dimensional candidate box has seven-dimensional geometric parameters.

[0036] In this application, the method for obtaining 3D candidate boxes and corresponding backbone feature maps includes: acquiring 3D point cloud data of the scene to be detected, performing preprocessing operations such as voxelization or feature encoding on the data, inputting the preprocessed data into the backbone network of a pre-trained 3D target detection model, extracting multi-scale backbone feature maps using the backbone network, and generating several 3D candidate boxes based on the backbone feature maps through a region proposal network, thereby obtaining a backbone feature map region corresponding to each 3D candidate box.

[0037] In this application, the region proposal network can adopt the anchor-box proposal structure in the prior art, that is, several convolutional layers are set sequentially on the backbone feature map to extract RPN features, and then the target existence probability and coordinate offset of each preset 3D anchor box are output through classification branch and regression branch respectively. In addition, the specific structure of the region proposal network can also refer to the RPN module disclosed in the existing 3D target detection model, such as the region proposal network of the Faster R-CNN series network and extended to 3D point cloud scene. This application does not limit the specific number of layers and parameter configuration of the region proposal network.

[0038] The input data is then processed by a neck network to obtain intermediate feature maps for sampling. The RPN generates RoIs and provides the geometric parameters for each RoI. The 7-dimensional geometric parameters of each RoI are shown below:

[0039] in, This represents the center coordinates of the candidate bounding box along the x-axis in a 3D coordinate system. This represents the center coordinates of the candidate bounding box along the y-axis in a 3D coordinate system. This represents the center coordinates of the candidate bounding box along the z-axis in the 3D coordinate system. This indicates the length of the candidate box along the x-axis. This indicates the length of the candidate box along the y-axis. This represents the height of the candidate bounding box along the z-axis. This represents the rotation angle of the candidate box relative to the z-axis.

[0040] S2. Linearly map the seven-dimensional geometric parameters into an attention query vector.

[0041] The 7-dimensional geometric parameters of each RoI are linearly mapped to the attention query space to obtain the query vector.

[0042] in For attention channel dimension (also used as (Default settings). Attention dimension: Default Specifically, The feature dimension of the query vector q is set to be the same as the dimension of the attention feature space in this embodiment. = ; This represents the feature dimension of the key vector, used to constrain the feature length of each row in the key matrix K; This represents the feature dimension of the value vector, used to constrain the feature length of each row in the value matrix V; in a typical embodiment, it is taken as... = And the value of d is a positive integer within the range of [128, 512].

[0043] S3. Construct a regular mesh within the 3D candidate bounding box and sample it in the backbone feature map to obtain the mesh feature matrix. Arrange a 3D regular mesh within each RoI, with a mesh side length of [missing information]. Grid point count In this application, .

[0044] A grid feature matrix is ​​obtained by sampling from the backbone feature map. Specifically, the coordinates of each grid sampling point in three-dimensional space are mapped to continuous coordinate positions in the feature coordinate system of the backbone feature map, and the corresponding feature vectors are obtained from the backbone feature map using trilinear interpolation or nearest neighbor interpolation. The feature vectors are stacked according to a preset order of grid points to obtain the following grid feature matrix:

[0045] in This represents the number of channels.

[0046] S4. Linearly map the grid feature matrix into a key matrix and a value matrix, respectively.

[0047] right Perform a linear mapping:

[0048] S5. Using the attention query vector as the query, perform multi-head attention aggregation on the key matrix and the value matrix to obtain the aggregated features of the three-dimensional candidate box.

[0049] query vector Expanded into a single query matrix The specific extension method is to add a one-dimensional row dimension to the original one-dimensional feature vector, so that the query matrix can be aligned with the key matrix and value matrix during matrix multiplication, and the same query vector can be shared between different attention heads.

[0050] by For key-value pairs (K / V), in the form of equation (1) For the query, perform multi-head scaled dot product attention and output RoI-level compressed features. .

[0051] S6. Based on the aggregated features, output the category and location regression results of the 3D candidate boxes through a shared fully connected layer.

[0052] Let the number of heads be Single-head dimension In this application, .Will They got separately ,in, Let represent the query matrix corresponding to the i-th attention head. Let represent the key matrix corresponding to the i-th attention head. Let represent the value matrix corresponding to the i-th attention head.

[0053] calculate

[0054] in, Let represent the attention weight matrix corresponding to the i-th attention head, used to characterize the weight distribution of a single query across G grid points. The feature dimension of a single attention head is represented by [symbol]. This indicates that the i-th attention head is in the value matrix. according to The output matrix obtained after weighted summation.

[0055] The outputs from each header are concatenated and then subjected to linear transformation and (optionally) residual normalization:

[0056] The expression represents the RoI-level output feature vector obtained after multi-head attention aggregation, linear transformation, and residual and normalization processing. Let represent the weight matrix used to map the concatenated multi-head outputs back to the attention feature dimension d. The representation is related to the weight matrix. The corresponding bias vector, This represents a short residual term, which is directly introduced from the input features of the attention module after dimensional alignment and added element-wise to the multi-head output. It is used to preserve input information and improve the stability of the training process. This is the layer normalization function.

[0057] After aggregation (Or the flattened features of the fallback path) undergo a nonlinear transformation, specifically: the feature vector is sequentially input into at least one fully connected network with an activation function, using a multilayer perceptron structure of "fully connected layer + ReLU activation + Dropout" to obtain the RoI-level feature representation. .based on The output class probability and bounding box regression value are calculated. On one hand, the input r is a fully connected layer for classification. The score vectors for each category are then normalized using either the Softmax or Sigmoid function to obtain the category probability corresponding to each candidate box. On the other hand, r is input into the fully connected regression layer. The regression parameter vector is obtained and used to predict the 3D position offset, size residual, and rotation angle residual of the candidate box, thereby completing the classification and regression output of the 3D candidate box. In this application, the Shared FC and output contain two implementation paths: the attention path and the backoff path.

[0058] Execute within the attention path: .

[0059] The input is equation (4). A two-layer MLP is recommended:

[0060] in For intermediate dimensions (e.g., 256–512). The RoI dimension is represented (typically aligned with the classification / regression head, e.g., 256). This path corresponds to `shared_fc_layer_attn` in the code; its input dimension is the same as the attention output dimension. Consistency, avoiding Large parameter matrices at the level of [specific parameters] result in more stable training. Execute on the fallback path:

[0061] in This is a combination of nonlinear transformations (such as ReLU) and (optionally) Dropout.

[0062] The input is a flattened mesh feature. A two-layer MLP is recommended:

[0063] This path corresponds to `shared_fc_layer` in the code (the original baseline approach). When or When the value is large, the number of path parameters and memory usage increase significantly. Therefore, this invention prioritizes using equation (5) when attention is enabled.

[0064] During network training, classification and regression losses (cross-entropy / focus loss, Smooth L1 / IoU Loss, etc.) are jointly optimized, and during inference, the confidence threshold and NMS are combined to obtain the final result.

[0065] During network training, classification and regression losses are jointly optimized. During inference, the confidence threshold and NMS are combined to obtain the final result. Specifically, cross-entropy loss or focus loss is used for the classification output of each RoI, and SmoothL1 loss and / or IoU loss is used for the regression output of positive RoIs.

[0066] Let the category prediction score vector of the nth RoI be... The corresponding real category is If there are K categories, then the Softmax probability is:

[0067] Its cross-entropy classification loss is

[0068] Where N is the number of RoIs participating in training, II This is an indicator function.

[0069] In the embodiment employing focus loss, equation (7) is replaced with the focus loss form:

[0070] in, α represents the predicted probability corresponding to the true category of the nth RoI, α is the balance factor, and γ is the focusing parameter. In this application, γ is preset according to the specific dataset.

[0071] Let the set of positive sample RoIs be , and for each positive sample RoI, define a regression target vector and a predicted regression vector, using element-wise SmoothL1 loss:

[0072] The regression loss is

[0073] Where β is the smoothing parameter and j is the regression component index.

[0074] In embodiments employing IoU loss, the loss can also be based on the 3D coordinates of the predicted bounding box and the ground truth bounding box. value Constructing IoU loss

[0075] The ultimate training objective is to obtain a weighted sum of classification loss and regression loss.

[0076] Or adopt in focus loss and IoU loss scenarios

[0077] in, and These are the weighting coefficients used to balance the various loss terms.

[0078] Furthermore, as a preferred embodiment of this application, the method of this embodiment further includes: S7 provides module start / stop and attention weight heatmap export functions for easy debugging and interpretation.

[0079] Specifically, according to A heatmap is used to observe key grid points within a RoI. In implementation, the first step is to... The system reconstructs the weight value of each grid point into a 3D or 2D weight distribution according to the arrangement order of the regular 3D grid within the RoI, and normalizes the weight values ​​to the [0,1] interval. Then, a heatmap is constructed based on the normalized weight distribution. For example, the weight values ​​are mapped to a pseudo-color image and overlaid with the corresponding RoI projection view or grid coordinates to generate an attention weight heatmap for visualizing key grid points within the RoI. The system controls whether to enable the attention calculation branch and whether to output the heatmap through preset switch parameters. When enabled, the corresponding heatmap data is exported or saved synchronously during the forward inference stage for network debugging and result interpretation.

[0080] In this application, the input / output tensor shapes are aligned with the regular RoI Head, allowing for direct replacement deployment.

[0081] This invention also discloses a 3D target detection device based on RoI conditional multi-head attention, used to implement the above-mentioned 3D target detection method based on RoI conditional multi-head attention, comprising: The candidate box acquisition module is used to acquire three-dimensional candidate boxes and their corresponding backbone feature maps. The three-dimensional candidate boxes have seven-dimensional geometric parameters. The first mapping module is used to linearly map the seven-dimensional geometric parameters into an attention query vector; A mesh construction module is used to construct a regular mesh within the 3D candidate box and sample it in the backbone feature map to obtain a mesh feature matrix. The second mapping module is used to linearly map the grid feature matrix into a key matrix and a value matrix, respectively. A multi-head attention aggregation module is used to perform multi-head attention aggregation on the key matrix and the value matrix with the attention query vector as the query, so as to obtain the aggregated features of the three-dimensional candidate box; The output module is used to output the category and location regression results of the 3D candidate boxes through a shared fully connected layer based on the aggregated features.

[0082] Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 3D target detection method based on RoI conditional multi-head attention, characterized in that, Includes the following steps: Obtain a three-dimensional candidate box and its corresponding backbone feature map, wherein the three-dimensional candidate box has seven-dimensional geometric parameters; The seven-dimensional geometric parameters are linearly mapped to an attention query vector; A regular grid is constructed within the three-dimensional candidate box, and samples are taken from the backbone feature map to obtain the grid feature matrix; The grid feature matrix is ​​linearly mapped to a key matrix and a value matrix, respectively. Using the attention query vector as the query, multi-head attention aggregation is performed on the key matrix and the value matrix to obtain the aggregated features of the 3D candidate box; Based on the aggregated features, the category and location regression results of the 3D candidate boxes are output through a shared fully connected layer.

2. The three-dimensional target detection method based on RoI conditional multi-head attention according to claim 1, characterized in that, Obtain the 3D candidate bounding boxes and their corresponding backbone feature maps, including: The process involves acquiring 3D point cloud data of the scene to be detected, performing preprocessing on the 3D point cloud data, inputting the preprocessed data into the backbone network of a pre-trained 3D target detection model, extracting multi-scale backbone feature maps using the backbone network, and generating several 3D candidate boxes based on the backbone feature maps through a region proposal network, thereby obtaining a backbone feature map region corresponding to each 3D candidate box.

3. The three-dimensional target detection method based on RoI conditional multi-head attention according to claim 1, characterized in that, The geometric parameter vector of the three-dimensional candidate box is: in, This represents the center coordinates of the candidate bounding box along the x-axis in a 3D coordinate system. This represents the center coordinates of the candidate bounding box along the y-axis in a 3D coordinate system. This represents the center coordinates of the candidate bounding box along the z-axis in the 3D coordinate system. This indicates the length of the candidate box along the x-axis. This indicates the length of the candidate box along the y-axis. This represents the height of the candidate bounding box along the z-axis. This represents the rotation angle of the candidate box relative to the z-axis.

4. A three-dimensional target detection method based on RoI conditional multi-head attention according to claim 3, characterized in that, The seven-dimensional geometric parameters are linearly mapped to the attention query vector according to the following formula: in, Let represent the weight matrix used to map the geometric parameter vector p to the attention query space. The representation is related to the weight matrix. The corresponding bias vector, d, represents the dimension of the feature space where the attention query vector q resides.

5. A three-dimensional target detection method based on RoI conditional multi-head attention according to claim 1, characterized in that, Construct a side with a length of within the three-dimensional candidate box. The regular grid is obtained A grid sampling point; and sampling is performed in the backbone feature map to obtain the following grid feature matrix: in This represents the number of backbone feature channels.

6. A three-dimensional target detection method based on RoI conditional multi-head attention according to claim 1, characterized in that, The grid feature matrix is ​​linearly mapped to the key matrix according to the following formula: in, Let represent the weight matrix used to map each grid feature vector from the channel dimension C to the attention key space dimension d. This represents a column vector of size G×1 with all elements having a value of 1, used to copy the bias term in the row direction. The representation is related to the weight matrix. The transpose of the corresponding bias vector is used to add the same bias to each row of the key matrix; The grid feature matrix is ​​linearly mapped to a value matrix according to the following formula: in, Let represent the weight matrix used to map each grid feature vector from the channel dimension C to the attention value space dimension d. The representation is related to the weight matrix. The transpose of the corresponding bias vector is used to add the same bias to each row of the value matrix.

7. A three-dimensional target detection method based on RoI conditional multi-head attention according to claim 1, characterized in that, Using the attention query vector as the query, perform multi-head attention aggregation on the key matrix and the value matrix, including: query vector Expanded into a single query matrix Obtain the key matrix Sum matrix ; Will They got separately ,in, Let represent the query matrix corresponding to the i-th attention head. Let represent the key matrix corresponding to the i-th attention head. Let represent the value matrix corresponding to the i-th attention head; calculate in, Let represent the attention weight matrix corresponding to the i-th attention head, used to characterize the weight distribution of a single query across G grid points. This represents the feature dimension of a single attention head. This indicates that the i-th attention head is in the value matrix. according to The output matrix obtained after weighted summation; The outputs from each header are concatenated and then subjected to linear transformation and residual normalization: in, The expression represents the RoI-level output feature vector obtained after multi-head attention aggregation, linear transformation, and residual and normalization processing. Let represent the weight matrix used to map the concatenated multi-head outputs back to the attention feature dimension d. The representation is related to the weight matrix. The corresponding bias vector, This represents a short residual term, which is directly introduced from the input features of the attention module after dimensional alignment and added element-wise to the multi-head output. It is used to preserve input information and improve the stability of the training process. This is the layer normalization function.

8. A three-dimensional target detection method based on RoI conditional multi-head attention according to claim 1, characterized in that, The method further includes: after obtaining the grid feature matrix, flattening the grid feature matrix, performing two-layer MLP aggregation processing, and outputting the category and location regression results of the three-dimensional candidate boxes.

9. A three-dimensional target detection device based on RoI conditional multi-head attention, used to implement the method as described in any one of claims 1-8, characterized in that, include: The candidate box acquisition module is used to acquire three-dimensional candidate boxes and their corresponding backbone feature maps. The three-dimensional candidate boxes have seven-dimensional geometric parameters. The first mapping module is used to linearly map the seven-dimensional geometric parameters into an attention query vector; A mesh construction module is used to construct a regular mesh within the 3D candidate box and sample it in the backbone feature map to obtain a mesh feature matrix. The second mapping module is used to linearly map the grid feature matrix into a key matrix and a value matrix, respectively. A multi-head attention aggregation module is used to perform multi-head attention aggregation on the key matrix and the value matrix with the attention query vector as the query, so as to obtain the aggregated features of the three-dimensional candidate box; The output module is used to output the category and location regression results of the 3D candidate boxes through a shared fully connected layer based on the aggregated features.

Images