Column feature coding target detection method based on space attention

Abstract

The invention discloses a column feature coding target detection method based on space attention, and the method comprises the steps: obtaining a point cloud data set in an automobile driving environment, sequentially carrying out the data enhancement and preprocessing, carrying out the point feature coding and height feature coding of a columnar point cloud in a column and between global columns, and carrying out the detection of a column feature coding target. The method comprises the following steps: re-weighting the relationship between points in a column by using a lightweight point attention mechanism, extracting multi-scale features through expansion convolution pooling, sequentially inputting a generated 2D pseudo image into an ECA module and a backbone network constructed based on BIFPN, carrying out feature extraction, sending an obtained 2D feature map into an SSD detection head, carrying out post-processing on the output of the detection head by using non-maximum suppression, and finally obtaining a final detection result. And obtaining a detection result. According to the method, the spatial structure and height details lost by cylindricity in a complex scene are obviously recovered, and the feature expressive power and the three-dimensional detection precision of the pseudo image are obviously improved.

Description

Technical Field

[0001] This invention relates to the field of 3D target detection in the field of autonomous driving, and more particularly to a target detection method based on column feature encoding using spatial attention. Background Technology

[0002] As a crucial component of autonomous driving systems, the accuracy and real-time performance of 3D object detection are essential for safe vehicle operation. The goal of 3D object detection is to identify and locate target objects such as vehicles, pedestrians, and obstacles from sensor data (such as LiDAR point cloud data and camera images), and to provide their 3D position, size, and category information.

[0003] Currently, 3D object detection technology is mainly based on LiDAR point cloud data, camera image data, or a fusion of both. Point cloud data differs significantly from image data; it is less affected by environmental factors such as lighting and can accurately capture the geometric characteristics of objects. Furthermore, point cloud-based 3D object detection methods can provide more accurate perception and localization due to their rich depth information. PointPillars is a popular point cloud-based 3D object detection method. It divides point cloud data into pillars and uses convolutional neural networks (CNNs) to extract features and detect objects from these pillars. By eliminating complex 3D convolutions and converting point cloud data into 2D pseudo-images, it can utilize more mature and efficient 2D convolutions. Therefore, PointPillars offers high computational performance and good detection accuracy, meeting the real-time requirements of autonomous driving systems.

[0004] However, existing PointPillars methods still have some limitations when handling complex scenes. In particular, the pillaring process forcibly compresses irregular and sparse point clouds into regular grids, ignoring the fine spatial relationships between points within a pillar and making it difficult to establish long-range dependencies across pillars. This results in a significant loss of effective information in sparse or non-uniformly distributed regions. Furthermore, height information, as an important feature in 3D space, is often simply normalized or averaged during the pillar encoding stage, failing to fully exploit the geometric features contained in the height dimension, thus affecting detection accuracy. Summary of the Invention

[0005] To address the issue of insufficient detection accuracy in complex scenes using existing pillar-based methods, this invention proposes a pillar feature encoding-based target detection method based on spatial attention. This method explicitly models point-to-point relationships within the pillar, restores spatial structure, and enhances the utilization of height features. By introducing dilated convolutional pooling layers, the receptive field of convolutional operations is expanded, enhancing the ability to capture global contextual information. Simultaneously, the spatial information of the original point cloud is preserved, improving the feature representation of the point cloud, thereby increasing the accuracy and robustness of 3D target detection and providing more reliable target detection technology support for autonomous driving systems.

[0006] The specific technical solution is as follows: A target detection method based on column feature encoding using spatial attention includes the following steps: S1: Obtain the point cloud dataset in the vehicle driving environment, and perform data augmentation and preprocessing in sequence; S2: The processed columnar point cloud is encoded with intra-column and global inter-column point features and height features. A lightweight point attention mechanism is used to reweight the inter-point relationships within the column. Then, multi-scale features are extracted through dilated convolution pooling to generate a 2D pseudo-image. S3: Input the 2D pseudo-image into the ECA module, adaptively learn the correlation between channels, and weight the feature channels; input the weighted feature map into the backbone network built based on BIFPN for feature extraction, perform bidirectional weighted feature fusion from top to bottom and from bottom to top through depthwise separable convolution, and stitch the features together in the channel dimension after unifying the number of channels to obtain the 2D feature map. S4: The 2D feature map is fed into the SSD detection head, and the output of the detection head is post-processed using non-maximum suppression to obtain the detection result.

[0007] Furthermore, in S1, the data enhancement includes the following operations in sequence: (1) Set the flip probability and perform a random flip operation on the point cloud and its corresponding prior box around the x-axis in the natural coordinate system; (2) Set the rotation radian and rotate the point cloud and its corresponding prior box around the z-axis in the natural coordinate system; (3) Set the scaling range and perform scale transformation on the point cloud and its corresponding prior box simultaneously.

[0008] Furthermore, in step S1, the data preprocessing includes the following operations in sequence: (1) Capture point cloud data within a specified range; (2) Randomly shuffle the order of the point cloud within the range to ensure the randomness of its point cloud distribution; (3) Project the shuffled point cloud data onto a two-dimensional bird's-eye view plane, and divide the two-dimensional bird's-eye view plane into multiple small squares, each square corresponding to a region of the original point cloud, forming a columnar network.

[0009] Furthermore, S2 is specifically implemented through the following sub-steps: S2.1: Obtain the point cloud obtained in S1. By calculating the offset of all points relative to the centroid and center of their respective columns, obtain the feature vector representation of the intra-column point cloud feature association, thus realizing intra-column point feature encoding. By calculating the offset of the point cloud coordinates in the column to the coordinates of the global center point, obtain the feature vector representation of the global inter-column point cloud feature association, thus realizing global inter-column point feature encoding. Use a multilayer perceptron to map the height histogram features to a high-dimensional space, thus realizing height feature encoding. S2.2: Feature vector representation of intra-cylinder point cloud features, feature vector representation of global inter-cylinder point cloud features, and height feature input Attention feature enhancement and extraction submodule; the Attention feature enhancement and extraction submodule includes: a first convolutional layer, a second convolutional layer, and a third convolutional layer; The first convolutional layer maps the number of channels of the input data from C to C / ratio through 1×1 convolution, and adjusts its shape to (b,n,C / ratio) through transpose operation to obtain intermediate feature A1, where b represents the batch of input points, n represents the number of points in a single sample, and ratio represents the scaling ratio. The second convolutional layer maps the number of channels of the input data from C to C / ratio through 1×1 convolution, and obtains the intermediate feature A2 with the shape (b, C / ratio, n). The third convolutional layer performs a 1×1 convolution on the input data, but does not change the number of channels C of the input data, to obtain the intermediate feature A3. Matrix multiplication is performed on intermediate features A1 and A2 to obtain a matrix of shape (b,n,n), which is then normalized using the Softmax function to obtain the attention weight matrix S. Intermediate feature A3 is multiplied by the transpose of the attention weight matrix S to obtain a weighted feature. The weighted feature is then added to the input data to obtain the enhanced feature. S2.3: Perform max pooling and average pooling on the enhanced features respectively; S2.4: Apply dilated convolution to the features extracted by max pooling and average pooling respectively to increase the receptive field and enrich the spatial context information, resulting in two dilated convolution features; S2.5: The height feature and the features after two dilated convolutions are added and fused, then mapped onto a two-dimensional plane and filled to obtain a 2D pseudo-image.

[0010] Furthermore, in S3, the 2D pseudo-image input ECA module performs adaptive global average pooling on the feature map of each channel to obtain a global feature vector; the global feature vector is then subjected to one-dimensional convolution to learn the correlation between channels, and its output is normalized by sigmoid to obtain the weight coefficient of each channel; the feature map of each channel is weighted according to the weight coefficient to obtain a weighted feature map, so as to improve the cross-channel interaction capability, highlight important features, and suppress unimportant features.

[0011] Furthermore, in step S3, obtaining the 2D feature map through the backbone network includes the following operations: (1) Extract features from the input feature map [C,H,W] through a 3×3 convolutional layer, and halve the number of channels to obtain a feature map F with dimensions [C / 2,H,W]; where C is the number of channels, H is the height, and W is the width; (2) By performing three downsampling operations in sequence, the spatial resolution is gradually reduced while the channel dimension is increased, resulting in three feature maps at different scales; the feature map obtained by the first downsampling operation is F. 1 [C,H / 2,W / 2], the feature map obtained by the second downsampling operation is F. 2 [2C,H / 4,W / 4], the feature map obtained by the third downsampling operation is F. 3 [4C,H / 8,W / 8]; (3) F 1 F 2 F 3 The input to BIFPN is processed by depthwise separable convolution to perform bidirectional weighted feature fusion from top to bottom and bottom to top, resulting in three feature maps F at different resolutions. OUT1 F OUT2 F OUT3 The depthwise separable convolution is divided into two parts. First, channel-wise convolution is used to find spatial features. Each channel uses a 3x3 convolution kernel. Then, 1x1 pointwise convolution is used to perform linear combination in the channel dimension to restore or change the number of channels and achieve cross-channel fusion. (4) For feature map F OUT1 F OUT2 F OUT3 Perform deconvolution operations on each feature map to restore them to a uniform size [2C,H / 2,W / 2]. After concatenation along the channel dimension, a 2D feature map with a size of [6C,H / 2,W / 2] is obtained.

[0012] Furthermore, three feature maps F with different resolutions were obtained. OUT1 F OUT2 F OUT3 The expression is as follows: In the formula, F C The fused feature maps; Spconv is a depthwise separable convolution, δ is the Swish activation function, upsample is the upsampling operation, and downsample is the downsampling operation; w1, w2, , , , , , , ε represents the trainable weights, and ε is the hyperparameter.

[0013] Furthermore, S4 is specifically implemented through the following sub-steps: S4.1: Preset prior bounding boxes for various targets in the input image. Each prior bounding box is represented by parameters (x,y,z,w,l,h,θ); where (x,y,z) are the center coordinates of the prior bounding box, w is the width of the prior bounding box, l is the length of the prior bounding box, h is the height of the prior bounding box, and θ is the rotation angle used to represent the orientation information of the prior bounding box. S4.2: Configure three prediction heads for classification, regression, and orientation classification to perform inference on the input image and output multiple predicted bounding boxes; the classification prediction head is used to determine whether each prior box contains the target and output which category the target belongs to; The regression prediction head predicts the coordinate offset and size scaling factor for each prior box, thereby correcting the position and size of the prior box to obtain the predicted bounding box; the coordinate offset is the correction value for the center coordinates (x,y,z), and the size scaling factor is the parameter for adjusting (w,l,h). The orientation classification prediction head is used to predict the actual orientation of the target by combining the initial orientation of the prior box; S4.3: After prediction, post-processing is performed to obtain optimized detection results: calculate the intersection-union ratio (IU) of each predicted bounding box with the ground truth box. For predicted bounding boxes with IU exceeding a preset threshold for the same target, non-maximum suppression is applied to retain the predicted bounding box with the highest confidence as the optimal box. The remaining predicted bounding boxes are redundant boxes that are detected repeatedly and are removed to ensure that only one optimal detection result is retained for each target. S4.4: Calculate three types of losses based on the prediction results: classification loss, localization loss, and orientation loss; the classification loss is used to reflect the accuracy of category judgment; the localization loss is used to measure the positional deviation between the predicted box and the ground truth box based on the intersection-union matching result; the orientation loss is used to characterize the error of orientation prediction. The classification loss, localization loss, and orientation loss are weighted and summed according to preset weights to obtain the total loss function. The parameters of the detection head are iteratively optimized through backpropagation, that is, S4.2-S4.4 are repeatedly executed until the total loss function reaches the preset threshold, and finally the detection result is output.

[0014] A target detection system based on spatial attention and column feature encoding, used to implement the target detection method based on spatial attention and column feature encoding, includes: a data augmentation module, a data preprocessing module, an encoding module, a backbone network module, and a detection module; The data augmentation module is used to expand and perturb the data to improve the model's generalization ability; the augmented data is then sent to the data preprocessing module. The data preprocessing module is used to crop and align the point cloud to avoid empty point clouds, providing efficient voxelization calculations for subsequent voxelization; the preprocessed data is then sent to the encoding module. The encoding module is used to encode point features and height features within and between global columns. It uses a lightweight point attention mechanism to reweight the relationships between points within the columns, and then extracts multi-scale features through dilated convolution pooling to generate a 2D pseudo-image, which is then output to the backbone network module. The backbone network module includes an ECA module and a backbone network built based on BIFPN, which is used to extract features from the 2D pseudo-images output by the encoding module to obtain 2D feature maps, which are then output to the detection module. The detection module is used to predict bounding boxes based on the input feature map, obtain classification loss, localization loss, and orientation loss, and sum them by weight to obtain the total loss function. Based on the total loss function, the detection head parameters in the detection module are iteratively optimized to obtain the final detection result.

[0015] The beneficial effects of this invention are: (1) This invention uses inter-column-intra-column joint feature encoding to explicitly restore the spatial structure and height details lost by columnarization in complex scenes while maintaining the efficient PointPillars framework, thus significantly improving the feature expressiveness and three-dimensional detection accuracy of pseudo-images.

[0016] (2) The feature extraction method proposed in this invention combines key point features and global features, increases the receptive field through dilated convolution and restores the dimension using linear layers, thereby enhancing the feature capture capability.

[0017] (3) In the backbone network proposed in this invention, the high-level semantics and low-level details can interact multiple times through bidirectional weighted feature fusion, which significantly alleviates the problems of gradient vanishing and information loss, improves the detection accuracy of small targets and suppresses redundant information. Attached Figure Description

[0018] Figure 1This is a flowchart of the target detection method based on spatial attention and column feature encoding in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the process of generating pseudo-images using the main feature enhancement extraction network in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the process of extracting multi-scale features by the Attention feature enhancement and extraction submodule in an embodiment of the present invention.

[0021] Figure 4 This is a flowchart of the data processing performed by the Efficient Channel Attention (ECA) module in an embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram of the backbone network constructed based on the Bidirectional Feature Pyramid Network (BIFPN) in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of the output of detection results by the detection head of the Single Shot Detector (SSD) in an embodiment of the present invention. Detailed Implementation

[0024] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The objectives and effects of the present invention will become clearer as a result. The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0025] like Figure 1 As shown, a target detection method based on column feature encoding using spatial attention includes the following steps: S1: Obtain the point cloud dataset from the vehicle's driving environment, and then perform data augmentation and preprocessing. This is achieved through the following sub-steps: S1.1: Acquire point cloud datasets from the vehicle's driving environment using LiDAR. In this embodiment, the KITTI dataset from public roads is acquired. The KITTI dataset is a widely used algorithm evaluation dataset in autonomous driving scenarios, containing point cloud data and image data. The point cloud data is obtained by scanning with a 64-line LiDAR. In the experiment, the dataset is divided into a test set and a training set, with 3769 data points in the test set and 3712 data points in the validation set.

[0026] S1.2: Data augmentation of the point cloud dataset. Appropriate data augmentation operations can effectively increase the diversity of point cloud data and enhance the model's generalization ability. In the natural coordinate system, the data augmentation operations include the following: (1.2.1) Randomly flip around the x-axis. Set the flip probability and perform a random flip operation on the point cloud and its corresponding prior box around the x-axis; in this embodiment, the flip probability is set to 50%.

[0027] (1.2.2) Global rotation around the z-axis. Set the rotation radian to perform a global rotation of the point cloud and prior box around the z-axis; in this embodiment, the rotation radian range is [0.78539816, 0.78539816].

[0028] (1.2.3) Scale scaling. Set the scaling range and perform scale scaling transformation on the point cloud and the prior bounding box simultaneously; in this embodiment, the scaling range is set to [0.95, 1.05].

[0029] S1.3: Preprocess the enhanced data, including the following operations in sequence: (1.3.1) Extract point cloud data within a specified range; in this embodiment, the specified range is [0, -39.68, -3, 69.12, 39.68, 1].

[0030] (1.3.2) Randomly shuffle the order of point clouds within the range to ensure the randomness of the point cloud distribution.

[0031] (1.3.3) Project the shuffled point cloud data onto the two-dimensional bird's-eye view (BEV) plane, and divide the BEV plane into multiple small squares, each square corresponding to a region of the point cloud (the point cloud enhanced in S1.2), forming a columnar network.

[0032] S2: The processed columnar point cloud is input into the columnar feature enhancement extraction network for global inter-column point feature encoding and height feature encoding to enrich the feature representation between the original point clouds. Then, a lightweight spatial attention mechanism is used to reweight the inter-point relationships within the columns, followed by dilated convolutional pooling to extract multi-scale features, ultimately generating a highly expressive 2D pseudo-image. The lightweight spatial attention mechanism provides a global context for the output features, where the context is selectively aggregated according to the point spatial attention weight matrix, enhancing the similarity semantic features between point clouds and improving the representational power of the input features.

[0033] like Figure 2 As shown, S2 is implemented through the following sub-steps: S2.1: Obtain the point cloud after S1 preprocessing, and encode the point features within the column and the global column for each column in the columnar network, as well as the height feature.

[0034] Specifically, by calculating the offset of all points relative to the centroid of their respective columns (i.e., the average position of all points) and the center, the feature vector representation of the point cloud features within the column is obtained, thereby realizing the point feature encoding within the column.

[0035] By calculating the offset between the point cloud coordinates in the column and the coordinates of the global center point, the feature vector representation of the global column point cloud feature association is obtained, thus realizing the global column point feature encoding.

[0036] A multilayer perceptron (MLP) is used to map the height histogram features to a high-dimensional space, thereby achieving height feature encoding. The formula for the mapped height histogram features is as follows: In the formula, Here, m represents the mapped height histogram features, and h represents the MLP operation used for feature dimensionality upscaling. i For the distribution of point cloud heights within the column, w m Let m be the learnable weights.

[0037] S2.2: The feature vector representations of the intra-pillar and global inter-pillar point cloud features obtained in S2.1, along with the height features, are input into the Attention feature enhancement and extraction submodule. In this submodule, the feature vector representations of the intra-pillar and global inter-pillar point cloud features are first compressed proportionally by a 1×1 convolution to reduce computational complexity while retaining core information. Then, through matrix operations on the two branch features and Softmax, the relationships between points are learned and adaptive attention weights are generated. Finally, these weights are used to perform global weighted aggregation on the original dimension point cloud features and make residual connections with the input features. This enhances the expressive power of point cloud features by fusing local details and global structural information, enabling the columnar network to better capture local and global structural information.

[0038] Specifically, such as Figure 3As shown, the Attention feature enhancement and extraction submodule contains three convolutional layers. The first convolutional layer (Conv1 in the figure) maps the number of channels of the input data from C to C / ratio using a 1×1 convolution, and adjusts its shape to (b, n, C / ratio) through a transpose operation to obtain intermediate feature A1, where b represents the batch of input points, n represents the number of points in a single sample, and ratio represents the scaling ratio. The second convolutional layer (Conv2 in the figure) maps the number of channels of the input data from C to C / ratio using a 1×1 convolution to obtain intermediate feature A2, with a shape of (b, C / ratio, n). The third convolutional layer (Conv3 in the figure) increases the dimensionality of the input data using a 1×1 convolution without changing the number of channels C of the input data to obtain intermediate feature A3. In this embodiment, the number of channels C is set to 64, and the scaling ratio is set to 8.

[0039] Matrix multiplication is performed on intermediate features A1 and A2 to calculate the pointwise similarity matrix, which has the shape (b,n,n). The matrix is ​​then normalized along the column direction using the Softmax function to obtain the attention weight matrix S. Matrix multiplication is performed on intermediate feature A3 and the transpose of attention weight matrix S to obtain the weighted feature. This weighted feature is then added to the original input data to obtain the enhanced feature.

[0040] S2.3: Perform max pooling and average pooling on the enhanced features obtained in S2.2. Max pooling is used to extract keypoint features, while average pooling is used to extract global features. Combining the two improves adaptability to environmental changes. The expressions for max pooling and average pooling are as follows: In the formula, X represents the feature representation of the point cloud after S2.2 enhancement, MaxPool is the max pooling operation, and X max X represents the result of max pooling; AveragePool represents the average pooling operation. ave This represents the average pooling result.

[0041] S2.4: Dilated convolution is applied to the features extracted by max pooling and average pooling to increase the receptive field and enrich the spatial context information. The expression for dilated convolution is as follows: In the formula, DilatedConv represents the dilated convolution operation. For X max The result after dilated convolution, For X aveThe result of dilated convolution, where r is the dilation rate.

[0042] S2.5: The height feature and the two features obtained after dilation and convolution are added and fused, then mapped onto a two-dimensional plane and filled to obtain a 2D pseudo-image. The expression for feature fusion is as follows: In the formula, The result after feature fusion. These are learnable or preset parameters used to control the fusion ratio of max pooling features and average pooling features.

[0043] S3: The 2D pseudo-image obtained in S2 is sequentially input into the ECA module and the backbone network built based on BIFPN for feature extraction, resulting in a 2D feature map. The ECA module adaptively learns the correlation between channels, weights the feature channels, and highlights important feature information. BIFPN preserves multi-scale features through bidirectional weighted fusion with learnable weights and introduces depthwise separable convolutions to reduce the number of parameters. This is implemented through the following sub-steps: S3.1: Send the 2D pseudo-image obtained in S2 into the ECA module, such as... Figure 4 As shown, this module first performs adaptive global average pooling on the feature maps of each channel to obtain a global feature vector; then, it performs a one-dimensional convolution on the global feature vector, which learns the correlation between channels; finally, it normalizes the output of the convolution operation using sigmoid to obtain the weight coefficients for each channel. Then, it weights the feature maps of each channel according to these weight coefficients, improving cross-channel interaction capabilities, highlighting important features, and suppressing unimportant features. The formula for the one-dimensional convolution kernel size in the one-dimensional convolution operation is as follows: In the formula, K is the kernel size, γ and As a custom hyperparameter, `odd` represents rounding down to the nearest odd number, which helps maintain symmetrical filling across the channel dimensions. In this embodiment, Choose 1, and γ should be 2.

[0044] S3.2: The output of the ECA module is fed into the backbone network built on BIFPN for feature extraction. For example... Figure 5 As shown, the backbone network module includes four standard convolutional layers, each using a 3×3 convolution with the ReLU activation function. In this embodiment, the first convolutional layer contains six 3×3 convolutions, the second and third convolutional layers each contain five 3×3 convolutions, and the fourth convolutional layer contains three 3×3 convolutions. To address the issue of premature downsampling of point cloud data leading to the loss of crucial information, the downsampling operation is removed from the first convolutional layer, and the number of network layers is increased to reduce the loss of original point cloud information. The backbone network specifically includes the following operations: (3.2.1) Using a feature map of dimension [C,H,W] as input (where C is the number of channels, H is the height, and W is the width), features are extracted through a 3×3 convolutional layer, while the number of channels is halved to obtain a feature map F of dimension [C / 2,H,W].

[0045] (3.2.2) Through three downsampling operations, the spatial resolution is gradually reduced while the channel dimension is increased (i.e., the number of channels of the feature map is doubled and the size is halved layer by layer), and deeper features are extracted layer by layer, finally resulting in three feature maps of different scales. The feature maps after the three downsampling operations are F 1 [C,H / 2,W / 2]、F 2 [2C,H / 4,W / 4]、F 3 [4C,H / 8,W / 8]; In this embodiment, in the feature extraction process, the dimension of the input feature map is [64,496,432], and the dimension of the feature map F obtained by the 3×3 convolutional layer is [32,496,432]. The three feature maps with different scales are: F 1 [64,248,216]、F 2 [128,124,108]、F 3 [256,62,54].

[0046] (3.2.3) F 1 F 2 F 3 The input to BIFPN is processed by depthwise separable convolution to perform bidirectional weighted feature fusion from top to bottom and bottom to top, resulting in three feature maps F at different resolutions. OUT1 F OUT2 F OUT3 The bidirectional weighted fusion includes upsampling, downsampling, depthwise separable convolution, and the Swish activation function. First, a ReLU activation function is used to suppress negative values, then normalization is performed along the channel dimension to obtain normalized weights. A depthwise separable convolution is then used to downsample the high-resolution feature map and weightedly fuse it with the low-resolution feature map, and an upsampling of the low-resolution feature map is used to weightedly fuse it with the high-resolution feature map. Finally, the Swish activation function is applied after the depthwise separable convolution to output a multi-scale feature map.

[0047] Depthwise separable convolution is divided into two parts. First, channel-wise convolution is used to find spatial features, with each channel using a separate 3x3 convolution kernel (without crossing channels). Then, 1x1 pointwise convolution is used to linearly combine the features along the channel dimension, restoring or changing the number of channels to achieve cross-channel fusion. The feature fusion formula is as follows: In the formula, F C F is the fused feature map output by BIFPN. 1 F is the feature map after the first downsampling. 2 F is the feature map after the second downsampling. 3 This is the feature map after the third downsampling; Spconv is depthwise separable convolution, δ is the Swish activation function, upsample is the upsampling operation, and downsample is the downsampling operation; w1, w2, ... , , , , , , ε represents the trainable weights, and ε is the hyperparameter.

[0048] (3.2.4) The three feature maps with different resolutions are deconvolved to restore them to a uniform size [2C, H / 2, W / 2]. After concatenation along the channel dimension, a 2D feature map of size [6C, H / 2, W / 2] is obtained. This 2D feature map contains features at different scales. In this embodiment, F OUT1 F OUT2 The deconvolution stride is 2, F OUT3 The deconvolution stride is 4; the size of the 2D feature map is [384, 248, 216].

[0049] S4: The 2D feature map obtained in S3 is fed into the SSD detection head. The output of the detection head is post-processed to obtain the detection results (including the category and location information of obstacles of different sizes), thus completing the autonomous driving target detection task. Figure 6 As shown, the SSD detection header includes the following steps: S4.1: The detection head presets prior bounding boxes for various targets in the input image (in this embodiment, it presets prior bounding boxes for three types of targets: vehicles, pedestrians, and bicycles). Each prior bounding box is represented by (x, y, z, w, l, h, θ) to ensure that it matches the actual shape and spatial distribution characteristics of the three types of targets. Among them, (x, y, z) are the center coordinates of the prior bounding box, (w, l, h) are the width, length, and height of the prior bounding box, and θ is the rotation angle used to represent the orientation information of the prior bounding box.

[0050] S4.2: Configure three prediction heads: classification, regression, and orientation classification. These heads perform inference on the input image and output multiple predicted bounding boxes. The classification prediction head determines whether each prior box contains the target and outputs the target's class. The regression prediction head predicts the coordinate offset and scaling factor for each prior box. The coordinate offset is the correction value for the center coordinates (x, y, z), and the scaling factor is the parameter used to adjust (w, l, h). By correcting the position and size of the prior box using the coordinate offset and scaling factor, the corrected predicted bounding box more accurately matches the actual position and size of the target. The orientation classification prediction head combines the initial orientation of the prior box with the predicted actual orientation of the target.

[0051] S4.3: After prediction, post-processing is performed to obtain optimized detection results: Calculate the intersection over union (IoU) ratio of each predicted bounding box to the ground truth box. For predicted bounding boxes with an IoU ratio exceeding a preset threshold for the same target, apply non-maximum suppression (NMS) to retain the predicted bounding box with the highest confidence as the optimal box. The remaining predicted bounding boxes are redundant boxes that are detected repeatedly and are removed to ensure that only one optimal detection result is retained for each target.

[0052] S4.4: Calculate three types of losses based on the prediction results: classification loss (reflecting the accuracy of category judgment), localization loss (based on IOU matching results, measuring the positional deviation between the predicted bounding box and the ground truth bounding box), and orientation loss (characterizing the error in orientation prediction). The three types of losses are weighted and summed according to preset weights to obtain the total loss function. The parameters of the detection head are iteratively optimized through backpropagation, i.e., S4.2-S4.4 are repeated until the total loss function reaches a preset threshold, ultimately outputting a stable and reliable detection result.

[0053] To implement the above-mentioned target detection method based on spatial attention and column feature encoding, this embodiment also proposes a target detection system based on spatial attention and column feature encoding. The system includes: a data augmentation module, a data preprocessing module, an encoding module, a backbone network module, and a detection module.

[0054] The data augmentation module is used to augment and perturb the data to improve the model's generalization ability; the output of this module is sent to the data preprocessing module.

[0055] The data preprocessing module is used to crop and align point clouds to avoid empty point clouds, providing efficient voxelization calculations for subsequent processing; the output of this module is sent to the encoding module.

[0056] The encoding module is used to encode point features within and between global columns, as well as height features. It uses a lightweight point attention mechanism to reweight the relationships between points within the columns, and then extracts multi-scale features through dilated convolution pooling to generate a 2D pseudo-image, which is then output to the backbone network module.

[0057] The backbone network module includes an ECA module and a backbone network built based on BIFPN, which is used to extract features from the 2D pseudo-images output by the encoding module to obtain 2D feature maps, which are then output to the detection module.

[0058] The detection module is used to predict bounding boxes based on the input feature map, obtain classification loss, localization loss, and orientation loss, and sum them by weight to obtain the total loss function. Based on the total loss function, the detection head parameters in the detection module are iteratively optimized to obtain the final detection result.

[0059] This invention proposes a target detection method based on point spatial attention feature encoding. By encoding point cloud features, it introduces feature vector representations of global inter-pillar point cloud feature association and intra-pillar point cloud feature association. A point attention mechanism enhances the feature correlation between intra-pillar and inter-pillar point clouds, enabling the model to better learn the relationships between point clouds and effectively improve the feature representation of high-level point clouds. By introducing an ECA attention mechanism, it adaptively learns the correlation between channels, allowing the model to focus more on key features, suppress redundant information, and improve the model's adaptability and robustness. Through a bidirectional weighted fusion method based on depthwise separable convolution, feature maps of different resolutions can effectively achieve feature interaction, improving the accuracy of small target detection.

[0060] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A target detection method based on column feature encoding using spatial attention, characterized in that, Includes the following steps: S1: Obtain the point cloud dataset in the vehicle driving environment, and perform data augmentation and preprocessing in sequence; S2: The processed columnar point cloud is encoded with intra-column and global inter-column point features and height features. A lightweight point attention mechanism is used to reweight the inter-point relationships within the column. Then, multi-scale features are extracted through dilated convolution pooling to generate a 2D pseudo-image. S3: Input the 2D pseudo-image into the ECA module, adaptively learn the correlation between channels, and weight the feature channels; input the weighted feature map into the backbone network built based on BIFPN for feature extraction, perform bidirectional weighted feature fusion from top to bottom and from bottom to top through depthwise separable convolution, and stitch the features together in the channel dimension after unifying the number of channels to obtain the 2D feature map. S4: The 2D feature map is fed into the SSD detection head, and the output of the detection head is post-processed using non-maximum suppression to obtain the detection result.

2. The target detection method based on spatial attention and column feature encoding according to claim 1, characterized in that, In step S1, data augmentation includes the following operations in sequence: (1) Set the flip probability and perform a random flip operation on the point cloud and its corresponding prior box around the x-axis in the natural coordinate system; (2) Set the rotation radian and rotate the point cloud and its corresponding prior box around the z-axis in the natural coordinate system; (3) Set the scaling range and perform scale transformation on the point cloud and its corresponding prior box simultaneously.

3. The target detection method based on spatial attention and column feature encoding according to claim 1, characterized in that, In step S1, the data preprocessing includes the following operations in sequence: (1) Capture point cloud data within a specified range; (2) Randomly shuffle the order of the point cloud within the range to ensure the randomness of its point cloud distribution; (3) Project the shuffled point cloud data onto a two-dimensional bird's-eye view plane, and divide the two-dimensional bird's-eye view plane into multiple small squares, each square corresponding to a region of the original point cloud, forming a columnar network.

4. The target detection method based on spatial attention and column feature encoding according to claim 1, characterized in that, S2 is specifically implemented through the following sub-steps: S2.1: Obtain the point cloud obtained in S1. By calculating the offset of all points relative to the centroid and center of their respective columns, obtain the feature vector representation of the intra-column point cloud feature association, thus realizing intra-column point feature encoding. By calculating the offset of the point cloud coordinates in the column to the coordinates of the global center point, obtain the feature vector representation of the global inter-column point cloud feature association, thus realizing global inter-column point feature encoding. Use a multilayer perceptron to map the height histogram features to a high-dimensional space, thus realizing height feature encoding. S2.2: Feature vector representation of intra-cylinder point cloud features, feature vector representation of global inter-cylinder point cloud features, and height feature input Attention feature enhancement and extraction submodule; the Attention feature enhancement and extraction submodule includes: a first convolutional layer, a second convolutional layer, and a third convolutional layer; The first convolutional layer maps the number of channels of the input data from C to C / ratio through 1×1 convolution, and adjusts its shape to (b,n,C / ratio) through transpose operation to obtain intermediate feature A1, where b represents the batch of input points, n represents the number of points in a single sample, and ratio represents the scaling ratio. The second convolutional layer maps the number of channels of the input data from C to C / ratio through 1×1 convolution, and obtains the intermediate feature A2 with the shape (b, C / ratio, n). The third convolutional layer performs a 1×1 convolution on the input data, but does not change the number of channels C of the input data, to obtain the intermediate feature A3. Matrix multiplication is performed on intermediate features A1 and A2 to obtain a matrix of shape (b,n,n), which is then normalized using the Softmax function to obtain the attention weight matrix S. Intermediate feature A3 is multiplied by the transpose of the attention weight matrix S to obtain a weighted feature. The weighted feature is then added to the input data to obtain the enhanced feature. S2.3: Perform max pooling and average pooling on the enhanced features respectively; S2.4: Apply dilated convolution to the features extracted by max pooling and average pooling respectively to increase the receptive field and enrich the spatial context information, resulting in two dilated convolution features; S2.5: The height feature and the features after two dilated convolutions are added and fused, then mapped onto a two-dimensional plane and filled to obtain a 2D pseudo-image.

5. The target detection method based on spatial attention and column feature encoding according to claim 1, characterized in that, In step S3, the 2D pseudo-image is input into the ECA module, which performs adaptive global average pooling on the feature map of each channel to obtain a global feature vector. The global feature vector is then subjected to one-dimensional convolution to learn the correlation between channels, and its output is normalized by sigmoid to obtain the weight coefficient of each channel. The feature map of each channel is weighted according to the weight coefficient to obtain a weighted feature map, so as to improve the cross-channel interaction capability, highlight important features, and suppress unimportant features.

6. The target detection method based on spatial attention and column feature encoding according to claim 1, characterized in that, In step S3, obtaining the 2D feature map through the backbone network includes the following operations: (1) Extract features from the input feature map [C,H,W] through a 3×3 convolutional layer, and halve the number of channels to obtain a feature map F with dimensions [C / 2,H,W]; where C is the number of channels, H is the height, and W is the width; (2) By performing three downsampling operations in sequence, the spatial resolution is gradually reduced while the channel dimension is increased, resulting in three feature maps at different scales; the feature map obtained by the first downsampling operation is F. 1 [C,H / 2,W / 2], the feature map obtained by the second downsampling operation is F. 2 [2C,H / 4,W / 4], the feature map obtained by the third downsampling operation is F. 3 [4C,H / 8,W / 8]; (3) F 1 F 2 F 3 The input to BIFPN is processed by depthwise separable convolution to perform bidirectional weighted feature fusion from top to bottom and bottom to top, resulting in three feature maps F with different resolutions. OUT1 F OUT2 F OUT3 The depthwise separable convolution is divided into two parts. First, channel-wise convolution is used to find spatial features. Each channel uses a 3x3 convolution kernel. Then, 1x1 pointwise convolution is used to perform linear combination in the channel dimension to restore or change the number of channels and achieve cross-channel fusion. (4) For feature map F OUT1 F OUT2 F OUT3 Perform deconvolution operations on each feature map to restore them to a uniform size [2C,H / 2,W / 2]. After concatenation along the channel dimension, a 2D feature map with a size of [6C,H / 2,W / 2] is obtained.

7. The target detection method based on spatial attention and column feature encoding according to claim 6, characterized in that, Three feature maps F with different resolutions were obtained. OUT1 F OUT2 F OUT3 The expression is as follows: ； ； ； ； In the formula, F C The fused feature maps; Spconv is a depthwise separable convolution, δ is the Swish activation function, upsample is the upsampling operation, and downsample is the downsampling operation; w1, w2, , , , , , , These are the trainable weights, and ε is the hyperparameter.

8. The target detection method based on spatial attention and column feature encoding according to claim 1, characterized in that, S4 is specifically implemented through the following sub-steps: S4.1: Preset prior bounding boxes for various targets in the input image. Each prior bounding box is represented by parameters (x,y,z,w,l,h,θ); where (x,y,z) are the center coordinates of the prior bounding box, w is the width of the prior bounding box, l is the length of the prior bounding box, h is the height of the prior bounding box, and θ is the rotation angle used to represent the orientation information of the prior bounding box. S4.2: Configure three prediction heads for classification, regression, and orientation classification to perform inference on the input image and output multiple predicted bounding boxes; the classification prediction head is used to determine whether each prior box contains the target and output which category the target belongs to; The regression prediction head predicts the coordinate offset and size scaling factor for each prior box, thereby correcting the position and size of the prior box to obtain the predicted bounding box; the coordinate offset is the correction value for the center coordinates (x,y,z), and the size scaling factor is the parameter for adjusting (w,l,h). The orientation classification prediction head is used to predict the actual orientation of the target by combining the initial orientation of the prior box; S4.3: After prediction, post-processing is performed to obtain optimized detection results: calculate the intersection-union ratio (IU) of each predicted bounding box with the ground truth box. For predicted bounding boxes with IU exceeding a preset threshold for the same target, non-maximum suppression is applied to retain the predicted bounding box with the highest confidence as the optimal box. The remaining predicted bounding boxes are redundant boxes that are detected repeatedly and are removed to ensure that only one optimal detection result is retained for each target. S4.4: Calculate three types of losses based on the prediction results: classification loss, localization loss, and orientation loss; the classification loss is used to reflect the accuracy of category judgment; the localization loss is used to measure the positional deviation between the predicted box and the ground truth box based on the intersection-union matching result; the orientation loss is used to characterize the error of orientation prediction. The classification loss, localization loss, and orientation loss are weighted and summed according to preset weights to obtain the total loss function. The parameters of the detection head are iteratively optimized through backpropagation, that is, S4.2-S4.4 are repeatedly executed until the total loss function reaches the preset threshold, and finally the detection result is output.

9. A target detection system based on spatial attention and column feature encoding, used to implement the target detection method based on spatial attention and column feature encoding as described in any one of claims 1-8, characterized in that, include: Data augmentation module, data preprocessing module, encoding module, backbone network module, detection module; The data augmentation module is used to expand and perturb the data to improve the model's generalization ability; the augmented data is then sent to the data preprocessing module. The data preprocessing module is used to crop and align the point cloud to avoid empty point clouds, providing efficient voxelization calculations for subsequent voxelization; the preprocessed data is then sent to the encoding module. The encoding module is used to encode point features and height features within and between global columns. It uses a lightweight point attention mechanism to reweight the relationships between points within the columns, and then extracts multi-scale features through dilated convolution pooling to generate a 2D pseudo-image, which is then output to the backbone network module. The backbone network module includes an ECA module and a backbone network built based on BIFPN, which is used to extract features from the 2D pseudo-images output by the encoding module to obtain 2D feature maps, which are then output to the detection module. The detection module is used to predict bounding boxes based on the input feature map, obtain classification loss, localization loss, and orientation loss, and sum them by weight to obtain the total loss function. Based on the total loss function, the detection head parameters in the detection module are iteratively optimized to obtain the final detection result.

Images