A target detection method based on wide-area receptive field spatial attention

Abstract

This invention discloses a target detection method based on wide-area receptive field spatial attention, comprising: preparing image datasets for training and testing; constructing a target detection network based on wide-area receptive field spatial attention, including four parts: Backbone, Neck, Head, and MSA; and using the trained network to extract features from the test set images. This invention captures pixel-level feature information from the perspective of a wide receptive field, while also considering the interaction between different feature information, significantly improving the feature extraction effect without significantly increasing the number of parameters and computational cost.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a target detection method based on wide-area receptive field spatial attention. Background Technology

[0002] Against the backdrop of the development of deep learning, convolutional neural networks have gained increasing acceptance and are being applied more and more widely. Deep learning-based object detection algorithms utilize convolutional neural networks (CNNs) to automatically select features, and then input these features into a detector for object classification and localization.

[0003] In neural network learning, generally speaking, the more parameters a model has, the stronger its expressive power and the greater the amount of information it can store. However, this can lead to information overload. By introducing an attention mechanism, focusing on information more critical to the current task from a large amount of input information, reducing attention to other information, and even filtering out irrelevant information, the problem of information overload can be solved, and the efficiency and accuracy of task processing can be improved.

[0004] In recent years, attention mechanisms have been widely used in various deep learning tasks, such as object detection, semantic segmentation, and pose estimation. Attention is divided into soft attention and hard attention. Soft attention mechanisms are further divided into three attention domains: spatial domain, channel domain, and hybrid domain. The spatial domain refers to the corresponding spatial transformation in the image. The channel domain directly focuses information on the global channels. The hybrid domain includes both channel attention and spatial attention. To enable the network to focus more attention on the region around salient objects, this invention proposes a wide-area receptive field spatial attention module to process the extracted feature maps. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a target detection method based on wide-area receptive field spatial attention, which improves the feature representation capability of the network without excessively increasing the number of model parameters. This invention mainly includes pooling operations, reshaping operations, dilated convolutional blocks, and upsampling operations, which greatly enhance the ability to represent important feature information.

[0006] To achieve the above objectives, the technical solution provided by this invention is a target detection method based on wide-area receptive field spatial attention, comprising the following steps:

[0007] Step 1: Prepare an image dataset for testing and training.

[0008] Step 2: Construct a target detection network based on wide-area receptive field spatial attention.

[0009] Step 3: Train the target detection network model based on wide-area receptive field spatial attention using the training set images.

[0010] Step 4: Use the network model trained in Step 3 to perform object detection on the test set images.

[0011] Furthermore, in step 1, all images are resized to 512×512 for multi-scale training, and data augmentation is performed on the image dataset using a series of operations: random flipping, padding, random cropping, normalization, and image distortion processing.

[0012] Furthermore, the target detection network based on wide receptive field spatial attention in step 2 consists of four parts: Backbone, Neck, Head, and MSA wide receptive field spatial attention. The Backbone uses a ResNet50 backbone network to extract image features. The Neck structure is used to connect the Backbone and Head to fuse features. The Head is used for object detection to achieve target classification and regression. MSA is placed between the Backbone and Neck, and between the Neck and Head.

[0013] The ResNet50 backbone network outputs four feature maps of different sizes [C1, C2, C3, C4] with strides of [4, 8, 16, 32] and channel sizes of [256, 512, 1024, 2048]. The Neck structure uses three feature maps from the Backbone [C2, C3, C4]. After a 1×1 convolution, the channels are reduced to 256. The feature maps are then fused using [P1, P2, P3] in the FPN structure. P3 is then downsampled twice to obtain P4 and P5. Finally, a 3×3 convolution is used to ablate the feature maps, outputting five feature maps of different sizes with strides of [8, 16, 32, 64, 128] and channel sizes of 256.

[0014] The structure of MSA is as follows: Let F∈R C×H×W Let F be the input tensor, where C, H, and W represent the channel, height, and width, respectively; then, F'∈R is obtained by halving the height and width of F through a 3×3 convolution. C×H / 2×W / 2 Then, after each of the two processes undergoes a regular convolution branch, F0∈R is obtained. 1×H / 2×W / 2 F1∈R is obtained by three depthwise separable convolution branches. C / 2×H / 2×W / 2 F2∈R C / 2×H / 2×W / 2 F3∈R C / 2×H / 2×W / 2 Then, F1, F2, and F3 are reshaped into M1, M2, and M3 through dimensional changes (from three-dimensional to two-dimensional), that is:

[0015]

[0016] M1, M2, and M3 have the same matrix shape [H / 2*W / 2, C / 2], where H / 2*W / 2 and C / 2 represent the rows and columns of the matrix. Multiplying M1, M2, and M3 respectively yields three relation matrices N1, N2, and N3. Each value in the relation matrix represents the relationship between every two pixels in the feature. The formulas for calculating N1, N2, and N3 are as follows:

[0017]

[0018] In the formula, M1 represents matrix multiplication. T M2 T M3 T Let M1, M2, and M3 be the transpose matrices respectively, and let N1, N2, and N3 have the shape [H / 2*W / 2, H / 2*W / 2], where H / 2*W / 2 and H / 2*W / 2 represent the rows and columns of the matrix respectively;

[0019] N1, N2, and N3 are reshaped into T1, T2, and T3, with shapes [H / 2*W / 2, H / 2, W / 2], where H / 2*W / 2, H / 2, and W / 2 represent the channel, height, and width, respectively. To obtain an output that includes more useful global priors, F0 is concatenated with T1, T2, and T3 to obtain feature F. M :

[0020] F M =concat[F0,T1,T2,T3] (3)

[0021] In the formula, F M ∈R (H / 2*W / 2)*3×H / 2×W / 2 H / 2, W / 2, (H / 2*W / 2)*3 represent height, width, and channel;

[0022] F M The input features are reshaped into Y1 to generate attention weights. Then, an interpolation algorithm is used to adjust the attention weights Y1 into Y2, resulting in a space with the same size as the input features. After that, a reshaping operation is used to reshape Y2 into a three-dimensional space with a size of [1, W, H]. Finally, the input features are multiplied by the Sigmoid function to obtain the final output.

[0023] Furthermore, in step 3, the training set image size is uniformly set to 512×512, the learning rate is set to 0.001, the batch size is set to 4, the training number is 12 epochs, and the learning rate is reduced to 1 / 10 of the original value in the 8th and 11th epochs.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] Compared with conventional spatial attention, the method proposed in this invention captures pixel-level feature information from the perspective of a wide receptive field, while also considering the interaction between different feature information. This greatly improves the feature extraction effect without significantly increasing the number of parameters and computational cost. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the network structure of the present invention.

[0027] Figure 2 This is a schematic diagram of the attention structure in the wide receptive field.

[0028] Figure 3 This is a schematic diagram illustrating the network detection effect of the present invention. Detailed Implementation

[0029] This invention provides a target detection method based on wide-area receptive field spatial attention. The technical solution of this invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] like Figure 1 As shown, the process of this embodiment of the invention includes the following steps:

[0031] Step 1: Prepare an image dataset for testing and training.

[0032] The COCO 2017 dataset was selected. It is a large and rich dataset for object detection, segmentation, and captioning, containing 80 categories for detection, namely "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "ship", "traffic light", and other common objects in daily life. It contains four files: annotations, test2017, train2017, and val2017. The train file contains 118,287 images, the val file contains 5,000 images, and the test file contains 28,660 images. The annotations are a collection of annotation types: object instances, object keypoints, and image captions, stored in JSON files.

[0033] All images were resized to 512×512 for multi-scale training. Data augmentation was applied to the image dataset using a series of operations: random flipping, padding, random cropping, normalization, and image distortion removal.

[0034] Step 2: Construct a target detection network based on wide-area receptive field spatial attention.

[0035] like Figure 1 As shown, the object detection network based on wide receptive field spatial attention consists of four parts: Backbone, Neck, Head, and MSA (Multiple receptive field Spatial Attention). The Backbone uses a ResNet50 backbone network to extract image features. This network outputs four feature maps of different sizes [C1, C2, C3, C4] with a stride of [4, 8, 16, 32] and a channel size of [256, 512, 1024, 2048]. The Neck structure connects the Backbone and Head for feature fusion. It uses three feature maps from the Backbone [C2, C3, C4], which are reduced to 256 channels after a 1×1 convolution. These are then fused using [P1, P2, P3] within the FPN structure. P3 is downsampled twice to obtain P4 and P5. Finally, a 3×3 convolution is used to ablate the feature maps, outputting five feature maps of different sizes with strides of [8, 16, 32, 64, 128] and a channel size of 256. The Head is used for object detection, enabling target classification and regression.

[0036] MSA's wide receptive field spatial attention mechanism is placed between the Backbone and Neck, and between the Neck and Head, i.e. Figure 1 The word "MSA" appears in eight positions.

[0037] The structure of MSA is as follows: Figure 2 As shown, let F∈R C×H×W Let F be the input tensor, where C, H, and W represent the channel, height, and width, respectively. To reduce parameters and computational cost, a 3×3 convolution is used to halve the height and width of F to obtain F'∈R. C×H / 2×W / 2 Then, after each of the two processes undergoes a regular convolution branch, F0∈R is obtained. 1×H / 2×W / 2 F1∈R is obtained from three depthwise separable convolutional branches. C / 2×H / 2×W / 2 F2∈R C / 2×H / 2×W / 2 F3∈R C / 2×H / 2×W / 2 Then, F1, F2, and F3 are reshaped into M1, M2, and M3 through dimensional changes (from three-dimensional to two-dimensional), that is:

[0038]

[0039] M1, M2, and M3 have the same matrix shape [H / 2*W / 2, C / 2], where H / 2*W / 2 and C / 2 represent the rows and columns of the matrix. Multiplying M1, M2, and M3 respectively yields three relation matrices N1, N2, and N3. Each value in the relation matrix represents the relationship between every two pixels in the feature. The formulas for calculating N1, N2, and N3 are as follows:

[0040]

[0041] In the formula, M1 represents matrix multiplication. T M2 T M3 T These are the transposes of M1, M2, and M3, respectively. N1, N2, and N3 have shapes [H / 2*W / 2, H / 2*W / 2], where H / 2*W / 2 and H / 2*W / 2 represent the rows and columns of the matrix, respectively. Matrix multiplication is beneficial for fusing richer feature information and for extracting features more carefully from a pixel perspective.

[0042] N1, N2, and N3 are reshaped into T1, T2, and T3 for the next feature fusion operation. The shapes of T1, T2, and T3 are [H / 2*W / 2, H / 2, W / 2], where H / 2*W / 2, H / 2, and W / 2 represent the channel, height, and width, respectively.

[0043] To obtain an output that incorporates more useful global priors, F0 is concatenated with T1, T2, and T3 to obtain feature F. M ∈R (H / 2*W / 2)*3×H / 2×W / 2 Where H / 2, W / 2, and (H / 2*W / 2)*3 represent height, width, and channel. F M The formula is as follows:

[0044] F M =concat[F0,T1,T2,T3] (3)

[0045] F M The input features are reshaped into Y1 to generate attention weights. Then, an interpolation algorithm is used to adjust the attention weights Y1 into Y2, resulting in a space size identical to the input features. Next, a reshaping operation reshapes Y2 into a three-dimensional space of size [1, W, H]. Finally, the output is obtained by multiplying the input features by the sigmoid function.

[0046] Step 3: Train the target detection network model based on wide-area receptive field spatial attention using the training set images.

[0047] The training set images were uniformly 512×512 pixels, the learning rate was set to 0.001, the batch size was set to 4, the training was conducted for 12 epochs, and the learning rate was reduced to 1 / 10 of its original value at the 8th and 11th epochs.

[0048] Step 4: Use the network model trained in Step 3 to perform object detection on the test set images.

[0049] Experimental environment: A Python compilation environment based on PyTorch 1.6, torchvision=0.7.0, CUDA 10.0, and CUDNN 7.4 was set up and implemented on the mmdetection 2.6 platform.

[0050] Experimental equipment: CPU: Intel Xeon E5-2683 V3@2.00GHz; RAM: 16GB; Graphics card: Nvidia GTX 2060super; Hard disk: 500GB.

[0051] To test the impact of the MSA wide receptive field spatial attention structure on object detection accuracy, comparative experiments were conducted on multiple networks. The evaluation metric used in the experiments was Average Precision (AP), and AP was selected as the optimal metric. 50 AP 75 AP S AP M AP L As the main evaluation criterion, AP 50 AP 75 This refers to taking the detection results from detectors with IoU thresholds greater than 0.50 and greater than 0.75. AP S AP M AP L The detection accuracy for small, medium, and large targets are respectively shown in Table 1.

[0052] Table 1. Effects of MSA spatial attention on different networks

[0053]

[0054] Table 1 shows the detection performance of the MSA wide-area receptive field spatial attention structure on the COCO 2017 dataset. As can be seen from the table, the performance improvement for each network ranges from 0.7% to 0.9%. Since images in the COCO 2017 dataset often contain a large number of complex objects, the type, scale, and pose of the target are often uncertain, posing some challenges to detection. For example, ATSS and VFNet, after incorporating MSA spatial attention, perform slightly worse in large target detection than the original networks. Overall, the wide-area receptive field spatial attention mechanism effectively extracts important features.

[0055] Select some test images to test the final result. From Figure 3 As can be seen, the proposed object detection network achieves excellent results. In the third image, where there is only one bird, the network accurately detects the object. It also achieves good detection results in other images with multiple objects. For the fourth and fifth images, it can still accurately identify the category when some objects are occluded. Furthermore, it performs well in detecting small objects and blurred images, such as the boat in the fourth image and the herd of horses in the eighth image. Overall, the network proposed in this invention accurately completes the object detection task and has excellent edge recognition performance.

[0056] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A target detection method based on wide-area receptive field spatial attention, characterized in that, Includes the following steps: Step 1: Prepare the image dataset for testing and training; Step 2: Construct a target detection network based on wide receptive field spatial attention. This network consists of four parts: Backbone, Neck, Head, and MSA (wide receptive field spatial attention). The Backbone uses a ResNet50 backbone network to extract image features. The Neck structure is used to connect the Backbone and Head to fuse features. The Head is used for object detection, achieving target classification and regression. The MSA is placed between the Backbone and Neck, and between the Neck and Head. The structure of MSA is as follows: Let F∈R C×H×W The input tensor is given by C, H, and W, which represent the channel, height, and width, respectively. The height and width of F are halved using a 3×3 convolution to obtain the input tensor. ∈ R C×H / 2×W / 2 Then, each is processed through a regular convolution branch to obtain... ∈R 1×H / 2×W / 2 F1 ∈ R is obtained by combining three depthwise separable convolutional branches. C / 2×H / 2×W / 2 F2∈R C / 2×H / 2×W / 2 F3∈R C / 2×H / 2×W / 2 Then, F1, F2, and F3 are reshaped into M1, M2, and M3 by changing their dimensions, i.e., from three-dimensional to two-dimensional. （1） M1, M2, and M3 have the same matrix shape [H / 2*W / 2, C / 2], where H / 2*W / 2 and C / 2 represent the rows and columns of the matrix. Multiplying M1, M2, and M3 respectively yields three relation matrices N1, N2, and N3. Each value in the relation matrix represents the relationship between every two pixels in the feature. The formulas for calculating N1, N2, and N3 are as follows: （2） In the formula, Represents matrix multiplication. , , Let M1, M2, and M3 be the transpose matrices respectively, and let N1, N2, and N3 have the shape [H / 2*W / 2, H / 2*W / 2], where H / 2*W / 2 and H / 2*W / 2 represent the rows and columns of the matrix respectively; Reshape N1, N2, and N3 into T1, T2, and T3, with shapes [H / 2*W / 2, H / 2, W / 2], where H / 2*W / 2, H / 2, and W / 2 represent the channel, height, and width, respectively. To obtain an output that includes more useful global priors, The features are obtained by combining them with T1, T2, and T3. : （3） In the formula, ∈R (H / 2*W / 2)*3×H / 2×W / 2 H / 2, W / 2, (H / 2*W / 2)*3 represent height, width, and channel; Will The attention weights are reshaped into Y1 to generate attention weights. Then, an interpolation algorithm is used to adjust the attention weights Y1 into Y2, resulting in a space with the same size as the input feature. After that, Y2 is reshaped into a three-dimensional space with a size of [1, W, H] through a reshaping operation. Finally, the input feature is multiplied by the Sigmoid function to obtain the final output. Step 3: Train the target detection network model based on wide-area receptive field spatial attention using the training set images; Step 4: Use the network model trained in Step 3 to perform object detection on the test set images.

2. The target detection method based on wide-area receptive field spatial attention as described in claim 1, characterized in that: In step 1, all images are resized to 512×512 for multi-scale training. Data augmentation is then applied to the image dataset using a series of operations: random flipping, padding, random cropping, normalization, and image distortion correction.

3. The target detection method based on wide-area receptive field spatial attention as described in claim 1, characterized in that: In step 2, the ResNet50 backbone network outputs four feature maps of different sizes [C1, C2, C3, C4] with strides of [4, 8, 16, 32] and channel sizes of [256, 512, 1024, 2048]. The Neck structure uses three feature maps from the Backbone [C2, C3, C4]. After a 1×1 convolution, the channels are reduced to 256. The feature maps are then fused using [P1, P2, P3] in the FPN structure. P3 is then downsampled twice to obtain P4 and P5. Finally, a 3×3 convolution is used to ablate the feature maps, outputting five feature maps of different sizes with strides of [8, 16, 32, 64, 128] and channel sizes of 256.

4. The target detection method based on wide-area receptive field spatial attention as described in claim 1, characterized in that: In step 3, the training set image size is uniformly set to 512×512, the learning rate is set to 0.001, the batch size is set to 4, the training number is 12 epochs, and the learning rate is reduced to 1 / 10 of the original value in the 8th and 11th epochs.

Images