An object detection method based on efficient spatial pyramid pooling network

Abstract

The present invention discloses a target detection method based on an efficient spatial pyramid pooling network. The technical solution of the detection method is as follows: an efficient spatial pyramid pooling network is added to the target detection network; the image to be detected is input into the backbone network of the target detection network for feature extraction, the efficient spatial pyramid pooling network processes the extracted features at different scales, and then fuses the extracted multi-scale feature maps, integrating the high resolution of the extracted large-scale features and the rich semantic information of the extracted small-scale features for output, thereby improving the detection network's ability to detect targets of different sizes. The efficient spatial pyramid pooling network proposed in the present invention processes the features extracted by the backbone network of the target detection network and fuses the processed features of different scales. Under the premise of ensuring that the complexity of the detection network remains basically unchanged, it effectively improves the accuracy of the current detection network for multi-scale target detection.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a target detection method based on an efficient spatial pyramid pooling network. Background Art

[0002] Object detection involves detecting specific targets from videos or images. Object detection technology belongs to the field of computer vision and specifically refers to the process of analyzing and processing a target's visual features before detecting it. This technology has widespread applications in military, video surveillance, human-computer interaction, and other fields.

[0003] Due to its widespread application, object detection currently occupies a crucial position in the field of computer vision. In object detection algorithms, spatial pyramid pooling (SPP) uses three different pooling kernel sizes (5×5, 9×9, and 13×13) to extract features when fusing features of different scales. Furthermore, the SPP module employs parallel branching, performing three different pooling operations in parallel to generate three feature maps. This fails to fully exploit the reusability of pooling (for example, a pooling operation with a 9×9 kernel is equivalent to two serial pooling operations with 5×5 kernels). Furthermore, the SPP module uses constant pooling kernels (5×5, 9×9, and 13×13), failing to fully account for the scale variations of the input feature maps. This makes it impossible to dynamically adjust the kernel size when the input feature maps are too large or too small, thus reducing the SPP module's generalization. In addition, the pooling branch of the SPP module only uses the maximum pooling operation when extracting features. From the perspective of the features themselves, different features represent different information, and only using the global maximum pooling will greatly suppress the diversity of features and reduce the detection accuracy of the target. Summary of the Invention

[0004] The purpose of this invention is to solve the above-mentioned defects in the prior art and provide a target detection method based on an efficient spatial pyramid pooling network.

[0005] The purpose of the present invention can be achieved by taking the following technical solutions:

[0006] A target detection method based on an efficient spatial pyramid pooling network, the detection method comprising the following steps:

[0007] S1. Input the image to be detected into the backbone network of the target detection network for feature extraction to obtain the feature map F in ∈R C ×H×W , where C represents the number of feature map channels, H and W are the height and width of the feature map, and R represents the real number domain;

[0008] S2, efficient spatial pyramid pooling network for target detection network to feature map F in Extract at different scales and output and Where C1 is the number of channels after dimensionality reduction, and then the feature map F1, and Fusion is performed according to the channel dimension to obtain the output feature map Where C2 is the number of output channels. The efficient spatial pyramid pooling network integrates the high-resolution advantages of the extracted large-scale features with the rich semantic information of the extracted small-scale features, improving the detection network's ability to detect objects of varying scales.

[0009] S3, feature map The input is fed into the neck network of the target detection network for deep feature extraction, and then the output feature map is fed into the prediction part for target classification and target regression.

[0010] Furthermore, the input to output of the efficient spatial pyramid pooling network is a cascade-connected dimensionality reduction convolution layer, a first pooling layer, a second pooling layer, a third pooling layer and a dimensionality increase convolution layer.

[0011] Furthermore, the process of step S2 is as follows:

[0012] S21, the feature map F in ∈R C×H×W In the input dimensionality reduction convolution, the convolution kernel of the dimensionality reduction convolution is 1×1, the step size is 1, and the output feature map is Where C1 is the number of channels after dimensionality reduction. Dimensionality reduction convolution can compress F in The number of channels is increased to reduce the computational complexity of the first, second, and third pooling layers.

[0013] S22, input the feature map F1 into the first pooling layer to obtain the feature map

[0014] S23, feature map Input to the second pooling layer to obtain the feature map

[0015] S24, feature map Input to the third pooling layer to obtain the feature map

[0016] S25, four feature maps F1, and Splice by channel dimension to get feature map The high-resolution advantages of the extracted large-scale features and the rich semantic information of the extracted small-scale features are integrated and output to improve the detection network's ability to detect targets of different sizes.

[0017] S26, then F Concat Input dimensionality-raising convolution, the convolution kernel of dimensionality-raising convolution is 1×1, the step size is 1, and the output feature map is Where C2 is the number of channels after dimensionality increase. In order to adapt to the channel number requirement of the input feature map of the subsequent neck network, the dimensionality increase convolution is adjusted to Concat The number of channels is obtained

[0018] Furthermore, in step S2, the structures of the first pooling layer, the second pooling layer, and the third pooling layer are the same, and are all connected in parallel by a maximum pooling with a pooling kernel of kxk and an average pooling with a pooling kernel of kxk, wherein the step size of the maximum pooling and the average pooling is 1 and the padding is k / 2. The calculation formulas of the first pooling layer, the second pooling layer, and the third pooling layer are: Where F is the input of the pooling layer, F P is the output of the pooling layer, and represents maximum pooling and average pooling with a pooling kernel size of k×k. From the perspective of the features themselves, different features represent different information. Using maximum pooling and average pooling together will increase feature diversity and improve the performance of the object detection network.

[0019] Furthermore, the size of k is given by the formula Adaptive decision, |m| odd Represents the odd number closest to m, H and W are the length and width of the input feature map. The efficient spatial pyramid pooling network uses a pooling operation with a dynamically changing pooling kernel. The pooling kernel can change dynamically according to the length and width of the input feature map, thereby improving the generalization ability of the efficient spatial pyramid pooling network.

[0020] The present invention has the following advantages and effects compared to the prior art:

[0021] (1) The existing spatial pyramid pooling network only uses the maximum pooling operation to aggregate the spatial information of feature maps, which greatly suppresses the diversity of features. The present invention adds mean pooling features on the basis of the maximum pooling operation when aggregating the spatial information of feature maps to obtain more diverse and richer features.

[0022] (2) The purpose of the existing spatial pyramid pooling network is to fuse multi-scale features. When the length and width of the input feature map are too large or too small, the pooling operation with a fixed pooling kernel size reduces the ability to extract local features that are too large or too small. The present invention introduces a pooling operation with a dynamically changing pooling kernel. The pooling kernel can change dynamically according to the length and width of the input feature map, thereby improving the generalization ability of the spatial pyramid pooling module. Therefore, the present invention uses three cascaded pooling kernels of k×k (the size of k is given by the formula Adaptive decision, |m|odd represents the odd number closest to m, H and W are the length and width of the input feature map, and the size of k changes dynamically according to the size of H and W) pooling is used to extract features and reduce the amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0024] Figure 1 This is a schematic diagram of a target detection method based on an efficient spatial pyramid pooling network disclosed in the present invention.

[0025] Figure 2 It is a structural diagram of a YOLOv5 target detection method based on an efficient spatial pyramid pooling network according to an embodiment of the present invention.

[0026] Figure 3 2 is a structural diagram of the CSP1_X and CSP2_X modules according to an embodiment of the present invention.

[0027] Figure 4 It is a structural diagram of the YOLOv3 target detection method based on an efficient spatial pyramid pooling network in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0029] Example

[0030] This example uses a real blood cell dataset from various organs. The dataset contains three categories: white blood cells, red blood cells, and platelets. The blood cell dataset contains a total of 5,000 images.

[0031] The method comprises the following steps:

[0032] T1. Processing the data set: First, split the data set into a training set of 4,000 samples and a validation set of 1,000 samples.

[0033] T2. Construct a YOLOv5 target detection method embedded in an efficient spatial pyramid pooling network. The structure diagram is as follows Figure 3 , the specific structure is as follows:

[0034] Enter the image of size 640x640x3 into Figure 2 The network structure includes: Focus, CBL (Conv-Bn-Leaky ReLU, CBL), CSP1_X, CSP2_X and efficient spatial pyramid pooling modules. The Focus module mainly contains four parallel slicing layers to process the input image. The CBL module contains convolutional layers, batch normalization layers and activation functions. Figure 4 As shown, the CSP1_X module contains a CBL block and X residual connection units (Res Unit); the CSP2_X module contains X CBL blocks.

[0035] T21, the backbone network is mainly used to extract image features, including the Focus structure and the CSP structure. The Focus structure takes a 640×640×3 input image, first slices it and then uses a convolution operation, and finally outputs a 320×320×32 feature map. The purpose is to stack the four adjacent positions of the image and focus the information of the length and width dimensions into the channel space to improve the receptive field of each point and reduce the loss of original information. Two CSP structures are designed in YOLOv5, such as Figure 4 As shown in Figure 2, the module first divides the feature map of the base layer into two parts and then combines them through a hierarchical structure across stages, which reduces the amount of computation and ensures accuracy. Represents the splicing operation. The backbone network uses CSP1_X, and the neck network uses CSP2_X structure;

[0036] T22. After embedding the efficient spatial pyramid pooling network into the YOLOv5 backbone network, the backbone network output feature maps are extracted at different scales, and then the extracted multi-scale feature maps are fused to integrate the high resolution of the extracted large-scale features and the rich semantic information of the extracted small-scale features for output.

[0037] The neck network of T23 and YOLOv5 uses a feature pyramid network (FPN) and a path aggregation network (PAN) network structure. The FPN aggregates features from top to bottom, passing high-level semantic features to the lower layers, but does not pass on positioning information. The PAN adds a bottom-up feature pyramid after the FPN, passing high-level semantic positioning information from the lower layers to the upper layers, thereby supplementing the FPN.

[0038] T24, the prediction part includes the bounding box loss function and the non-maximum suppression (NMS) algorithm. YOLOv5 uses the GIOU (Generalized Intersection Over Union) loss function as the bounding box loss function. In the post-processing process of object detection, a weighted NMS method is used to screen many target boxes.

[0039] T3. Experimental Environment. This example experimental environment requires only general hardware configuration and a graphics processing unit (GPU) to accelerate computing. Model building, training, and testing of training results are all completed within the Pytorch deep learning framework, using the Compute Unified Device Architecture (CUDA), which enables GPUs to solve complex computational problems. The specific operating environment configuration required for this example experiment is shown in Table 1.

[0040] Table 1. Experimental operating environment parameters of this embodiment

[0041]

[0042] T4. Train the model. During YOLOv5 model training, when the loss value on the validation set stabilizes, the model has converged. To achieve optimal performance for the YOLOv5 model, the number of iterations was set to 300, the learning rate to 0.01, the learning rate momentum to 0.937, and the weight decay coefficient to 0.0005. Due to hardware configuration issues, the training batch size was set to 8. The model's loss function value on the validation set dropped sharply between 0 and 50 iterations, then slowly decreased between 50 and 200 iterations. After 200 iterations, the model's loss value stabilized, indicating that the model reached its optimal state.

[0043] T5. Test the model, input the validation set into the convolutional neural network, obtain the test results, and calculate the final mean average precision.

[0044] This embodiment achieved a mean average precision of 91.41% on a validation set of 1,000 red blood cells, which is 1.36% higher than the YOLOv5 object detection network without an embedded efficient spatial pyramid pooling network, further verifying the effectiveness of the present invention.

[0045] Example

[0046] This example uses a real cat and dog dataset, which contains two categories: cats and dogs. The cat and dog dataset has a total of 3680 images.

[0047] The method comprises the following steps:

[0048] T1. Processing the data set: First, split the data set into a training set of 3,000 samples and a validation set of 680 samples.

[0049] T2. Build a YOLOv3 target detection network based on an efficient spatial pyramid pooling network. The structure diagram is as follows Figure 4 , the specific structure is as follows:

[0050] T21, the backbone network of the YOLOV3 target detection network based on the efficient spatial pyramid pooling network includes a CBL module and several Res modules connected in sequence, such as Figure 2 As shown in the figure, the CBL module consists of a 1x1 convolution, a batch normalization layer, and a LeakyReLU. In this embodiment, the input image size is 608x608. After downsampling and feature fusion in the backbone network, feature maps of three scales, 76x76, 38x38, and 19x19, are output. The downsampling multiples are 8, 16, and 32, respectively.

[0051] The T22 and Res modules consist of two connected CBL modules. The addition of the Res module introduces residual connections, reducing the repetition of convolutional layers and thus reducing the difficulty of network training. While maintaining the same downsampling factor, the number of Res modules can be adjusted based on the image's feature complexity to balance the network's feature extraction capabilities with the number of network parameters. Specifically, for more complex images, increasing the number of Res modules can improve the network's feature extraction capabilities, while for simpler images, reducing the number of Res modules can reduce the number of network parameters and computational complexity.

[0052] T23. After embedding the efficient spatial pyramid pooling network into the YOLOv5 backbone network, the backbone network output feature maps are extracted at different scales, and then the extracted multi-scale feature maps are fused to integrate the high resolution of the extracted large-scale features and the rich semantic information of the extracted small-scale features for output.

[0053] T24, efficient spatial pyramid network connected with neck network.

[0054] T25, the prediction part includes the bounding box loss function and the non-maximum suppression (NMS) algorithm.

[0055] T3. Experimental Environment: This example uses a Linux server with standard hardware and a GPU to train a deep convolutional neural network. The YOLOv3 model is built using the Pytorch deep learning framework and CUDA, which enables GPUs to solve complex computational problems. Specifically, the hardware and software used in this example are: GeForce RTX 2080Ti GPU; Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz; CentOS 8.3.2011 operating system; Python 3.6.13; Pytorch deep learning framework; and CUDA 11.2.

[0056] T4. Training the model. During the training process on the validation set, YOLOv3 converges when its loss value reaches a stable state. To achieve optimal performance during training, this embodiment sets the number of iterations, learning rate, learning rate momentum, and weight decay coefficient to 300, 0.01, 0.937, and 0.0005, respectively. Due to hardware configuration, the training batch size is set to 8. YOLOv3's loss function decreases sharply within the iteration range of 0 to 50. The loss function decreases slowly from iteration 50 to iteration 200. After 200 iterations, the loss function stabilizes, and the YOLOv3 model reaches its optimal state.

[0057] T5. Test the model by inputting the validation set into the YOLOv3 target detection network based on the efficient pyramid pooling network, obtaining the detection results, and calculating the final mean average precision.

[0058] This embodiment achieves a mean average precision of 95.11% on a validation set of 680 cat and dog images, which is 3.22% higher than the YOLOv3 object detection algorithm without an embedded efficient pyramid pooling network, further verifying the effectiveness of the present invention.

[0059] In summary, the above embodiment discloses a target detection method based on an efficient spatial pyramid pooling network, which includes the following technologies: adding an efficient spatial pyramid pooling network to the detection network, processing the features extracted by the backbone network at different scales, and then fusing the extracted multi-scale feature maps, integrating the high resolution of the extracted large-scale features and the rich semantic information of the extracted small-scale features for output, thereby improving the detection network's ability to detect targets of different sizes. The serial spatial pyramid pooling network proposed in this invention processes the features extracted by the backbone network in the target detection algorithm and fuses the processed features of different scales. Under the premise of ensuring that the complexity of the detection network remains basically unchanged, the accuracy of the detection network for multi-scale target detection is effectively improved. The present invention has the characteristics of low computational cost and strong model learning ability.

[0060] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A target detection method based on an efficient spatial pyramid pooling network, characterized in that: The detection method comprises the following steps: S1. Input the image to be detected into the backbone network of the target detection network for feature extraction to obtain the feature map F in ∈R C×H×W , where C represents the number of feature map channels, H and W are the height and width of the feature map, and R represents the real number domain; S2, efficient spatial pyramid pooling network for target detection network to feature map F in Different scale extractions are performed. The input to output of the efficient spatial pyramid pooling network is a cascaded dimensionality reduction convolution layer, a first pooling layer, a second pooling layer, a third pooling layer, and a dimensionality increase convolution layer. The structures of the first pooling layer, the second pooling layer, and the third pooling layer are the same, and are all connected in parallel by a maximum pooling with a pooling kernel of kxk and an average pooling with a pooling kernel of kxk. The step size of the maximum pooling and the average pooling is 1, and the padding is k / 2. The calculation formulas of the first pooling layer, the second pooling layer, and the third pooling layer are as follows: Where F is the input of the pooling layer, F P is the output of the pooling layer, and Indicates maximum pooling and average pooling with a pooling kernel size of k×k; Output and Where C1 is the number of channels after dimensionality reduction, and then the feature map F1, and Fusion is performed according to the channel dimension to obtain the output feature map Where C2 is the number of output channels; the process is as follows: S21, the feature map F in ∈R C×H×W Input dimensionality reduction convolution layer, the convolution kernel of dimensionality reduction convolution layer is 1×1, the step size is 1, and the output feature map is Where C1 is the number of channels after dimensionality reduction; S22, input the feature map F1 into the first pooling layer to obtain the feature map S23, feature map Input to the second pooling layer to obtain the feature map S24, feature map Input to the third pooling layer to obtain the feature map S25, four feature maps F1, and Splice by channel dimension to get feature map S26, then F Concat Input dimensionality-raising convolution, the convolution kernel of dimensionality-raising convolution is 1×1, the step size is 1, and the output feature map is S3, feature map The input is fed into the neck network of the target detection network for deep feature extraction, and then the output feature map is fed into the prediction part for target classification and target regression.

2. A target detection method based on an efficient spatial pyramid pooling network according to claim 1, characterized in that The size of k is given by the formula Decision, |m| odd represents the odd number closest to m, H and W are the length and width of the input feature map.

Images