Small target detection method based on smooth interactive compression network

Abstract

The present invention discloses a small target detection method based on a smooth interactive compression network, and its implementation steps are as follows: constructing and training a smooth interactive compression network, utilizing a smooth interactive module in the network to robustly learn semantic information of image features within a multi-scale range, and utilizing a cross-attention submodule in the smooth interactive compression network to dynamically fuse feature maps of different resolutions according to the content of the input image. The white box pruning submodule in the smooth interactive compression network is utilized to reduce the complexity of the model without reducing the performance. The present invention mainly solves the problem of detecting small and weak targets in images with complex backgrounds under the premise of limited hardware computing power, and has the advantages of strong dependence between feature maps, good effect of dynamically fusing feature maps of different resolutions, low missed detection rate and false alarm rate, small network computing demand and simple structure, thus being easily adapted to existing hardware.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and further relates to a small target detection method based on a smooth interactive compression network in the field of image detection technology. The present invention uses a small target detection method based on a smooth interactive compression network to detect small targets in images. This method can be used to detect small targets in images with complex backgrounds, even when hardware computing power is limited. Background Art

[0002] In recent decades, due to the lack of publicly available infrared small target datasets, small target detection has been based on model-driven design methods. Due to the interference of noise within the image receiver and the long distance between the target to be detected and the detector, the target to be detected occupies a very small area in the image, with few available features, small scale, lack of texture, and easy loss of detailed information. Existing technologies cannot solve the problem of complex backgrounds, and the algorithm stability is poor. It is not suitable for detecting dark targets. For infrared images with dark targets and complex backgrounds, some strong clutter signals may be as sparse as the target signals, resulting in an increased false alarm rate. In addition, in existing deep learning methods, feature learning mainly relies on convolutional neural networks, whose localization weakens the ability to capture large-scale dependencies, which can easily lead to missed detections and false alarms. At the same time, the huge computational cost requirements of convolutional neural networks cause them to consume excessive computing resources.

[0003] East China University of Science and Technology (ECUST) has filed a patent application titled "A Method for Infrared Small Target Detection Based on Attention-Guided Pyramid Fusion" (Patent Application No. 202211539561.X, Publication No. CN 116071676A). This method implements the following steps: First, a training and test dataset of infrared images is acquired and preprocessed: the infrared images are cropped to a uniform size and pixel normalized. Second, a neural network based on an encoder-decoder architecture is constructed and the preprocessed infrared images are input into the neural network. Third, the output images of each layer of the neural network encoder are fed into a multi-scale fusion module to enrich target information. The output image of the multi-scale fusion module is then concatenated with the output image of the decoder at the same level and passed through an attention mechanism module to amplify key information. Fourth, the output image of the neural network encoder is fed into a context pyramid module to associate the image with contextual information. The resulting output image is then fed into the decoder. In the fifth step, the difference between the detected image and the true labeled image is used as the loss function, and the network parameters are iteratively trained to optimize. After the loss stabilizes, the images in the test dataset are input into the trained neural network, and the decoder output image is the final detection result. The disadvantage of this method is that it is limited by the locality of the convolution kernel when encoding long-distance related features, which weakens the network's ability to capture the shape and position of the target in complex scenes, and is prone to missed detections and false alarms. Due to its limited ability to extract single-layer features, the context pyramid module cannot handle complex background clutter, resulting in a low target signal-to-noise ratio and a high false alarm rate in the detection results. At the same time, the complex structure of the neural network and the huge computational cost require it to consume excessive computing resources.

[0004] Zhejiang University has disclosed a method for detecting small spatial targets based on deep learning semantic feature separation in its patent application, "Spatial Small Target Detection Method Based on Deep Learning Semantic Feature Separation" (patent application number: 202211656664.4, publication number CN 116206219 A). The method implements the following steps: First, preprocess the input image to eliminate noise interference and improve the learning ability of small targets. Second, a neural network is used to process the target image and extract semantic and texture information through deconvolution. Third, a detector is used to generate a probability map of the target area. This probability map is then filtered to process the target semantic and texture features, eliminating background interference, preserving the target foreground features, and sparsifying the target feature representation space. Fourth, a Transformer network is used to query and process the target semantic and texture feature information, extracting correlations between target features and generating multiple sets of target prediction boxes. In the fifth step, the Hungarian algorithm is used to dynamically match the predicted box information with the ground-truth box information. By minimizing the loss function, an optimal dynamic matching mechanism between the predicted and ground-truth boxes is achieved, enabling target prediction without relying on prior information. However, this method has drawbacks. Because it uses probabilistic graph filtering to extract small target features, it can only suppress uniform backgrounds to a certain extent and cannot address complex backgrounds. Furthermore, the correlation between feature maps is poor, and the ability to capture large-scale dependencies is limited. This method utilizes the Transformer network structure, which is complex, has numerous parameters, and suffers from severe feature map attenuation. This makes it easy for feature information of small targets to be overwhelmed by surrounding background features deep within the network. The algorithm also suffers from poor stability and high computational complexity, making it difficult to be compatible with existing hardware. Summary of the Invention

[0005] The purpose of the present invention is to address the deficiencies in the above-mentioned prior art and propose a small target detection method based on a smooth interactive compression network to solve the problem that the filter cannot process complex background clutter, resulting in a low target signal-to-noise ratio in the detection result and an inability to resolve complex backgrounds; the convolutional neural network has difficulty in encoding long-distance related features, the fusion effect of different features is poor, and it is very easy to produce missed detections and false alarms; the neural network structure is complex and has many parameters, resulting in a huge amount of computational demand for small target detection in images and difficulty in adapting to existing hardware.

[0006] The technical idea for achieving the purpose of the present invention is that the smooth interaction module constructed by the present invention uses sparse sampling features obtained by using hole convolution kernels with different expansion rates, and then uses the normalized attention coefficient of the sparse sampling features to update the central node, which increases the dependency between feature maps, effectively solves the defect that the filter of the prior art cannot handle complex background clutter, and improves the signal-to-noise ratio of the target in the detection result. The cross-attention submodule constructed by the present invention splices two feature maps along the channel dimension, and performs global average pooling to obtain one-dimensional context features, and sends the one-dimensional context features to the dense perceptron to obtain normalized fusion weights, effectively solving the defect that the convolutional neural network of the prior art is difficult to encode long-distance related features and has poor fusion effect of different features, thereby reducing the missed detection rate and false alarm rate of small target detection in the image. The white box pruning submodule constructed by the present invention replaces the convolution kernel of the complex network with three parallel convolution kernels, and converts the operation equivalently into a series of one-dimensional convolutions. All convolution kernels are shared in the same sliding window, and can cover different significant areas in different environments. While enhancing the skeleton, it reduces the number of parameters and computational complexity, compresses the convolutional neural network, and effectively solves the defects of the existing neural network structure with complex parameters. It greatly reduces the computational complexity required for detecting small targets in images and is easy to adapt to existing hardware.

[0007] To achieve the above object, the specific implementation steps of the present invention include the following:

[0008] Step 1, build cross-attention submodule:

[0009] Build a main branch consisting of a connection layer, a first fully connected layer, a first activation layer, a second fully connected layer, a second activation layer, a weight summing unit, and an output layer in series. The first input layer is connected between the connection layer and the weight summing unit of the main branch module, and the second input layer is connected between the connection layer and the weight summing unit of the main branch to form a cross-attention submodule. Set the channel parameters of the first and second input layers to 32 and 16 respectively. Use the ReLU function and the Softmax function as the activation functions of the first and second activation layers, respectively.

[0010] Step 2: Build the white-box pruning submodule:

[0011] Build a white-box pruning submodule consisting of an input layer, a convolution group, a convolution layer, and an output layer in series. Set the channel parameter of the input layer to 32. Set the convolution kernel size in the convolution layer to 3×3 and the sliding step size to 1. Set the channel parameter of the output layer to 16.

[0012] Step 3: Build the cross-compression module:

[0013] Build a main branch consisting of the second input layer, the white-box pruning submodule, the cross-attention submodule, and the output layer in series. The first input layer is also connected to the cross-attention submodule in the main branch to form a cross-compression module. Set the channel parameters of the first and second input layers to 32 and 16 respectively; set the channel parameter of the input layer to 1.

[0014] Step 4: Build a smooth interactive compression network:

[0015] Build a main branch consisting of the input layer, the first convolution group, the second convolution group, the third convolution group, the second smoothing interaction module, the deconvolution group, the cross-compression module, and the output layer in series. The first smoothing interaction module is connected between the second convolution group and the deconvolution group in the main branch, and the first convolution group in the main branch is also directly connected to the cross-compression module.

[0016] The structures of the first and second smooth interaction modules are the same, and are both composed of a smooth input layer, a hole convolution group, a smoothing processing layer, a convolution layer, and a smooth output layer connected in series in sequence; the size of the convolution kernel in the convolution layer of the first and second smooth interaction modules is set to 1×1, and the sliding step size is set to 1; the channel parameters of the smooth input layer in the first and second smooth interaction modules are set to 32 and 64 respectively; the channel parameters of the smooth output layer in the first and second smooth interaction modules are set to 32 and 64 respectively;

[0017] Step 5: Generate training set:

[0018] Select at least 800 images to form a sample set, input each image in the sample set into the torchvision module for normalization, and form all the normalized samples into a training set;

[0019] Step 6: Train the smooth interactive compression network:

[0020] The training set is input into the smooth interactive compression network, and the network parameters are iteratively updated by the AdaGrad optimizer through the stochastic gradient descent method until the network's intersection-over-union loss function converges, thus obtaining a trained smooth interactive compression network.

[0021] Step 7: Detect small targets:

[0022] The normalized image of the small target to be detected is input into the trained smooth interactive compression network; the area composed of multiple pixels with a pixel value of 1 in the image output by the network is regarded as the detected small target area, and the pixel value of 0 represents the area where the small target is not detected.

[0023] Compared with the prior art, the present invention has the following advantages:

[0024] First, the smooth interaction module constructed in this invention expands the receptive field without increasing the computational effort, while also increasing the dependencies between feature maps. This overcomes the drawback of existing filters that cannot handle complex background clutter, allowing this invention to improve the signal-to-noise ratio of targets in detection results.

[0025] Second, the cross-attention submodule within the cross-compression module designed by the present invention adjusts the contribution of inputs of different resolutions based on the content of the input image, dynamically fusing feature maps of different resolutions. This overcomes the difficulties of existing convolutional neural networks in encoding long-range correlated features and the poor fusion of different features. This significantly reduces the missed detection rate and false alarm rate of small object detection in images, thereby improving the detection accuracy of small objects.

[0026] Third, the white-box pruning submodule within the cross-compression module designed by this invention compresses convolutional neural networks while increasing the interpretability of channel pruning, reducing model complexity without sacrificing performance. This overcomes the drawbacks of existing neural network architectures, which often involve complex and numerous parameters. This approach minimizes the computational requirements for small object detection in images, while maintaining a simple network structure that is easily implemented and compatible with existing hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Flowchart for the implementation of the present invention;

[0028] Figure 2 Schematic diagram of the structure of the smooth interaction module of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of the cross-focus submodule of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of the white box pruning submodule of the present invention;

[0031] Figure 5 This is a schematic structural diagram of the cross-compression module of the present invention;

[0032] Figure 6 Schematic diagram of the structure of the smooth interactive compression network of the present invention;

[0033] Figure 7 is a schematic diagram of an image to be measured in an embodiment of the present invention;

[0034] Figure 8 Schematic diagram of the actual position and shape of a small target in an image to be measured according to an embodiment of the present invention;

[0035] Figure 9 Schematic diagram of the position and shape of a small target detected in an image to be detected in an embodiment of the present invention;

[0036] Figure 10 This is a simulation result diagram of simulation experiment 1 of the present invention;

[0037] Figure 11 This is a simulation result diagram of simulation experiment 2 of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described below with reference to the accompanying drawings and embodiments.

[0039] Reference Figure 1 , further describing the implementation steps of the embodiment of the present invention.

[0040] Step 1: Build a smooth interaction module.

[0041] Reference Figure 2 , the structure of the smooth interaction module constructed by the present invention is further described.

[0042] In step 1.1, build a smooth interaction module consisting of an input layer, a dilated convolution group, a smoothing layer, a convolution layer, and an output layer in series. The dilated convolution group consists of the first, second, third, fourth, and fifth dilated convolution layers in parallel.

[0043] In step 1.2, the convolution kernel size in the convolution layer is set to 1×1, and the sliding step is set to 1; the expansion rates in the first to fifth dilated convolution layers are set to 0, 1, 2, 3, and 4, respectively, and the sliding step is set to 1.

[0044] Step 2: Construct the cross-attention submodule.

[0045] Reference Figure 3 , further describing the structure of the cross-focus submodule constructed by the present invention.

[0046] Step 2.1: Build a cross-attention submodule that includes the first input layer, the second input layer, the connection layer, the first fully connected layer, the first activation layer, the second fully connected layer, the second activation layer, the weighted summing unit, and the output layer. The first input layer, the connection layer, the first fully connected layer, the first activation layer, the second fully connected layer, the second activation layer, the weighted summing unit, and the output layer are connected in series. The first input layer is also connected to the weighted summing unit, and the second input layer is also connected to the connection layer and the weighted summing unit.

[0047] In step 2.2, the channel parameters of the first and second input layers are set to 32 and 16 respectively; the activation function of the first activation layer adopts the ReLU function, and the activation function of the second activation layer adopts the Softmax function.

[0048] Step 3: Build the white-box pruning submodule.

[0049] Reference Figure 4 , the structure of the white box pruning submodule constructed by the present invention is further described.

[0050] In step 3.1, build a white-box pruning submodule consisting of an input layer, a convolutional group, a convolutional layer, and an output layer in series. The convolutional group consists of the first convolutional layer, the second convolutional layer, and the third convolutional layer in parallel.

[0051] In step 3.2, set the channel parameter of the input layer to 32; set the convolution kernel sizes in the first to third convolution layers to 3×1, 1×3, and 3×1, respectively, and set the sliding step size to 1; set the convolution kernel size in the convolution layer to 3×3, and set the sliding step size to 1; set the channel parameter of the output layer to 16.

[0052] Step 4: Build a cross-compression module.

[0053] Reference Figure 5 , the structure of the white box pruning submodule constructed by the present invention is further described.

[0054] Step 4.1: Build a cross-compression module including the first input layer, the second input layer, the white-box pruning submodule, the cross-attention submodule, and the output layer. The second input layer, the white-box pruning submodule, the cross-attention submodule, and the output layer are connected in series, and the first input layer is also connected to the cross-attention submodule.

[0055] In step 4.2, set the channel parameters of the first and second input layers to 32 and 16 respectively; set the channel parameter of the input layer to 1.

[0056] Step 5: Build a smooth interactive compression network.

[0057] Reference Figure 6 , further describing the structure of the smooth interactive compression network constructed by the present invention.

[0058] Build a smooth interactive compression network including input layer, first convolution group, second convolution group, third convolution group, deconvolution group, first smooth interaction module, second smooth interaction module, cross compression module, and output layer.

[0059] Among them, the input layer, the first convolution group, the second convolution group, the third convolution group, the second smooth interaction module, the deconvolution group, the cross compression module, and the output layer are connected in series in sequence; the first convolution group is also connected to the cross compression module, the second convolution group is also connected to the first smooth interaction module, and the first smooth interaction module is also connected to the deconvolution group.

[0060] The first to third convolution groups are composed of a first convolution layer, a first pooling layer, a second convolution layer, and a second pooling layer with the same structure, which are connected in series in sequence.

[0061] The convolution kernel size of the first and second convolutional layers is set to 3×3, the sliding step size is set to 1, and the output channels of the first to third residual convolution groups are set to 16, 32, and 64 respectively; the activation function of the first and second pooling layers is the ReLU function.

[0062] The deconvolution group is composed of a first deconvolution layer, a first pooling layer, a second deconvolution layer, and a second pooling layer connected in series.

[0063] The convolution kernel size of the first and second deconvolution layers is set to 3×3, the sliding step size is set to 1, the activation function of the first and second pooling layers uses the ReLU function, and the output channels of the deconvolution group are set to 32.

[0064] Step 6: Generate training set and test set.

[0065] In step 6.1, the embodiment of the present invention selects 1001 images from the public IRSTD-1K dataset to form a sample set, 80% of the images in the sample set form a training sample set, and 20% of the images form a test sample set.

[0066] In step 6.2, each image in the training sample set is input into the torchvision module for normalization. All normalized training samples form the training set. The same method is used for the test sample set to obtain the test set.

[0067] Step 7: Train the smooth interactive compression network.

[0068] The training set is input into the smooth interactive compression network. For the training set, the cross entropy loss function based on knowledge distillation is used, AdaGrad is used as the optimizer, the learning rate is 0.06, the training process includes 2000 epochs, and the weight decay is 10 -4 , the batch size is 12. After the selected images are input into the smooth interactive compression network, the loss value of the smooth interactive compression network is calculated, and the gradient optimization algorithm of the stochastic gradient descent method is used to optimize the network parameters. The weight value of the smooth interactive compression network is iteratively updated until the network's intersection-over-union loss function converges, and a trained smooth interactive compression network is obtained.

[0069] The loss function is as follows:

[0070]

[0071] Among them, L SRepresents the image input to the smooth interactive compression network. After one round of iteration, the network loss value outputted, mean(·) represents the averaging operation, γ represents the weight factor with a real value in the range of [0,1], τ represents the loss weight factor with a real value in the range of [0,1], H and W represent the height and width of the output image of the smooth interactive compression network, m and n represent the pixel indices of the output image of the smooth interactive compression network along the height dimension and width dimension, respectively, |·| represents the absolute value operation, represents the probability that the pixel in the mth row and nth column of the image output by the smooth interactive compression network output layer belongs to the real target, r mn Represents the category of the pixel at row m and column n in the input image, Represents the pixel at row m and column n in the output image of the first convolutional group in the smooth interactive compression network.

[0072] Step 8: Detect small targets.

[0073] The test set is fed into the trained smooth interactive compression network to detect small objects in the image. The network outputs an image where the pixel values ​​are 1 and the pixel values ​​are 0, indicating that no small objects have been detected.

[0074] The following is a combination of the embodiments of the present invention and Figure 7 、 8 , 9, further describe the present invention.

[0075] Figure 7 In the embodiment of the present invention, an original image without any preprocessing is randomly selected from the test sample set. The small target to be detected in the image is located under a complex cloud layer and occupies very few pixels, making it difficult to distinguish from the complex background. After preprocessing, the image is input into the smooth interactive compression network constructed and trained by the present invention. The small target detection result output by the network is as follows: Figure 8 shown. Figure 8 The weak white pixel on the left side shows the detected Figure 7 The location of small and weak targets, Figure 8 The shape of the detected weak target is shown in the gray box in the lower right corner.

[0076] Figure 9 The test samples of the present invention are concentrated with Figure 7 The true position and true shape of the small target corresponding to the image to be measured. Figure 9 The white weak pixel on the left is the real position information of the weak target in the image to be tested, and the white box in the lower right corner shows the real shape of the weak target.

[0077] Will Figure 8 and Figure 9 By comparison, it is obvious that the position and shape information of the small target detected by the method of the present invention are highly similar to the real information, indicating that the smooth interactive compression network constructed and trained by the present invention has obvious effects in suppressing background and enhancing targets, the difference layer generation module extracts small target features accurately, and the fourth-order Adams guidance module fuses features of each layer well, which greatly improves the accuracy of small target detection.

[0078] The effect of the present invention can be further demonstrated through the following simulation.

[0079] 1. Simulation experiment conditions.

[0080] The software platform of the simulation experiment of the present invention adopts the Linux operating system and Pycharm2021.1 professional edition, and the hardware platform adopts NVIDIA RTX A2000 GPU.

[0081] 2. Simulation content and result analysis.

[0082] The simulation experiment 1 of the present invention is to randomly select four pictures from the test sample set, and after preprocessing, input them into the trained network of the present invention and five trained networks disclosed in the prior art (PSTNN, IPI, MDvsFA, ACM network and ALC network) respectively to perform weak target detection, and obtain six groups of 24 weak target detection result pictures. The four pictures randomly selected from the test set, the 24 prediction result pictures and the four real small target pictures are compared, and the 8 groups of 32 pictures in total are obtained as follows Figure 10 The pixels in the red box in each image show the position and shape of the small target, the pixels in the blue circle represent the small target pixels that were missed, and the pixels in the yellow circle represent the small target pixels that were falsely reported.

[0083] Test results such as Figure 10 As shown, from Figure 10It can be seen from the figure that each of the four input images contains a small target to be detected. For the PSTNN method, there are two missed detections and two false alarms in its detection results, and both missed detections and false alarms occur in the detection results of picture 2; for the IPI method, there are two missed detections and two false alarms in its detection results, and there are many missed pixels; for the MDvsFA method, there is one missed detection and three false alarms in its detection results, and the three false alarms appear in the same picture, indicating that the method has poor robustness in detecting small targets; for the ACM network, there is one missed detection and three false alarms in its detection results; for the ALC network, there is one missed detection and one false alarm in its detection results; for the method of the present invention, there are no missed detections or false alarms in its detection results, and the position and size of the small target in the network output image are very close to the actual small target image, retaining the details and contour information of the small target, and the detection result is significantly better than the existing technology;

[0084] The prior art used in the simulation experiment of the present invention refers to:

[0085] PSTNN refers to the PSTNN method proposed by Hong Zhang et al. in their paper "Infrared Small Target Detection Based on Partial Sum of the Tensor Nuclear Norm" (Remote Sensing for Target Object Detection and Identification, 13 February 2019).

[0086] IPI refers to the IPI method proposed by Landan Zhang et al. in their paper "Infrared Patch-Image Model for Small Target Detection in a Single Image" (IEEE Transactions on ImageProcessing Volume: 22, Issue: 12, December 2013 4996-5009).

[0087] MDvsFA refers to the MDvsFA method proposed by Huan Wang et al. in their paper "Miss Detection vs. False Alarm: Adversarial Learning for Small Object Segmentation in Infrared Images" (Proceedings of the IEEE / CVF International Conference on Computer Vision (ICCV), 2019, pp. 8509-8518), a small target detection method.

[0088] ACM network refers to the ACM network proposed by Yimian Dai et al. in their paper "Asymmetric Contextual Modulation for Infrared Small Target Detection" (Proceedings of the IEEE / CVF Winter Conference on Applications of Computer Vision (WACV), 2021, pp.950-959) in a weak target detection method.

[0089] The ALC network refers to the ALC network proposed in the weak target detection method proposed by Yimian Dai et al. in their paper "Attentional Local Contrast Networks for Infrared Small Target Detection" (IEEE Transactions on Geoscience and Remote Sensing, 05 January 2021, 9813-9824).

[0090] Simulation experiment 2 of the present invention is to input the test set into the trained network of the present invention and four trained networks disclosed in the prior art (MDvsFA, SK network, ACM network, ALC network) to perform weak target detection.

[0091] The prior art used in the simulation experiment 2 of the present invention refers to:

[0092] The SK network refers to the SK network method proposed by Xiang Li et al. in their paper "Selective Kernel Networks" (Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 510-519) in a small target detection method.

[0093] The simulation experiment of the present invention uses the intersection over union (IoU), normalized intersection over union (nIoU), receiver operating characteristic (ROC) curve, and detection rate P d , false alarm rate F a The existing weak target detection methods are evaluated by the floating point operation number FLOPs evaluation index. IoU and nIoU are defined as:

[0094]

[0095]

[0096] Where T, P, and TP represent the number of true pixels, correctly predicted pixels, and pixels predicted to be true. N represents the total number of image samples in the training set, and i represents the index of an image sample in the training set. Larger values ​​of IoU and nIoU indicate better detection performance.

[0097] True positive rate (P d ) represents the proportion of true values ​​predicted to be correct to the total true values, and the false positive rate (F a ) represents the proportion of false values ​​predicted to be correct to the total false values:

[0098]

[0099]

[0100] Among them, FP, TN, and FN represent the number of pixels predicted to be correct and false, the number of pixels predicted to be false and false, and the number of pixels predicted to be false and true, respectively. The larger the true positive rate of a network test, the better the detection effect, and the smaller the false positive rate, the better the detection effect. The ROC curve describes the P d With F a For the ROC indicator, the ROC function graph of a network increases with the false positive rate. The higher the true positive rate, the better the detection result of the network. For the FLOPs indicator, the smaller the FLOPs value of a network, the simpler the network structure.

[0101] The test set is input into the trained network of the present invention and the four trained networks disclosed in the prior art respectively, and the obtained output weak target detection result pictures are processed to obtain IoU, nIoU, P d 、F a The evaluation indicators of P and FLOPs are shown in Table 1. d 、F a Draw five ROC curves as follows Figure 11 shown.

[0102] Table 1: Comparison table of evaluation indicators

[0103]

[0104] It can be seen from Table 1 that compared with the four existing technologies, the IoU, nIoU, and P obtained by the network test proposed by the present invention are d Both are maximum, F a The FLOPs and FLOPs are the smallest, and the five evaluation indicators are better than the existing technology, which shows that compared with the existing technology, the network built by the present invention not only has the simplest structure, but also has the best effect in detecting small targets. Figure 11 It can be seen that the true positive rate of the ROC function graph corresponding to the network of the present invention increases fastest with the increase of the false positive rate, and is higher than the ROC function graph of the prior art, which proves that the network proposed in the present invention has the best effect in detecting weak targets. In summary, the present invention is superior to the model of the prior art.

Claims

1. A small target detection method based on a smooth interactive compression network, characterized in that: The cross attention submodule, the white box pruning submodule, the first and second smooth interaction modules of the cross compression module in the smooth interactive compression network are respectively constructed; the steps of the detection method include the following: Step 1, build cross-attention submodule: Build a main branch consisting of a connection layer, a first fully connected layer, a first activation layer, a second fully connected layer, a second activation layer, a weight summing unit, and an output layer in series. The first input layer is connected between the connection layer and the weight summing unit of the main branch module, and the second input layer is connected between the connection layer and the weight summing unit of the main branch to form a cross-attention submodule. Set the channel parameters of the first and second input layers to 32 and 16 respectively. Use the ReLU function and the Softmax function as the activation functions of the first and second activation layers, respectively. Step 2: Build the white-box pruning submodule: Build a white-box pruning submodule consisting of an input layer, a convolution group, a convolution layer, and an output layer in series. Set the channel parameter of the input layer to 32. Set the convolution kernel size in the convolution layer to 3×3 and the sliding step size to 1. Set the channel parameter of the output layer to 16. Step 3: Build the cross-compression module: Build a main branch consisting of the second input layer, the white-box pruning submodule, the cross-attention submodule, and the output layer in series. The first input layer is also connected to the cross-attention submodule in the main branch to form a cross-compression module. Set the channel parameters of the first and second input layers to 32 and 16 respectively; set the channel parameter of the input layer to 1. Step 4: Build a smooth interactive compression network: Build a main branch consisting of the input layer, the first convolution group, the second convolution group, the third convolution group, the second smoothing interaction module, the deconvolution group, the cross-compression module, and the output layer in series. The first smoothing interaction module is connected between the second convolution group and the deconvolution group in the main branch, and the first convolution group in the main branch is also directly connected to the cross-compression module. The structures of the first and second smooth interaction modules are the same, and are both composed of a smooth input layer, a hole convolution group, a smoothing processing layer, a convolution layer, and a smooth output layer connected in series in sequence; the size of the convolution kernel in the convolution layer of the first and second smooth interaction modules is set to 1×1, and the sliding step size is set to 1; the channel parameters of the smooth input layer in the first and second smooth interaction modules are set to 32 and 64 respectively; the channel parameters of the smooth output layer in the first and second smooth interaction modules are set to 32 and 64 respectively; Step 5: Generate training set: Select at least 800 images to form a sample set, input each image in the sample set into the torchvision module for normalization, and form all the normalized samples into a training set; Step 6: Train the smooth interactive compression network: The training set is input into the smooth interactive compression network, and the network parameters are iteratively updated by the AdaGrad optimizer through the stochastic gradient descent method until the network's intersection-over-union loss function converges, thus obtaining a trained smooth interactive compression network. Step 7: Detect small targets: The normalized image of the small target to be detected is input into the trained smooth interactive compression network; The area composed of multiple pixels with a pixel value of 1 in the image output by the network is regarded as the detected small target area, and the pixel value of 0 represents the area where the small target is not detected.

2. The small target detection method based on smooth interactive compression network according to claim 1 is characterized in that: The weight summation unit described in step 1 performs the weight summation operation as follows: P T =Y·W1·P+Y·W2·P′ Among them, P T represents the normalized fusion weight matrix output by the weight summation unit, Y represents the feature coefficient output by the second activation layer in the cross-attention submodule, which is a matrix with 1 row and 2 columns, W1 and W2 represent the weight fusion coefficients, both of which are matrices with 2 rows and 1 column, and P′ and P represent the input image matrices of the first and second input layers in the cross-attention submodule, respectively.

3. The small target detection method based on smooth interactive compression network according to claim 1, characterized in that: The convolution group described in step 2 consists of the first convolutional layer, the second convolutional layer, and the third convolutional layer in parallel; the convolution kernel sizes in the first to third convolutional layers are set to 3×1, 1×3, and 3×1, respectively, and the sliding step size is set to 1.

4. The small target detection method based on smooth interactive compression network according to claim 1, characterized in that: The smoothing layer described in step 4 is smoothed by the following formula: in, represents the jth pixel in the fusion feature image output by the smoothing layer, ∑ represents the summation operation, Softmax(·) represents the Softmax function, α 2 Indicates the size of the smoothing operation window, n i represents the i-th directed edge in the smoothing operation window, C represents the directed edge located in the center of the smoothing operation window, and n i and C are both d-dimensional vectors, where the value of d is equal to the number of input channels of the smooth input layer in the smooth interactive compression network. The superscript T represents the transposition operation, and W q and W k Both represent d k A matrix with d rows and d columns, d k is a parameter with an integer value in the range [1,10], W q and W k Each element in the matrix is ​​an attention fusion coefficient with a real value in the range of [-1,1], W v Represented as d o A matrix with d rows and d columns, d o The value of W is equal to the number of output channels of the smooth output layer in the smooth interactive compression network. v Each element in the matrix is ​​an attention output coefficient with a real value in the range of [-1,1].

5. The small target detection method based on smooth interactive compression network according to claim 1, characterized in that: The dilated convolution group described in step 5 is composed of the first dilated convolution layer, the second dilated convolution layer, the third dilated convolution layer, the fourth dilated convolution layer, and the fifth dilated convolution layer in parallel; the expansion rates in the first to fifth dilated convolution layers are set to 0, 1, 2, 3, and 4, respectively, and the sliding step sizes are all set to 1.

6. The small target detection method based on smooth interactive compression network according to claim 1, characterized in that: The structures of the first to third convolution groups described in step 5 are the same, consisting of the first convolution layer, the first pooling layer, the second convolution layer, and the second pooling layer connected in series in sequence; the convolution kernel size of the first and second convolution layers is set to 3×3, and the sliding step size is set to 1; the activation function of the first and second pooling layers uses the ReLU function; the output channels of the first to third residual convolution groups are set to 16, 32, and 64, respectively.

7. The small target detection method based on smooth interactive compression network according to claim 1, characterized in that: The deconvolution group described in step 5 is composed of the first deconvolution layer, the first pooling layer, the second deconvolution layer, and the second pooling layer connected in series in sequence; the convolution kernel sizes of the first and second deconvolution layers are both set to 3×3, the sliding step sizes are both set to 1, the activation functions of the first and second pooling layers are both ReLU functions, and the output channels of the deconvolution group are set to 32.

8. The small target detection method based on smooth interactive compression network according to claim 1, characterized in that: The loss function described in step 6 is as follows: Among them, L S Represents the image input to the smooth interactive compression network. After one round of iteration, the network loss value outputted, mean(·) represents the averaging operation, γ represents the weight factor with a real value in the range of [0,1], τ represents the loss weight factor with a real value in the range of [0,1], H and W represent the height and width of the output image of the smooth interactive compression network, m and n represent the pixel indices of the output image of the smooth interactive compression network along the height dimension and width dimension, respectively, |·| represents the absolute value operation, represents the probability that the pixel in the mth row and nth column of the image output by the smooth interactive compression network output layer belongs to the real target, r mn Represents the category of the pixel at row m and column n in the input image, Represents the pixel at row m and column n in the output image of the first convolutional group in the smooth interactive compression network.

Images