A method for small target detection in super-resolution images based on deep learning

Abstract

The present invention belongs to the field of image detection based on artificial intelligence, and proposes a method for detecting small targets in super-resolution images based on deep learning. The detection of small targets in super-resolution images is achieved by optimizing the yolov8 image detection model. By preprocessing the input image and reconstructing the super-resolution image, the problem of slow model detection speed and increased calculation caused by the large size of the super-resolution image is solved. Secondly, as the network deepens, by fusing the high-dimensional and low-dimensional features of the feature extraction layer of the model, the feature information of small target objects extracted by the model is not easily lost, thereby improving the accuracy of small target detection.

Description

Technical Field

[0001] The present invention relates to the field of image detection based on artificial intelligence, and in particular to a method for detecting small targets in super-resolution images based on deep learning. Background Art

[0002] The object detection algorithm based on deep learning convolutional neural network can detect large and medium objects well, but small objects occupy fewer pixels in the image and have lower resolution, making them difficult to detect, especially small objects in super-resolution images. Moreover, as the network deepens and the image becomes more abstract, small objects in the high-level feature layer occupy fewer pixels, making it difficult to extract information about small objects, and their average detection accuracy is much lower than that of large objects.

[0003] The existing small target detection is generally to optimize the mainstream target detection algorithm, such as enhancing small target samples, optimizing training methods, anchor-free mechanism and feature fusion methods. The FSSD model adds a lightweight feature fusion module to the SSD algorithm, uses bilinear interpolation to convert feature maps to the same scale, and fuses them to generate a new feature pyramid, making full use of feature information at each layer to improve the small target detection rate. In view of the problem that the coupling of features of different scales in the feature pyramid (Feature Pyramid Networks, FPN) will affect the performance of small target detection, an extended feature pyramid network (Extended Feature Pyramid Network, EFPN) model is proposed. The feature texture transfer module fuses semantic content and regional texture features, and extracts reliable regional details for small target detection to improve the accuracy. Based on the semantic segmentation task, a deep learning-based weak small target detection method is proposed. The fully convolutional recursive network is used to learn the features of weak small targets in complex backgrounds, and residual learning and recursive operations are used in the network to optimize network performance. Although the above methods have improved the detection accuracy of small targets to a certain extent, there are still the following problems for small target detection in super-resolution images:

[0004] (1) The pixels of super-resolution images are generally 3000*3000 or above. When the image is scaled to the normal image size for detection, the pixel information of small targets will be less, making it more difficult for the model to extract its feature information and detect small target objects.

[0005] (2) In order to obtain more information about small target objects, the target detection model is directly used to detect super-resolution images. This increases the number of model parameters, the required video memory, and the model detection speed. Summary of the invention

[0006] In order to solve the technical problem that the prior art cannot accurately detect small target objects in super-resolution images, the present invention proposes a super-resolution image small target detection method based on deep learning.

[0007] The specific plan is as follows:

[0008] A super-resolution image small target detection method based on deep learning.

[0009] S1: Super-resolution image reconstruction: The image is reconstructed using a sliding window cropping method, and the reconstructed image is obtained as a sub-image and cropped image information;

[0010] S11: Cropping a super-resolution image into at least two images of the same size, with small target objects distributed in the cropped images;

[0011] S12: Set the crop overlap ratio to keep small target objects intact;

[0012] S13: Reconstruct the image to obtain the reconstructed image as a sub-image and crop the image information.

[0013] S2: Data preprocessing: Enhance the image reconstructed by S1 through geometric transformation and intensity transformation to obtain an enhanced image;

[0014] S21: The geometric transformation includes random rotation, scaling, and flipping; the intensity transformation includes brightness and color changes.

[0015] S3: Feature fusion: The image enhanced by S2 is input into the backbone network for feature extraction to obtain low-dimensional feature information and high-dimensional feature information of the image; the high-dimensional feature information and the low-dimensional feature information are input into the feature fusion network for fusion to obtain the fusion features of each layer; the fusion features of each layer are input into the adaptive feature pooling layer for fusion to obtain the final fusion features;

[0016] S31: Extract features of the input image through the backbone network to obtain low-dimensional feature information and high-dimensional feature information of the image; the backbone network is the CSPDarkNet53 network, which is composed of the CSPNet network and the DarkNet53 network;

[0017] S32: fusing high-dimensional feature information and low-dimensional feature information through a feature fusion network to obtain fusion features of each layer; the feature fusion network adopts a feature fusion structure based on a PANet network, which is composed of a convolutional layer, an upsampling layer, and a downsampling layer, including: a top-down path aggregation module (PA) and a bottom-up path aggregation module (PAM);

[0018] S33: The fused feature information of each layer in the PAM module is fused through the pooling operation of the adaptive feature pooling layer.

[0019] The adaptive feature pooling layer first performs region of interest alignment on each feature layer to obtain the corresponding region of interest, i.e., ROI region, and then compresses the extracted ROI region into a one-dimensional feature vector, and then fuses different feature maps by taking the maximum value or the sum, and finally predicts the bounding box and category of the fused result.

[0020] S4: Detection model result output: The final fusion feature of S3 is input into the detection head module of the yolov8 image detection model for prediction, and the output is the small target detection result of each sub-image after image reconstruction. The coordinate information and confidence of the detected object on each image are used as the detection result relative to the sub-image;

[0021] S41: The method for outputting the detection model result is: using the detection head module of the yolov8 image detection model to predict the final fusion feature to obtain the coordinate information, confidence and category information of the small target on the sub-image, and outputting the coordinate information and confidence of the small target on the sub-image as the final detection result;

[0022] S5: Image restoration: According to the cropped image information in step S1, the coordinate information and confidence of all sub-images in S4 are restored and mapped to the original super-resolution image as the detection result on the original super-resolution image;

[0023] The steps of image restoration are:

[0024] S51: Obtain the cropped image information in S1, including: the coordinate information (x, y) of the upper left corner of each sub-image after cropping corresponding to the original image, and the size of each sub-image;

[0025] S52: The coordinate information of the small target object detected in each cropped sub-image obtained in S4, including the upper left corner coordinates (x1, y1), the lower right corner coordinates (x2, y2); plus the original image coordinate information (x, y) corresponding to each sub-image, the obtained coordinate information of the small target object on the original image is (x1, y1, x2, y2);

[0026] S53: Mapping the target onto the original super-resolution image using the new coordinate information to obtain the detection result of the original image;

[0027] S6: Non-maximum suppression algorithm NMS result deduplication: Use the non-maximum suppression algorithm to filter out the duplicate detection results in the original super-resolution image detection results of S5 due to the cropping overlap;

[0028] The steps of the non-maximum suppression algorithm NMS are:

[0029] S61: Generate a set of detection box sets B and the confidence S corresponding to each detection box through the model;

[0030] S62: Sort the detection boxes according to the confidence level, first select the detection box M with the maximum confidence level, remove it from the set B and put it into the final detection result data set D;

[0031] S63: The remaining cells in set B whose overlap value with detection box M is greater than the threshold Detection frames smaller than the threshold are directly deleted, and detection frames smaller than the threshold are retained;

[0032] (1)

[0033] Where M is the detection box with the maximum score, is the i-th detection box, is the NMS threshold, is the confidence of the i-th detection box, iou is used to measure M and The overlap of two detection frames; if the iou value is greater than or equal to the threshold, it is 0 and the detection frame is deleted. If it is less than the threshold, it is the corresponding confidence and the detection frame is retained;

[0034] S64: Select the detection frame with the second largest confidence and compare it with the remaining detection frames, repeat step S63 until all the frames are compared, and take the detection frame retained in the final detection data set D as the final result.

[0035] Beneficial effects:

[0036] The present invention proposes a method for detecting small targets in super-resolution images based on deep learning, and realizes the detection of small targets in super-resolution images by optimizing the yolov8 image detection model. By preprocessing the input image and reconstructing the super-resolution image, the problem of slow model detection speed and increased calculation caused by the large size of the super-resolution image is solved. Secondly, as the network deepens, by fusing the high-dimensional and low-dimensional features of the feature extraction layer of the model, the feature information of small target objects extracted by the model is not easily lost, thereby improving the accuracy of small target detection. Specifically:

[0037] First, in the image reconstruction step, the present invention uses a sliding cropping method to reconstruct the image, and the image overlap rate is 20% during cropping to ensure the integrity of the image in the cropping area. Secondly, in the data preprocessing step, in order to ensure the diversity of data and the generalization of the detection model, data enhancement methods such as rotation, flipping, scaling, and color change are used to increase training data and improve the robustness of the model when training the model. Third, in the feature fusion step, the input image is feature extracted through the backbone network to obtain high-dimensional feature information of the image. The high-dimensional feature information contains less information about small target objects, so the information of small target objects is supplemented by fusing low-dimensional feature information. Fourth, in the model output step: the extracted target feature information is used to use the detection head module to predict its position information, confidence and category information, and finally return the coordinate information and confidence of the target object on the image. Fifth, in the image restoration step, the model outputs the coordinate information and confidence of the target object detected in the cropped image. The output coordinate information is relative to the cropped image. The output coordinate information is mapped to the original super-resolution image through the cropping information to obtain the detection result of the original image. Sixth, in the NMS result deduplication step: when overlapping the image and cropping the detection results to map them to the original image, some results will be repeated. The non-maximum suppression algorithm is used to deduplicate the results using the predicted confidence and position information to obtain the final detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 Flowchart of a method for small object detection in super-resolution images based on deep learning.

[0039] Figure 2 The original image with super resolution in the embodiment.

[0040] Figure 3 Effect diagram of a direct clipping and overlapping clipping method in an embodiment.

[0041] Figure 4 A target detection effect diagram of a direct cropping and overlapping cropping method in an embodiment.

[0042] Figure 5 In the embodiment, a network structure diagram of the improved yolov8 image detection model based on the feature fusion structure of the PANet network is shown.

[0043] Figure 6 A network structure diagram of a prediction feature detection result based on adaptive feature pooling output in an embodiment.

[0044] Figure 7 An effect diagram of a super-resolution small target detection result in an embodiment.

[0045] Figure 8The effect diagram of restoring the cropped image and deduplicating the result in the embodiment. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0047] like Figure 1 As shown, the specific steps of the present invention include:

[0048] S1: Super-resolution image reconstruction: The image is reconstructed using a sliding window cropping method, and the reconstructed image is obtained as a sub-image and cropped image information;

[0049] S2: Data preprocessing: Enhance the image reconstructed by S1 through geometric transformation and intensity transformation to obtain an enhanced image;

[0050] S3: Feature fusion: The enhanced image in S2 is input into the backbone network for feature extraction to obtain low-dimensional feature information and high-dimensional feature information of the image; the high-dimensional feature information and the low-dimensional feature information are input into the feature fusion network for fusion to obtain the fusion features of each layer; the fusion features of each layer are input into the adaptive feature pooling layer for fusion to obtain the final fusion features;

[0051] S4: Detection model result output: The final fusion feature of S3 is input into the detection head module of the yolov8 image detection model for prediction, and the output is the small target detection result of each sub-image after image reconstruction; the small target detection result includes coordinate information and confidence;

[0052] S5: Image restoration: According to the cropped image information in step S1, the coordinate information and confidence of all sub-images in S4 are restored and mapped to the original super-resolution image as the original super-resolution image detection result;

[0053] S6: Non-maximum suppression algorithm NMS result deduplication: Use the non-maximum suppression algorithm to filter out the duplicate detection results caused by cropping overlapping parts in the original super-resolution image detection results of S5.

[0054] Specific content includes:

[0055] 1. Super-resolution image reconstruction

[0056] Super-resolution images are often very large in size and have high resolution, so the sliding window cropping method is used to reconstruct the image; a super-resolution image is cropped into multiple images, and small target objects are distributed in the cropped images. Since the detection objects in the image are cropped during the cropping process, a certain cropping overlap rate needs to be set to ensure the integrity of the small target objects;

[0057] Specifically, during the cropping process, the window size of the sliding window cropping is set to 600*600, the sliding spacing is 480, and the cropping overlap rate is 20%.

[0058] For example, Figure 2 The original image is shown. Figure 3 The effects of direct cropping and overlapping cropping are shown. Figure 4 The final target detection results of direct cropping and overlapping cropping are shown. Figure 4 (a) is the detection result of directly cropping the blocks. It can be seen that the yellow box in the figure is a small target that has not been detected. Figure 3 As can be seen in (a), the small target that is not recognized is cropped from the middle, resulting in incompleteness, so it cannot be recognized. Figure 4 (b) is detected in the overlapping cropped patches, and Figure 3 (b) fully retains Figure 3 The small object that is cropped in (a). Therefore, overlapping cropping can make up for the unrecognized situation caused by cropping in direct cropping.

[0059] 2. Data preprocessing

[0060] In order to make the training data more diverse, data augmentation is used in the model training process to increase the amount of data while improving the generalization ability and robustness of the network. And by performing data augmentation on a small number of samples, the proportion of sample imbalance can be reduced. This processing specifically includes geometric transformation and intensity transformation. Geometric transformation includes random rotation, scaling, and flipping; intensity transformation includes changes in brightness, color, etc.

[0061] Specifically, image rotation: the image rotation angle is randomly selected from -30 to 30 degrees; image flipping: the image is flipped up and down along the x-axis or the image is flipped left and right along the y-axis; image scaling: the image is randomly scaled between 0.8 and 1.2 times; color change: color transformation, adjustment of HSV color space: the hue (H) parameter should be adjusted within the range of 0 to 360 degrees, the saturation (S) parameter should be adjusted within the range of 0% to 100%, and the brightness (V) parameter should also be adjusted within the range of 0% to 100%.

[0062] 3. Feature Fusion

[0063] like Figure 5 As shown in the figure, the network structure diagram of the improved yolov8 image detection model based on the feature fusion structure of the PANet network is composed of a backbone network, a feature fusion network, an adaptive feature pooling layer and a detection head.

[0064] The image is input into the backbone network for feature extraction to obtain low-dimensional feature information and high-dimensional feature information of the image;

[0065] Specifically, the backbone network uses CSPDarkNet53, which is composed of CSPNet and DarkNet53 modules. CSPNet, which stands for Cross Stage Partial Networks, solves the problem of gradient information duplication in network optimization in large convolutional neural network frameworks and reduces the number of model parameters. DarkNet53 consists of 53 convolutional layers and residual structures, which ensures accuracy while improving network speed.

[0066] By inputting high-dimensional feature information and low-dimensional feature information into the feature fusion network for fusion, the fusion features of each layer are obtained;

[0067] Specifically, the feature fusion network adopts a feature fusion structure based on the PANet network, which consists of convolutional layers, upsampling layers, and downsampling layers. It contains two main modules: top-down path aggregation module (PA): This module starts from a lower-resolution feature map and passes information up layer by layer through upsampling operations. At each level, the low-resolution feature map is horizontally connected and point-by-point added to the high-resolution feature map to integrate information of different scales; bottom-up path aggregation module (PAM): This module starts from a high-resolution feature map and passes information down layer by layer through downsampling operations. Its structure is as follows: Figure 5 (b) Similarly, at each level, the high-resolution feature map is laterally connected and point-wise added with the low-resolution feature map.

[0068] The fused feature information of each layer in the PAM module is fused through the pooling operation of the adaptive feature pooling layer;

[0069] Specifically, the adaptive feature pooling layer structure is as follows: Figure 6 As shown. In FPN, each feature map will output a prediction result. The reason why FPN does this is that the receptive field is proportional to the network depth, that is, the deeper the network, the larger the receptive field of the pixels on the network. But in fact, the different characteristics of feature maps at different levels are not only the receptive field, but also their different focuses. Based on this idea, a pooling operation that fuses feature maps of all layers is proposed - the adaptive feature pooling layer. The adaptive feature pooling layer first performs region of interest alignment on each feature layer to obtain the corresponding region of interest, that is, the ROI region, and then compresses the extracted ROI into a one-dimensional feature vector, and then fuses different feature maps by taking the maximum value or the sum, and finally predicts the bounding box and category of the fused result.

[0070] The feature fusion structure based on the PANet network proposes the following innovations on the FPN (feature pyramid) network, which significantly improves the performance of the model on the target detection network:

[0071] (1) By improving the feature fusion network in the feature pyramid structure, the structure of the feature pyramid is strengthened and the path of high-level and low-level feature fusion is shortened;

[0072] (2) A more flexible adaptive feature pooling is proposed. Previously, the feature pooling of FPN only took values ​​from high-level features, but now it operates on features at all scales.

[0073] In order to demonstrate the effectiveness of the proposed feature fusion structure based on the PANet network, an experiment was conducted. The dataset used is the COCO dataset, which contains 115k images for training and 5k images for verification. The test-dev dataset contains 20k images with pixel-level instance segmentation annotations of 80 categories. The model was trained on the train-2017 subset and tested on the test-dev dataset. The test results of the PANet, RentinaNet, and Mask R-CNN models on the COCO dataset are shown in the table:

[0074]

[0075] It can be clearly seen from the data in Table 1 that the feature fusion structure based on the PANet network significantly improves the average precision mAP of the model. This shows that the features extracted by the PANet network are richer in information, which is extremely beneficial to the learning process of the model, thereby significantly enhancing the recognition accuracy of the model.

[0076] 4. Model Output

[0077] The detection model outputs the super-resolution image through the detection head module to reconstruct the small target detection results of multiple sub-images. The position information of the detection object on each image is detected relative to the sub-image. The detection head of the detection model is as follows: Figure 5 As shown in (d).

[0078] Specifically, the detection head includes a position detection head and a category classification head. The position detection head consists of a convolutional layer and a deconvolutional layer, which is used to generate detection results and is responsible for predicting the bounding box regression value of each anchor box and the confidence of the target existence; the category classification head consists of a global average pooling, which classifies each feature map and outputs the probability distribution of each category by reducing the dimension of the feature map. The detection results output by the model are as follows: Figure 7 shown.

[0079] 5. Image restoration

[0080] Since the position information of the small target object predicted by each sub-image is relative to each sub-image, it is necessary to restore and map all the position information to the original super-resolution image. First, the cropped image information in S1 is obtained, including: the coordinate information (x, y) of the upper left corner of each sub-image after cropping corresponding to the coordinate information of the original image, the size of each sub-image, the size is set to 600*600, and the overlap rate of the cropped image is set to 20%; the coordinate information of the small target object detected in each sub-image after cropping obtained by S4, including: the coordinates of the upper left corner (x1, y1), the coordinates of the lower right corner (x2, y2); plus the coordinate information (x, y) of each sub-image corresponding to the original image, the obtained coordinate information of the small target object on the original image is (x1, y1, x2, y2); the target is mapped to the original super-resolution image using the new coordinate information to obtain the detection result of the original image.

[0081] 6. Deduplication of NMS results

[0082] In the process of super-resolution image reconstruction, the image is cropped with sliding windows. In order to ensure the integrity of small target objects, the cropping overlap rate is set to 20%. When restoring the image, the overlapping parts will be detected multiple times, and there will be duplication in the output results. Therefore, the non-maximum suppression algorithm is used to filter the repeated detection results. The restored image after deduplication is as follows Figure 8 shown.

[0083] The non-maximum suppression algorithm NMS first generates a set of detection boxes B and the confidence S corresponding to each detection box through the model, sorts these detection boxes according to the scores, first selects the detection box M with the maximum confidence, removes it from the set B and puts it into the final detection result data set D. At this time, the remaining detection boxes in the set B with an overlap value greater than the threshold value with the detection box M are removed. The detection frames with a value smaller than the threshold are directly deleted, and the detection frames with a value smaller than the threshold are retained. This process is repeated again and again, and the detection frame with the second largest confidence is selected and compared with the remaining detection frames until all the frames are compared. The detection frames retained in the final detection data set D are the final results.

[0084] (1)

[0085] Where M is the detection box with the maximum confidence. is the i-th detection box, is the NMS threshold, is the confidence of the i-th detection box, iou is used to measure M and The overlap degree of two detection frames; if the iou value is greater than or equal to the threshold, it is 0 and the detection frame is deleted. If it is less than the threshold, it is the corresponding confidence and the detection frame is retained.

[0086] Specifically, Set to 0.5.

[0087] Specifically, in order to demonstrate the effectiveness of the proposed model, an ablation experiment was conducted. The dataset used was a super-resolution small target image dataset, which is a collection of satellite aerial photography datasets. The dataset contains 30 categories of target objects, including aircraft (11 categories), ships (9 categories), and vehicles (10 categories). The dataset contains a total of 16,000 images, of which 80% are training sets and 20% are test sets. Training and verification were conducted on this dataset, and ablation experiments were conducted on adding overlapping cropping blocks and PANet networks, respectively, with the baseline being the yolov8 target detection model.

[0088]

[0089] From the data in Table 2, it can be observed that the introduction of the feature fusion structure based on the PANet network into the baseline model significantly improves the accuracy of the model. This shows that the features extracted by the PANet network are rich in more valuable information, which effectively enhances the accuracy of recognition. In addition, the baseline model that only uses the overlapping cropping block technology shows a higher accuracy improvement than the direct use of the PANet feature fusion structure, which emphasizes the advantage of the overlapping cropping block technology in retaining smaller target information, and also shows that the cropping strategy has a significant impact on the model accuracy. When these two methods are integrated into the baseline model at the same time, further accuracy improvements are seen, proving that combining these two techniques is more efficient than applying them separately.

[0090] The above specific implementation modes are preferred embodiments of the present invention and cannot be used to limit the present invention. Any other changes or other equivalent replacement methods that do not deviate from the technical solution of the present invention are included in the protection scope of the present invention.

Claims

1. A method for detecting small targets in super-resolution images based on deep learning, characterized in that: S1: Super-resolution image reconstruction: The image is reconstructed using a sliding window cropping method, and the reconstructed image is obtained as a sub-image and cropped image information; S2: Data preprocessing: Enhance the image reconstructed by S1 through geometric transformation and intensity transformation to obtain an enhanced image; S3: Feature fusion: The image enhanced by S2 is input into the backbone network for feature extraction to obtain low-dimensional feature information and high-dimensional feature information of the image. The backbone network is the CSPDarkNet53 network; the high-dimensional feature information and the low-dimensional feature information are input into the feature fusion network for fusion to obtain the fusion features of each layer. The feature fusion network adopts a feature fusion structure based on the PANet network; the fusion features of each layer are input into the adaptive feature pooling layer for fusion to obtain the final fusion features; S4: Detection model result output: The final fusion feature of S3 is input into the detection head module of the yolov8 image detection model for prediction, and the output is the small target detection result of each sub-image after image reconstruction; the small target detection result includes coordinate information and confidence; S5: Image restoration: According to the cropped image information in step S1, the coordinate information and confidence of all sub-images in S4 are restored and mapped to the original super-resolution image as the original super-resolution image detection result; S6: Non-maximum suppression algorithm NMS result deduplication: Use the non-maximum suppression algorithm to filter out the duplicate detection results in the original super-resolution image detection results of S5 due to the cropping overlap. The specific steps are as follows: S61: Generate a set of detection box sets B and the confidence S corresponding to each detection box through the model; S62: Sort the detection boxes according to the confidence level, first select the detection box M with the maximum confidence level, remove it from the set B and put it into the final detection result data set D; S63: directly delete the remaining detection frames in set B whose overlap values ​​with detection frame M are greater than the threshold, and retain the detection frames whose overlap values ​​are less than the threshold; Where M is the detection box with the maximum score, is the i-th detection box, is the NMS threshold, is the confidence of the i-th detection box, iou is used to measure M and The overlap of two detection frames; if the iou value is greater than or equal to the threshold, it is 0 and the detection frame is deleted. If it is less than the threshold, it is the corresponding confidence and the detection frame is retained; S64: Select the detection frame with the second largest confidence and compare it with the remaining detection frames, repeat step S63 until all the frames are compared, and take the detection frame retained in the final detection data set D as the final result.

2. A method for detecting small targets in super-resolution images based on deep learning according to claim 1, characterized in that: The method for super-resolution image reconstruction described in S1 is: S11: Cropping a super-resolution image into at least two images of the same size, with small target objects distributed in the cropped images; S12: Set the crop overlap ratio to keep small target objects intact; S13: Reconstruct the image to obtain the reconstructed image as a sub-image and crop the image information.

3. A method for detecting small targets in super-resolution images based on deep learning according to claim 1, characterized in that: The geometric transformation in S2 includes random rotation, scaling, and flipping; the intensity transformation includes brightness and color changes.

4. A method for detecting small targets in super-resolution images based on deep learning according to claim 1, characterized in that: The CSPDarkNet53 network is composed of a CSPNet network and a DarkNet53 network; the feature fusion structure based on the PANet network is composed of a convolutional layer, an upsampling layer and a downsampling layer, including: a top-down path aggregation module (PA) and a bottom-up path aggregation module (PAM); the final fusion feature is obtained by fusing the fusion feature information of each layer in the PAM module through the pooling operation of the adaptive feature pooling layer.

5. A method for detecting small targets in super-resolution images based on deep learning according to claim 4, characterized in that: The adaptive feature pooling layer first performs region of interest alignment on each feature layer to obtain the corresponding region of interest, i.e., ROI region, and then compresses the extracted ROI region into a one-dimensional feature vector, and then fuses different feature maps by taking the maximum value or the sum, and finally predicts the bounding box and category of the fused result.

6. A method for detecting small targets in super-resolution images based on deep learning according to claim 1, characterized in that: The method for outputting the detection model results in step S4 is: use the detection head module of the yolov8 image detection model to predict the final fusion features to obtain the coordinate information, confidence and category information of the small target on the sub-image, and output the coordinate information and confidence of the small target on the sub-image as the final detection result.

7. A method for detecting small targets in super-resolution images based on deep learning according to claim 1, characterized in that: The method for image restoration in step S5 is: S51: Obtain the cropped image information in S1, including: the coordinate information (x, y) of the upper left corner of each sub-image after cropping corresponding to the original image, and the size of each sub-image; S52: The coordinate information of the small target object detected in each cropped sub-image obtained in S4, including the upper left corner coordinates (x1, y1), the lower right corner coordinates (x2, y2); plus the original image coordinate information (x, y) corresponding to each sub-image, the obtained coordinate information of the small target object on the original image is (x1, y1, x2, y2); S53: Mapping the target onto the original super-resolution image using the new coordinate information to obtain the detection result of the original image.

Images