Small target detection method, device, electronic device and storage medium

Abstract

The present invention relates to the field of image data processing technology, and provides a small target detection method, device, electronic device and storage medium, comprising: inputting a remote sensing image to be detected into a small target detection network, wherein the small target detection network comprises: a CSPDarknet53 network and a Focus network; extracting features of the remote sensing image to be detected through a convolution block to obtain a first image; and capturing features of the first image through a first deep residual network to obtain multiple second images; performing channel processing on the remote sensing image to be detected through a Focus layer to obtain a third image; and capturing features of the third image through a second deep residual network to obtain multiple fourth images, wherein the number of the multiple fourth images is the same as the number of the multiple second images; and determining the small target detection result corresponding to the remote sensing image to be detected according to the multiple second images and the multiple fourth images. The detection accuracy of small targets is improved.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology, and in particular to a small target detection method, device, electronic equipment and storage medium. Background Art

[0002] Target detection is an important research direction in the field of computer vision, which is mainly used to identify and locate specific targets. In recent years, with the development of remote sensing technology, remote sensing images (such as optical images and synthetic aperture radar (SAR) images) have been used more and more widely. Remote sensing images are multi-source and heterogeneous, and target detection for multi-source heterogeneous remote sensing images has become a research hotspot. Among them, the detection of weak targets is very challenging because these weak targets are affected by their own size and image resolution. They only occupy a few pixels to a dozen pixels in remote sensing images. The amount of target information that these weak targets can provide is limited. The detection of weak targets is very challenging due to factors such as small sample size, small target size, complex background, and weak feature expression ability.

[0003] In summary, how to improve the detection accuracy of small targets has become an urgent problem to be solved. Summary of the invention

[0004] The present invention provides a small target detection method, device, electronic device and storage medium, which are used to solve the defect of low small target detection accuracy in the prior art. The dual features of small targets in remote sensing images to be detected are extracted and processed by CSPDarknet53 network and Focus network, thereby improving the detection accuracy of small targets.

[0005] The present invention provides a small target detection method, comprising the following steps.

[0006] The remote sensing image to be detected is input into the small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network.

[0007] The convolution block is used to extract features of the remote sensing image to be detected to obtain a first image; and the first deep residual network is used to capture features of the first image to obtain multiple second images.

[0008] Through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images.

[0009] A small target detection result corresponding to the remote sensing image to be detected is determined according to the multiple second images and the multiple fourth images.

[0010] The present invention also provides a small target detection device, comprising the following modules.

[0011] A data processing module is used to input the remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; through the convolution block, feature extraction is performed on the remote sensing image to be detected to obtain a first image; and through the first deep residual network, feature capture is performed on the first image to obtain multiple second images; through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images.

[0012] The target detection module is used to determine the small target detection result corresponding to the remote sensing image to be detected according to the multiple second images and the multiple fourth images.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the above-described small target detection methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the small target detection method described in any one of the above methods is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned small and dim target detection methods.

[0016] The small target detection method, device, electronic device and storage medium provided by the present invention are implemented by inputting a remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; through the convolution block, feature extraction is performed on the remote sensing image to be detected to obtain a first image; and through the first deep residual network, feature capture is performed on the first image to obtain multiple second images; through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images; according to the multiple second images and the multiple fourth images, the small target detection result corresponding to the remote sensing image to be detected is determined. This method uses the CSPDarknet53 network and the Focus network to extract and process the dual features of small targets in remote sensing images, thereby improving the detection accuracy of small targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the small target detection method provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of a twin network based on a CSPDarknet53 network and a Focus network provided by the present invention.

[0020] Figure 3 This is one of the structural schematic diagrams of the weak target detection network provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the Focus layer provided by the present invention.

[0022] Figure 5 This is the second structural diagram of the weak target detection network provided by the present invention.

[0023] Figure 6 It is a schematic diagram of the traditional void convolution kernel stacking effect and the hybrid void convolution kernel stacking effect provided by the present invention.

[0024] Figure 7 It is a schematic diagram of a scene for determining a second feature image based on a small target detection network provided by the present invention.

[0025] Figure 8 It is a schematic diagram of a scene for determining a third characteristic image based on PAN provided by the present invention.

[0026] Fig. 9 It is a schematic diagram of a scene for determining the fourth characteristic image based on CBAM provided by the present invention.

[0027] Fig.10 It is a schematic diagram of a scenario for determining a small target detection result provided by the present invention.

[0028] Fig.11a It is one of the schematic diagrams of the small target detection result provided by the present invention.

[0029] Fig.11b This is the second schematic diagram of the small target detection result provided by the present invention.

[0030] Fig.11c This is the third schematic diagram of the small target detection result provided by the present invention.

[0031] Fig.11d This is the fourth schematic diagram of the small target detection result provided by the present invention.

[0032] Fig.11e It is a schematic diagram of the definition of the predicted rotation rectangular frame provided by the present invention.

[0033] Fig.11f It is a schematic diagram of a scene of fitting a true rotated rectangular frame regression provided by the present invention.

[0034] Fig.12 It is a schematic diagram of the result of augmenting the initial remote sensing image sample provided by the present invention.

[0035] Fig.13 It is a structural schematic diagram of a small target detection device provided by the present invention.

[0036] Fig.14 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be noted that the execution subject involved in the embodiments of the present invention may be a small target detection device or an electronic device. Optionally, the electronic device may include: a computer, a mobile terminal, a wearable device, etc.

[0039] The embodiment of the present invention is further described below by taking an electronic device as an example.

[0040] Figure 1 It is a flowchart of the small target detection method provided by the present invention. Figure 1 As shown, the method includes the following steps 101 to 104.

[0041] Step 101: Input the remote sensing image to be detected into the small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network.

[0042] In order to avoid the single feature extraction network (such as CSPDarknet53 network) being unable to extract the features of small and weak targets in an all-round way, the network architecture design and implementation of the small and weak target detection network is based on the YOLOv4 network, adding the Focus feature enhancement network (abbreviated as: Focus network) and the feature fusion between each layer of the network to optimize the network architecture. In this way, the CSPDarknet53 network and the Focus network become a twin-like feature extraction structure, forming a twin network (i.e., a small and weak target detection network) (such as Figure 2 As shown in the figure), the blue part is the CSPDarknet53 network and the green part is the Focus network.

[0043] It should be noted that although the YOLO network is the earliest proposed single-step detection method and is one of the most efficient detection algorithms so far, the YOLOv4 network has always faced the problem of poor performance in detecting small and weak targets. The small and weak target detection network involved in the embodiment of the present invention is based on the YOLOv4 network. While maintaining the simplicity and clarity of the structural network, it can improve and optimize the network structure for small and weak targets that are loosely fixed and densely moved, thereby improving the detection performance of small and weak targets in complex environments. In other words, the YOLOv4 network is selected as the basic network of the small and weak target detection network, and improvements are made for the difficulties of complex scenes and small sizes of small and weak targets. Dense connections are added to the backbone network and a small-scale detection layer is added to reduce the missed detection rate of small and weak targets, thereby improving the detection performance of small and weak targets.

[0044] In the process of optimizing the YOLOv4 network, firstly, the Focus network is introduced into the Backbone network to form a branch similar to the CSPDarknet53 network to enhance the feature extraction capability; secondly, the shallow features (such as the images output by the convolution block and the Focus layer respectively) are connected to the deep features (such as the images output by each deep convolutional network) to enhance the information fusion capability of the features; thirdly, on the basis of the original three-layer detection layer, an additional small-size detection layer for weak and small targets is added to solve the missed detection phenomenon caused by dense arrangement of weak and small targets in complex scenes.

[0045] For example, Figure 3 is a schematic diagram of the structure of the weak target detection network provided by the present invention, such as Figure 3 As shown, the small target detection network includes: CSPDarknet53 network and Focus network, CSPDarknet53 network includes: convolution block and the first deep residual network, and Focus network includes: Focus layer and the second deep residual network.

[0046] Step 102: extract features of the remote sensing image to be detected through a convolution block to obtain a first image; and capture features of the first image through a first deep residual network to obtain multiple second images.

[0047] When applying the CSPDarknet53 network in the small target detection network, the electronic device can first perform feature extraction on the remote sensing image to be detected through a convolution block, achieve a first downsampling of the remote sensing image to be detected, and obtain a first image; then input the first image into a first deep residual network, capture features of the first image through the first deep residual network, achieve secondary feature extraction of the remote sensing image to be detected, and obtain multiple second images with higher accuracy.

[0048] In some embodiments, the convolution block may include: a two-dimensional convolutional layer (2D Convolutional Layer), batch normalization (Batch Normalization, BN) and a non-linear activation function; the electronic device extracts features of the remote sensing image to be detected through the convolution block to obtain a first image, which may include: extracting features of the remote sensing image to be detected through the two-dimensional convolutional layer to obtain a first sub-image; normalizing the first sub-image through batch normalization to obtain a second sub-image; and enhancing features of the second sub-image through a non-linear activation function to obtain the first image.

[0049] In the process of feature extraction of the remote sensing image to be detected through the convolution block, firstly, the two-dimensional convolution layer can effectively extract local features from the remote sensing image to be detected through the mechanism of local connectivity and weight sharing, and through the stacking of convolution layers, more advanced and abstract features can be extracted layer by layer to obtain the first sub-image with higher accuracy. In addition, the whole process can reduce the number of parameters of the weak target detection network, reduce the risk of overfitting, and improve the efficiency of feature extraction; then, batch normalization has a certain regularization effect, which can reduce the dependence of the weak target detection network on the initial weights and improve the generalization ability of the weak target detection network. In the processing of the remote sensing image to be detected, this means that the weak target detection network can better adapt to the remote sensing data of different times and regions to obtain the second sub-image; finally, the nonlinear activation function introduces nonlinear factors into the weak target detection network, so that the weak target detection network can learn and represent more complex features, realize the feature enhancement of the second sub-image, and obtain the first image with higher accuracy.

[0050] In some embodiments, the first deep residual network includes: a first-layer residual network and multiple first residual networks, the first-layer residual network is connected in series with the first first residual network, and the multiple first residual networks are connected in series; the electronic device captures features of the first image through the first deep residual network to obtain multiple second images, which may include: extracting features of the first image through the first-layer residual network to obtain a first feature image; determining the corresponding second image according to the first image and the first target image through each first residual network; wherein the first target image corresponding to the first first residual network is the first feature image; the first target images corresponding to the remaining first residual networks are the images output by the first residual network of the previous adjacent layer.

[0051] In the process of capturing features of the first image through the first deep residual network, since the residual networks included in the first deep residual network are connected in series, the first image can be firstly extracted through the first residual network to obtain the first feature image; then, the first image is input into each first residual network, and the first feature image is input into the first first residual network. Specifically, the first feature image and the first image are processed by the first first residual network to obtain the corresponding second image; then, the first image and the image output by the first residual network of the previous adjacent layer are processed by the remaining first residual networks to obtain the corresponding second image. Based on this, the electronic device will obtain as many second images as there are first residual networks. In the whole process, a very deep network structure can be constructed by connecting multiple residual networks in series, so that the weak target detection network can learn more complex and abstract feature representations and obtain multiple second images with higher accuracy.

[0052] In some embodiments, the electronic device determines the corresponding second image based on the first image and the first target image through each first residual network, which may include: for each first residual network, superimposing and fusing the first image and the first target image to obtain a fused image; using the first residual network to continuously extract features of the fused image according to a first preset number of times to obtain the corresponding second image.

[0053] For each first residual network, when the first image and the first target image are input into the corresponding first residual network, the two images can be superimposed and fused through the first residual network to obtain a fused image; then, features of the fused image are continuously extracted according to a first preset number of times to obtain a second image corresponding to the first residual network.

[0054] It can be understood that one first residual network corresponds to one first preset number, wherein any two first preset numbers may be the same or different, and no specific limitation is made here.

[0055] Exemplarily, the first deep residual network includes: a first-layer residual network P1 and four first residual networks, namely, a first residual network P2, a first residual network P3, a first residual network P4 and a first residual network P5. Among them, the first preset number of times corresponding to the first-layer residual network P1 is 1; the first preset number of times corresponding to the first residual network P2 is 2; the first preset number of times corresponding to the first residual network P3 is 8; the first preset number of times corresponding to the first residual network P4 is 8; the first preset number of times corresponding to the first residual network P5 is 4.

[0056] Step 103: Perform channel processing on the remote sensing image to be detected through the Focus layer to obtain a third image; and perform feature capture on the third image through the second deep residual network to obtain multiple fourth images, where the number of the multiple fourth images is the same as the number of the multiple second images.

[0057] When applying the Focus network in the small target detection network, the electronic device can first perform channel processing on the remote sensing image to be detected through the Focus layer, realize channel dimension reduction of the remote sensing image to be detected, and obtain a third image; then input the third image into the second deep residual network, perform feature capture on the third image through the second deep residual network, realize secondary feature extraction of the remote sensing image to be detected, and obtain multiple fourth images with higher accuracy.

[0058] Optionally, the electronic device performs channel processing on the remote sensing image to be detected through the Focus layer to obtain a third image, which may include: taking a pixel value for every other pixel of the sample image to be detected through the Focus layer to obtain four sub-images; and performing splicing and convolution operations on the four sub-images to obtain the third image.

[0059] Through the Focus layer (such as Figure 4 As shown in the figure, in the process of channel processing of the remote sensing image to be detected, the Focus layer takes a pixel value for every other pixel in a remote sensing image to be detected, which is similar to the neighboring downsampling (Sample), so that four sub-images are obtained. These four sub-images are complementary and similar in shape, but no information is lost. Based on this, the image width and height information are concentrated in the channel space, and the input channel is expanded by 4 times, that is, the concatenated image becomes 12 channels compared to the original red-green-blue (RGB) three-channel mode. Finally, the new image is subjected to the convolution operation (Normal Conv) to obtain a double-downsampled feature image without information loss, that is, a third image with higher accuracy.

[0060] In some embodiments, the second deep residual network includes: multiple second residual networks, and the multiple second residual networks correspond one-to-one and are the same as the multiple first residual networks. That is, the network structure of the Focus network skips the first layer of the residual network in the first deep residual network, and is the same as the network structure of the multiple first residual networks.

[0061] The electronic device captures features of the third image through a second deep residual network to obtain multiple fourth images, which may include: extracting features of the second target image through each second residual network to obtain corresponding fourth images; wherein the second target image corresponding to the first second residual network is the third image; and the second target images corresponding to the remaining second residual networks are images output by the second residual network of the previous adjacent layer.

[0062] In the process of capturing features of the third image through the second deep residual network, since the residual networks included in the second deep residual network are connected in series, the features of the third image can be extracted through the first second residual network to obtain the corresponding fourth image, and the fourth image can be input into the second residual network of the next adjacent layer; for the remaining second residual networks, the image output by the second residual network of the previous adjacent layer can be extracted to obtain the corresponding fourth image. Based on this, the electronic device will obtain as many fourth images as there are second residual networks, so that the number of fourth images is the same as the number of second images. In the whole process, a very deep network structure can be constructed by connecting multiple second residual networks in series, so that the weak target detection network can learn more complex and abstract feature representations and obtain multiple fourth images with higher accuracy.

[0063] For example, Figure 5 is a schematic diagram of the structure of the weak target detection network provided by the present invention, such as Figure 5 As shown, the small target detection network includes: CSPDarknet53 network and Focus network, CSPDarknet53 network includes: convolution block and the first deep residual network, and Focus network includes: Focus layer and the second deep residual network.

[0064] Among them, the first deep residual network includes: a first-layer residual network P1 and four first residual networks, namely, a first residual network P2, a first residual network P3, a first residual network P4 and a first residual network P5, and these five residual networks are connected in series in sequence; the second deep residual network includes: four second residual networks, namely, a second residual network Q1, a second residual network Q2, a second residual network Q3 and a second residual network Q4, and these four residual networks are connected in series in sequence.

[0065] It should be noted that the first residual network P2 is the same as the second residual network Q1, the first residual network P3 is the same as the second residual network Q2, the first residual network P4 is the same as the second residual network Q3, and the first residual network P5 is the same as the second residual network Q4.

[0066] It should be noted that there is no time sequence limitation for step 102 and step 103 .

[0067] Step 104: Determine the small target detection result corresponding to the remote sensing image to be detected according to the multiple second images and the multiple fourth images.

[0068] In some embodiments, the electronic device determines the small and weak target detection result corresponding to the remote sensing image to be detected based on multiple second images and multiple fourth images, which may include: for the same first residual network and second residual network, through hybrid dilated convolution (HDC), the second image corresponding to the first residual network and the fourth image corresponding to the second residual network are spliced ​​and feature extracted to obtain a second feature image; based on the multiple second feature images, the small and weak target detection result corresponding to the remote sensing image to be detected is determined.

[0069] Among them, HDC has two main functions: First, it can expand the receptive field without increasing parameters. In deep network structures, in order to increase the receptive field and reduce the amount of calculation, downsampling is always performed. Although this can increase the receptive field, the spatial resolution is reduced. In order to not lose resolution and expand the receptive field at the same time, dilated convolution can be used; second, it can capture multi-scale context information. Dilated convolution has a parameter that can set the dilation rate, expressed as r. The dilation rate is generally set by filling r-1 zeros on the convolution kernel. Different dilation rates correspond to receptive fields of different sizes, and image features of different scales are obtained through receptive fields of different sizes. Based on this, the introduction of mixed dilated convolution can enhance the feature extraction ability of the weak target detection network, thereby improving the detection performance of weak targets.

[0070] Combination Figure 5 It can be seen that in the process of determining the small target detection result, the electronic device introduces a mixed hole convolution in order to increase the receptive field while keeping the small target positioning information of the image intact. At this time, the second image corresponding to the first residual network P2 and the fourth image corresponding to the second residual network Q1 can be spliced ​​through the mixed hole convolution to obtain a spliced ​​image, and then the feature extraction of the spliced ​​image is performed to obtain a second feature image. Based on this, the electronic device can determine the four second feature images through the mixed hole convolution, and then determine the small target detection result corresponding to the remote sensing image to be detected.

[0071] It should be noted that the above process uses a 3×3 convolution kernel and a hybrid dilated convolution with a dilation rate r of [1, 2, 5] to replace the 3×3 traditional dilated convolution, that is, the traditional dilated convolution is replaced by the batch normalization activation layer (Hybrid Dilated Connvolution BN Leaky ReLU Layer) corresponding to the hybrid dilated convolution.

[0072] For example, Figure 6Schematic diagram of the stacking effect of the traditional dilated convolution kernel and the mixed dilated convolution kernel provided by the present invention. Figure 6 (a) is used to characterize the effect of stacking traditional hole convolution kernels; Figure 6 (b) in Fig. 1 is used to characterize the stacking effect of mixed hole convolution kernels. Figure 6 By comparing (a) and (b) in the figure, it can be seen that the stacking effect of the mixed void convolution kernel is obviously better than the stacking effect of the traditional void convolution kernel.

[0073] In some embodiments, the electronic device stitches and extracts features of the second image corresponding to the first residual network and the fourth image corresponding to the second residual network to obtain a second feature image, including: for the last identical first residual network and second residual network, stitching the second image corresponding to the first residual network and the fourth image corresponding to the second residual network; obtaining a stitched image, and continuously extracting features of the stitched image according to a second preset number of times to obtain a corresponding second feature image.

[0074] Combination Figure 5 It can be seen that the electronic device, through HDC, performs feature extraction only once on the stitched image corresponding to the first residual network P2 and the second residual network Q1; through HDC, performs feature extraction only once on the stitched image corresponding to the first residual network P3 and the second residual network Q2; through HDC, performs feature extraction only once on the first residual network P4 and the second residual network Q3; through HDC, performs feature extraction only a second preset number of times (e.g., 3 times) on the stitched image corresponding to the first residual network P5 and the second residual network Q4, and obtains corresponding second feature images respectively.

[0075] For example, Figure 7 is a schematic diagram of a scene for determining a second feature image based on a weak target detection network provided by the present invention, combined with Figure 5 ,from Figure 7It can be seen from the figure that: the remote sensing image Input (608, 608, 3) to be detected is input into the weak target detection network; in the CSPDarknet53 network, the remote sensing image to be detected is subjected to feature extraction through the convolution block to obtain the first image DarknetConv2D_BN_Mish (608, 608, 32); the first residual network P1 is used to extract features of the first image once to obtain the first feature image; the first residual network P2 is used to capture features of the first feature image twice to obtain the second image A1; the first residual network P3 is used to capture features of the fused image corresponding to the first image and the second image A1 eight times to obtain the second image A2; the first residual network P4 is used to capture features of the fused image corresponding to the first image and the second image A2 eight times to obtain the second image A3; the first residual network P5 is used to capture features of the fused image corresponding to the first image and the second image A3 four times to obtain the second image A4. At this time, the electronic device can obtain four second images.

[0076] In the Focus network, the remote sensing image to be detected is processed through the Focus layer to obtain the third image Focus (304, 304, 64); the remote sensing image to be detected is captured twice through the second residual network Q1 to obtain the fourth image B1; the fourth image B1 is captured eight times through the second residual network Q2 to obtain the fourth image B2; the fourth image B2 is captured eight times through the second residual network Q3 to obtain the fourth image B3; the fourth image B3 is captured four times through the second residual network Q4 to obtain the fourth image B4. At this time, the electronic device can obtain four fourth images.

[0077] The electronic device uses HDC to extract features from the stitched image corresponding to the second image A1 and the fourth image B1 to obtain the second feature image C1; uses HDC to extract features from the stitched image corresponding to the second image A2 and the fourth image B2 to obtain the second feature image C2; ​​uses HDC to extract features from the stitched image corresponding to the second image A3 and the fourth image B3 to obtain the second feature image C3; uses HDC to extract features from the stitched image corresponding to the second image A4 and the fourth image B4 to obtain the second feature image C4. At this point, the electronic device can obtain four second feature images.

[0078] Depend on Figure 7It can be seen that after the CSPDarknet53 network and the Focus network process the remote sensing images to be detected, the various structural features are rearranged. In order to make full use of the features of each layer of the twin network, the above process fuses the residual results of each level in the two networks to combine different hierarchical features into a higher-order feature image, so as to learn the complementary relationship between different features.

[0079] Optionally, the electronic device determines the small and weak target detection result corresponding to the remote sensing image to be detected based on multiple second feature images, including: the electronic device inputs the multiple second feature images into a path aggregation network (PAN) to obtain multiple third feature images, wherein the number of third feature images is the same as the number of second feature images; for each third feature image, the electronic device inputs the third feature image into a convolutional block attention module (CBAM) to obtain a fourth feature image, and then determines the small and weak target detection result based on the predicted rotated rectangular box in the fourth feature image and combined with large aircraft suppression.

[0080] Among them, PAN is used to effectively fuse low-level features with high-level features through bottom-up path enhancement, thereby improving the accuracy of the third feature image.

[0081] CBAM is used to select the salient target features that need to be focused on in the third feature image for feature extraction, and suppress unimportant background features, thereby reducing false alarm targets and improving target detection performance. The core idea of ​​CBAM is to integrate the channel dimension and the spatial dimension, break through the defect that the original attention mechanism only focuses on a single dimension, realize the suppression of weak target background features, and highlight the salient target features.

[0082] For example, Figure 8 is a schematic diagram of a scene for determining a third characteristic image based on PAN provided by the present invention, combined with Figure 7 ,from Figure 8 It can be seen that: the four second feature images are input into PAN to realize context feature extraction, and four third feature images are obtained, namely the third feature image D1, the third feature image D2, the third feature image D3 and the third feature image D4.

[0083] For example, Fig. 9 is a schematic diagram of a scene for determining the fourth characteristic image based on CBAM provided by the present invention, such as Fig. 9As shown, this is the CBAM corresponding to the i-th third feature image Ei among the four third feature images. Specifically, the third feature image Ei is input into the CBAM, and the channel feature is extracted from the third feature image Ei through the channel attention module included in the CBAM, and then the element product operation is performed on the third feature image Ei to obtain the operation result; then the spatial feature is extracted from the operation result through the spatial attention module included in the CBAM, and then the element product operation is performed on the operation result to obtain the fourth feature image Ei.

[0084] Combined with step 101 to step 104, Fig.10 is a schematic diagram of a scene for determining a small target detection result provided by the present invention, combined with Figure 5-Figure 9 ,from Fig.10 It can be seen that the process of determining the weak target detection result involves the weak target detection network, 4 HDCs, PAN and 4 CBAMs. Specifically, the electronic device inputs the remote sensing image to be detected into the weak target detection network and the corresponding 4 HDCs to obtain 4 second feature images; the electronic device inputs the 4 second feature images into the PAN to obtain 4 third feature images; the electronic device then inputs the 4 third feature images into the corresponding CBAMs respectively to obtain 4 fourth feature images; finally, the electronic device determines the weak target detection result corresponding to the remote sensing image to be detected based on the 4 fourth feature images in combination with non-maximum suppression.

[0085] For example, Fig.11a It is one of the schematic diagrams of the small target detection result provided by the present invention; Fig.11b This is the second schematic diagram of the small target detection result provided by the present invention; Fig.11c This is the third schematic diagram of the small target detection result provided by the present invention; Fig.11d FIG. 4 is a fourth schematic diagram of the small target detection result provided by the present invention. Figures 11a to 11d As shown, the feature information of each detection result graph is richer and the recall rate of target detection is high.

[0086] In addition, for the predicted rotation rectangle in the fourth feature image (such as Fig.11e As shown), it is defined as (x, y, w, h, θ). Among them, (x, y) represents the coordinates of the center point of the predicted rotated rectangle; w represents the lower right corner vertex of the predicted rotated rectangle as the origin, and the first edge encountered when rotating upward in the positive direction of the x-axis is not necessarily the long side of the predicted rotated rectangle, but may also be the short side. After determining w, the other side is h, and the rotation angle is θ, with a value range of [-90°, 0°).

[0087] In order to reduce the difficulty of fitting the weak target detection network, the rotating anchor point is usually used to fit the real rotating rectangular frame regression. The specific fitting method is as follows: Fig.11fAs shown. Fig.11f It can be seen that the blue dotted line represents the best anchor point corresponding to the grid on the last layer of feature image , that is, the anchor point with the largest Intersection over Union (IoU) with the real rotated rectangle among the three preset anchor points. The red rectangle is the real rotated rectangle. The grid coordinates are , the offset of the center point is ( , ). The purpose of real rotation rectangle regression is to learn arrive Mapping, offset ( , ), and the rotation angle θ'.

[0088] In summary, steps 101 to 104 add a feature enhancement module (i.e., deep residual network) and shallow cross-scale feature fusion based on the structure of the YOLOv4 network. The feature pyramids of SPP and PAN are used in the YOLOv4 network to enhance the detection effect. The output feature images are downsampled by 8 times, 16 times, and 32 times, respectively. That is, when the detected target is less than 8 pixels × 8 pixels, it will be difficult to detect the detected target in the final output feature image. In order to make more use of weak and small target information, the following improvements are made: first, the feature image that has been downsampled once is superimposed on the input end of each residual network to increase the detail information in the deep feature image; then, the feature enhancement module is connected after the 76×76 feature image to enhance the ability to extract small target information in the feature image of this size. The feature enhancement module replaces the 5×5 convolution layer with two 3×3 continuous convolutions, because the area occupied by weak and small targets in the remote sensing image to be detected is originally small. After multiple scale reductions, the size in the feature image becomes smaller, and the 5×5 convolution result is too large. Using two continuous convolutions can not only meet the feature extraction of weak and small targets, but also reduce the number of parameters. The feature image first passes through a multi-branch structure composed of convolution kernels of different sizes, and then increases the receptive field through HDC. Finally, the output of the convolution layer of different sizes is fused to achieve the purpose of fusing different features.

[0089] In addition, although the YOLOv4 network has improved the detection capability of small and weak targets through feature fusion, there are still some problems that have not been solved. The feature image of the 8-fold downsampling output used to detect small and weak targets in the YOLOv4 network has limited ability to obtain the position information of small and weak targets without fusing the feature image of the shallow network. Therefore, when the detected target is less than 8 pixels × 8 pixels, the detection accuracy will be significantly reduced, resulting in the problem of missed detection of small and weak targets in the target area where the target area is closely arranged. In order to allow the small and weak target detection network to obtain more feature information of small and weak targets and improve the detection accuracy of small and weak targets, the original 3-layer detection layer in the CSPDarknet53 network can be expanded to 4-layer detection layer, and the original P3, P4, and P5 effective feature layers are changed to P2, P3, P4, and P5. The newly constructed and expanded P2 layer inherits the P3, P4, and P5 layers to form a new Neck layer. The number of newly expanded detection layer grids is 152 × 152. The actual receptive field corresponding to each grid is 4 pixels. Based on this, for small targets, prediction can be made through the 4th detection layer, which greatly improves the detection performance of small targets.

[0090] In an embodiment of the present invention, the remote sensing image to be detected is input into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; through the convolution block, feature extraction is performed on the remote sensing image to be detected to obtain a first image; and through the first deep residual network, feature capture is performed on the first image to obtain multiple second images; through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of multiple fourth images is the same as the number of multiple second images; according to the multiple second images and the multiple fourth images, the small target detection result corresponding to the remote sensing image to be detected is determined. This method improves the detection accuracy of small targets by extracting and processing the dual features of small targets in the remote sensing image to be detected through the CSPDarknet53 network and the Focus network.

[0091] In order to better understand the embodiments of the present invention, the training process of the small target detection network is described in detail below.

[0092] It should be noted that in the process of training the small target detection network, the input data is the target remote sensing image sample, and the output data is the predicted small target detection result.

[0093] Optionally, the target remote sensing image samples are determined based on the following steps: obtaining initial remote sensing image samples (such as optical image samples and SAR image samples); and performing augmentation processing on the initial remote sensing image samples to obtain target remote sensing image samples.

[0094] Optical image samples have problems such as scale diversity, special viewing angle (i.e., remote sensing shooting angle is usually a bird's-eye view), multi-directionality, background complexity, and strong interference. In addition, for the task of detecting small and weak targets, small and weak targets are also limited by the imbalance of optical image samples, strong background interference, and imaging conditions, resulting in poor representability of optical image samples and affecting the final detection and recognition performance of the small and weak target detection network. Therefore, the augmentation processing corresponding to the optical image samples can include: rotation / flip enhancement of the optical image samples, degree of freedom-saturation-brightness (HSV) color perturbation enhancement, CutMix enhancement, and Mosaic enhancement.

[0095] SAR image samples have scattering, phase, geometry and other characteristics, but SAR image samples only have a single band, and because SAR image samples are easily disturbed by imaging conditions, the imaging quality of SAR image samples is poor and the target feature information is not obvious. Therefore, the augmentation processing corresponding to SAR image samples can include: rotation / flip enhancement, grayscale transformation enhancement and Copy-Paste enhancement of SAR image samples.

[0096] For example, Fig.12 It is a schematic diagram of the result of augmenting the initial remote sensing image sample provided by the present invention. The initial remote sensing image sample includes: optical image sample and SAR image sample.

[0097] In order to find the commonality of small and weak targets from a large number of image features, the target detection technology in deep learning neural networks can be used to construct and train the small and weak target detection network. Aiming at the difficulties of small and weak target detection in complex environments, network technology improvements are made, mainly including six points, namely: ① network architecture optimization and implementation based on YOLOv4, ② fine feature extraction of mixed void convolution, ③ background suppression under attention mechanism, ④ false alarm suppression of focus loss and prior knowledge, ⑤ arbitrary direction target detection based on dynamic anchor learning, and ⑥ multi-source and multi-resolution remote sensing image multi-model reasoning. Among them, the network architecture optimization and implementation based on YOLOv4 is the selection and improvement of the basic network algorithm of small and weak target detection technology, and the other five technologies are further improved based on the proposed network architecture.

[0098] It should be noted that the training process of the weak target detection network is the same as the detection application process. The difference is that during the training process, the weak target detection network also needs to determine the loss value, and then based on the loss value, the parameters of the weak target detection network are updated to obtain a trained weak target detection network.

[0099] Among them, the loss value includes focal loss and angle loss.

[0100] Specifically, (i) the process of determining the focal loss is as follows: according to the first formula and the second formula, the target formula is derived, and the target formula is used to determine the binary cross entropy; according to the target formula, hyperparameters, loss coefficients and penalty factors of different sample categories, the focal loss formula is determined, and the focal loss formula is used to determine the focal loss.

[0101] Among them, the first formula is: ; The second formula is: ; The target formula is: .

[0102] represents binary cross entropy; ∈{±1} represents the actual category; ∈[0,1] represents the probability estimation of the sample category by the weak target detection network; represents the target probability estimate.

[0103] The focal loss formula is: .

[0104] indicates focal loss; Represents the sum of loss coefficients of different sample categories; represents a hyperparameter; Represents the penalty factor.

[0105] in, , Indicates the number of categories of samples; Indicates The category of the samples; Represents the amount of variation or difference across all sample categories; Represents a weight coefficient between [0,1], which is used to adjust the ratio of the focus loss value to the improved focus loss value in each focus loss function calculation process; Represents time (or number of iterations, steps, etc.) The relevant variable or parameter.

[0106] , =1 means the first The target class is a positive sample. =0 means the first The class target is a negative sample; Represents the number of positive sample anchor boxes for the current input image sample, It represents the number of negative sample anchor boxes for the current input image sample.

[0107] It should be noted that the factor The degree of attenuation of the probability of negative samples can be controlled. When the detection result of weak targets is correct, Approaching 0, when the weak target detection result is wrong, Approaching 1.

[0108] The purpose of the above focal loss value is to increase the loss of difficult samples and reduce the loss of easy-to-classify simple samples, so that the weak object detection network can focus more on learning difficult samples, thereby achieving detection performance comparable to that of two-stage detection.

[0109] (ii) The process of determining the angle loss is as follows: Determine the angle loss based on the angle loss function.

[0110] Among them, the angle loss function is:

[0111] .

[0112] Indicates angle loss; represents the angle rotation coefficient; Representation Grid No. An indicator variable for whether the target exists in the prediction box; Indicates the optimal anchor point position; Indicates the initial position of the anchor point. Represents the total number of grids, Indicates the total number of prediction boxes.

[0113] During the actual training process, it was found that the angle output was not stable enough. After analyzing the angle loss function, it was found that since the original anchor point did not carry angle information, which was equivalent to 0°, the original anchor point was directly predicted to have an angle information of [-90°, 0°). The span was too large and the weak target detection network was difficult to converge. Therefore, the learning interval was shortened and the original anchor point was predicted from -45°. The prediction interval was [-45°, 0°) and [0°, 45°). The original learning interval was halved to reduce the learning difficulty and define the above angle loss function.

[0114] The small target detection device provided by the present invention is described below. The small target detection device described below and the small target detection method described above can be referred to each other.

[0115] Fig.13 : is a schematic diagram of the structure of the small target detection device provided by the present invention, such as Fig.13 As shown, the device includes: a data processing module 1301 and a target detection module 1302.

[0116] The data processing module 1301 is used to input the remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; through the convolution block, feature extraction is performed on the remote sensing image to be detected to obtain a first image; and through the first deep residual network, feature capture is performed on the first image to obtain multiple second images; through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images.

[0117] The target detection module 1302 is used to determine the small target detection result corresponding to the remote sensing image to be detected according to the multiple second images and the multiple fourth images.

[0118] Optionally, the first deep residual network includes: a first-layer residual network and multiple first residual networks, the first-layer residual network is connected in series with the first first residual network, and the multiple first residual networks are connected in series; a data processing module 1301 is specifically used to extract features of the first image through the first-layer residual network to obtain a first feature image; through each first residual network, according to the first image and the first target image, determine the corresponding second image; wherein the first target image corresponding to the first first residual network is the first feature image; the first target images corresponding to the remaining first residual networks are the images output by the first residual network of the previous adjacent layer.

[0119] Optionally, the second deep residual network includes: multiple second residual networks, which correspond one-to-one to and are the same as the multiple first residual networks; a data processing module 1301, specifically used to extract features of the second target image through each second residual network to obtain a corresponding fourth image; wherein the second target image corresponding to the first second residual network is the third image; and the second target images corresponding to the remaining second residual networks are images output by the second residual network of the previous adjacent layer.

[0120] Optionally, the target detection module 1302 is specifically used to, for the same first residual network and second residual network, perform splicing and feature extraction on the second image corresponding to the first residual network and the fourth image corresponding to the second residual network through mixed void convolution to obtain a second feature image; and determine the weak target detection result corresponding to the remote sensing image to be detected based on multiple second feature images.

[0121] Optionally, the data processing module 1301 is specifically used to superimpose and fuse the first image and the first target image for each first residual network to obtain a fused image; and use the first residual network to continuously extract features of the fused image according to a first preset number of times to obtain the corresponding second image.

[0122] Optionally, the target detection module 1302 is specifically used to splice the second image corresponding to the first residual network and the fourth image corresponding to the second residual network for the last identical first residual network and the second residual network; obtain a spliced ​​image, and continuously extract features of the spliced ​​image according to a second preset number of times to obtain a corresponding second feature image.

[0123] Optionally, the convolution block includes: a two-dimensional convolution layer, batch normalization and a non-linear activation function; the data processing module 1301 is specifically used to perform feature extraction on the remote sensing image to be detected through the two-dimensional convolution layer to obtain a first sub-image; normalize the first sub-image through the batch normalization to obtain a second sub-image; and enhance the features of the second sub-image through the non-linear activation function to obtain the first image.

[0124] Fig.14 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig.14As shown, the electronic device may include: a processor (processor) 1410 , a communication interface (Communications Interface) 1420 , a memory (memory) 1430 and a communication bus 1440 , wherein the processor 1410 , the communication interface 1420 , and the memory 1430 communicate with each other through the communication bus 1440 . The processor 1410 can call the logic instructions in the memory 1430 to execute a small target detection method, which includes: inputting a remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; through the convolution block, feature extraction is performed on the remote sensing image to be detected to obtain a first image; and through the first deep residual network, feature capture is performed on the first image to obtain multiple second images; through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images; according to the multiple second images and the multiple fourth images, the small target detection result corresponding to the remote sensing image to be detected is determined.

[0125] In addition, the logic instructions in the above-mentioned memory 1430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0126] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the small target detection method provided by the above methods, and the method includes: inputting the remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a first Two deep residual networks; through the convolution block, the features of the remote sensing image to be detected are extracted to obtain a first image; and through the first deep residual network, the features of the first image are captured to obtain multiple second images; through the Focus layer, the channel processing of the remote sensing image to be detected is obtained to obtain a third image; and through the second deep residual network, the features of the third image are captured to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images; according to the multiple second images and the multiple fourth images, the weak target detection result corresponding to the remote sensing image to be detected is determined.

[0127] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the small and weak target detection method provided by the above-mentioned methods when executed, the method comprising: inputting the remote sensing image to be detected into a small and weak target detection network, wherein the small and weak target detection network comprises: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network comprises: a convolution block and a first deep residual network, and the Focus network comprises: a Focus layer and a second deep residual network; performing feature extraction on the remote sensing image to be detected through the convolution block to obtain a first image; and performing feature capture on the first image through the first deep residual network to obtain multiple second images; performing channel processing on the remote sensing image to be detected through the Focus layer to obtain a third image; and performing feature capture on the third image through the second deep residual network to obtain multiple fourth images, the number of the multiple fourth images being the same as the number of the multiple second images; determining the small and weak target detection result corresponding to the remote sensing image to be detected according to the multiple second images and the multiple fourth images.

[0128] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A small target detection method, characterized in that: include: Inputting the remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; By using the convolution block, the remote sensing image to be detected is subjected to feature extraction to obtain a first image; and by using the first deep residual network, the first image is subjected to feature capture to obtain a plurality of second images; Through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain a plurality of fourth images, where the number of the plurality of fourth images is the same as the number of the plurality of second images; Determining a small target detection result corresponding to the remote sensing image to be detected according to the plurality of second images and the plurality of fourth images; The step of determining the small target detection result corresponding to the remote sensing image to be detected according to the plurality of second images and the plurality of fourth images includes: For the same first residual network and second residual network, by mixed dilated convolution, a second image corresponding to the first residual network and a fourth image corresponding to the second residual network are spliced ​​and feature extracted to obtain a second feature image; Determining a small target detection result corresponding to the remote sensing image to be detected according to the plurality of second feature images; The step of stitching and extracting features from the second image corresponding to the first residual network and the fourth image corresponding to the second residual network to obtain a second feature image includes: For the last identical first residual network and second residual network, stitching the second image corresponding to the first residual network and the fourth image corresponding to the second residual network; obtaining a stitched image, and continuously extracting features from the stitched image according to a second preset number of times to obtain a corresponding second feature image; Determining the small target detection result corresponding to the remote sensing image to be detected according to the multiple second feature images includes: Inputting the plurality of second feature images into a path aggregation network to obtain a plurality of third feature images, wherein the number of the third feature images is the same as the number of the second feature images; For each third feature image, input the third feature image into a convolutional attention mechanism module to obtain a fourth feature image; Based on the predicted rotated rectangular box in the fourth feature image and combined with non-maximum suppression, the small target detection result is determined.

2. The small target detection method according to claim 1, characterized in that: The first deep residual network includes: a first-layer residual network and multiple first residual networks, the first-layer residual network is connected in series with the first first residual network, and the multiple first residual networks are connected in series; the first image is feature captured by the first deep residual network to obtain multiple second images, including: Performing feature extraction on the first image through the first-layer residual network to obtain a first feature image; Determine, through each first residual network, a corresponding second image according to the first image and the first target image; Among them, the first target image corresponding to the first first residual network is the first feature image; the first target images corresponding to the remaining first residual networks are the images output by the first residual network of the previous adjacent layer.

3. The small target detection method according to claim 2, characterized in that: The second deep residual network includes: a plurality of second residual networks, the plurality of second residual networks correspond to and are identical to the plurality of first residual networks one by one; the third image is captured by the second deep residual network to obtain a plurality of fourth images, including: Through each second residual network, feature extraction is performed on the second target image to obtain a corresponding fourth image; Among them, the second target image corresponding to the first second residual network is the third image; the second target images corresponding to the remaining second residual networks are the images output by the second residual network of the previous adjacent layer.

4. The small target detection method according to claim 2, characterized in that: The determining, through each first residual network, a corresponding second image according to the first image and the first target image includes: For each of the first residual networks, superimposing and fusing the first image and the first target image to obtain a fused image; The first residual network is used to continuously extract features of the fused image according to a first preset number of times to obtain the corresponding second image.

5. The small target detection method according to any one of claims 1 to 4, characterized in that: The convolution block includes: a two-dimensional convolution layer, batch normalization and a nonlinear activation function; the convolution block is used to extract features of the remote sensing image to be detected to obtain a first image, including: Through the two-dimensional convolution layer, feature extraction is performed on the remote sensing image to be detected to obtain a first sub-image; Normalizing the first sub-image by the batch normalization to obtain a second sub-image; The second sub-image is feature enhanced by the nonlinear activation function to obtain the first image.

6. A device for implementing the small target detection method according to any one of claims 1 to 5, characterized in that: include: A data processing module, used for inputting the remote sensing image to be detected into a small target detection network, wherein the small target detection network includes: a CSPDarknet53 network and a Focus network, the CSPDarknet53 network includes: a convolution block and a first deep residual network, and the Focus network includes: a Focus layer and a second deep residual network; through the convolution block, feature extraction is performed on the remote sensing image to be detected to obtain a first image; and through the first deep residual network, feature capture is performed on the first image to obtain multiple second images; through the Focus layer, channel processing is performed on the remote sensing image to be detected to obtain a third image; and through the second deep residual network, feature capture is performed on the third image to obtain multiple fourth images, and the number of the multiple fourth images is the same as the number of the multiple second images; The target detection module is used to determine the small target detection result corresponding to the remote sensing image to be detected according to the multiple second images and the multiple fourth images.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the small target detection method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the small target detection method according to any one of claims 1 to 5 is implemented.

Images