Image small target detection method and device, electronic equipment and storage medium

Abstract

The invention provides an image small target detection method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a to-be-detected image; inputting the to-be-detected image into a target detection model to obtain a target detection result output by the target detection model; wherein the target detection model comprises a feature extraction network, the feature extraction network comprises a stacking fusion convolution block and a convolution attention module, and the stacking fusion convolution block is used for performing channel-by-channel convolution and point-by-point convolution on the to-be-detected image to generate a first feature map; and the convolution attention module is used for carrying out attention weight integration on a second feature map in a channel dimension and a space dimension to obtain an enhanced feature map, and the second feature map is obtained based on the first feature map. According to the invention, more and more detailed feature information can be extracted from the image, and feature loss is avoided, so that the target detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a method, device, electronic equipment and storage medium for detecting small objects in an image. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images have become an indispensable data source for scientific research and application fields. However, with the significant improvement in the resolution of remote sensing images, the amount of information contained in each image has exploded, posing a severe challenge to traditional automated parsing and classification technologies. Even more challenging is that even if the functions and uses of the targets are completely different, they may show indistinguishable appearance similarities in remote sensing images, a phenomenon that greatly blurs the classification boundaries.

[0003] In addition, the environment of remote sensing images is complex and changeable, and the background is complicated. Not only are there a large number of small target objects and they are densely distributed, but the target scales vary significantly. These factors interweave and further increase the difficulty of target detection and recognition. Therefore, how to achieve efficient and accurate similar target recognition technology has become a technical problem that needs to be solved urgently in the current remote sensing image field. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for detecting small targets in an image, which are used to solve the defects of the prior art in that the detection accuracy is insufficient due to complex background, dense targets and large differences in target scales.

[0005] The present invention provides a method for detecting small targets in an image, comprising: Acquire the image to be detected; Inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model; Among them, the target detection model includes a feature extraction network, which includes a stacked fusion convolution block and a convolution attention module. The stacked fusion convolution block is used to perform channel-by-channel and point-by-point convolution on the image to be detected to generate a first feature map. The convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map. The second feature map is obtained based on the first feature map.

[0006] According to a method for detecting small targets in an image provided by the present invention, the feature extraction network includes a plurality of feature extraction modules connected in series, and the image to be detected is input into a target detection model to obtain a target detection result output by the target detection model, including: Based on each feature extraction module in the feature extraction network, feature extraction and enhancement are performed on the image to be detected to obtain an enhanced feature map; Based on the feature fusion network of the target detection model, multi-scale feature fusion is performed on the enhanced feature map to obtain a fused feature map; Based on the detection module of the target detection model, the fused feature map is applied to perform target detection to obtain the target detection result.

[0007] According to a method for detecting small targets in an image provided by the present invention, any feature extraction module includes a stacked fusion convolution block, a bottleneck module, a splicing module, and a convolution attention module. The feature extraction modules in the feature extraction network are used to extract and enhance the features of the image to be detected to obtain an enhanced feature map, including: Based on the stacked fusion convolution block of any one of the feature extraction modules, perform channel-by-channel convolution and point-by-point convolution on the input feature map to obtain a first feature map, where the input feature map is determined based on the image to be detected or an enhanced feature map output by a previous feature extraction module; Based on the bottleneck module of any one of the feature extraction modules, sequentially compress, extract features and expand the first feature map to obtain a processed feature map; Based on the splicing module of any one of the feature extraction modules, the input feature map and the processed feature map are spliced ​​to obtain a second feature map; Based on the convolutional attention module of any of the feature extraction modules, the attention weights of the second feature map are integrated in the channel dimension and the spatial dimension to obtain the enhanced feature map.

[0008] According to a method for detecting small targets in an image provided by the present invention, the stacked fusion convolution block based on any one of the feature extraction modules performs channel-by-channel convolution and point-by-point convolution on the input feature map to obtain a first feature map, including: Based on the channel-by-channel convolution unit of the stacked fusion convolution block, a convolution operation is performed on each channel of the input feature map to obtain a feature map to be integrated; Based on the point-by-point convolution unit of the stacked fusion convolution block, information between channels of the feature map to be integrated is integrated to obtain the first feature map.

[0009] According to a small target detection method in an image provided by the present invention, the convolution attention module based on any one of the feature extraction modules performs attention weight integration on the second feature map in the channel dimension and the spatial dimension to obtain the enhanced feature map, including: Based on the channel attention unit of the convolutional attention module, applying the second feature map to generate each channel weight, and multiplying the each channel weight with the second feature map to obtain a channel weighted feature map; Based on the spatial attention unit of the convolutional attention module, the channel-weighted feature map is applied to generate each spatial position weight, and each spatial position weight is multiplied by the channel-weighted feature map to obtain the enhanced feature map.

[0010] According to a method for detecting small targets in an image provided by the present invention, the feature fusion network includes at least one upsampling link, the upsampling link includes an upsampling prediction module and a feature recombination module, and the step of performing feature upsampling in the upsampling link includes: Based on the upsampling prediction module, the enhanced feature map is compressed in number of channels to obtain a compressed feature map, the compressed feature map is encoded, and the encoded feature map is upsampled and predicted and feature expanded to obtain an extended feature map, and the extended feature map is normalized to obtain a channel upsampling kernel; Based on the feature recombination module, a regional feature map of a preset size is extracted from the enhanced feature map, and a dot product operation is performed between the regional feature map and the channel upsampling kernel to obtain an upsampling feature map.

[0011] According to a method for detecting small targets in an image provided by the present invention, the loss function of the target detection model is a focus normalization loss function, which is determined based on a focus mean square error loss function and a distance metric loss function, the focus mean square error loss function is determined based on a modulation factor and a standard mean square error of each sample, and the distance metric loss function is determined based on the distance between bounding boxes.

[0012] The present invention also provides an image small target detection device, comprising: An acquisition unit, used for acquiring an image to be detected; A detection unit, used to input the image to be detected into a target detection model to obtain a target detection result output by the target detection model; Among them, the target detection model includes a feature extraction network, which includes a stacked fusion convolution block and a convolution attention module. The stacked fusion convolution block is used to perform channel-by-channel and point-by-point convolution on the image to be detected to generate a first feature map. The convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map. The second feature map is obtained based on the first feature map.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the above-described methods for detecting small targets in an image is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the image small target detection method as described in any one of the above is implemented.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned small object detection methods in an image.

[0016] The image small target detection method, device, electronic device and storage medium provided by the present invention can more effectively extract features from the image to be detected by introducing stacked fusion convolution blocks and convolutional attention modules in the feature extraction network of the target detection model, so that the model can more accurately identify the target in the image. The stacked fusion convolution block can reduce the model parameters and the amount of calculation while maintaining the feature extraction capability, and capture the feature information of different scales and types in the image, while the convolutional attention module further integrates the attention weights in the channel dimension and the spatial dimension, so that the model can focus on more critical and useful features in the image, thereby generating enhanced features and improving the accuracy and efficiency of feature extraction. In addition, the design of the stacked fusion convolution block and the convolutional attention module enables the model to retain more detailed information when extracting features, thereby more effectively capturing the features of small targets in the image and improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a schematic diagram of the structure of the YOLOv5s network in the prior art; Figure 2 It is a schematic diagram of the process of the image small target detection method provided by the present invention; Figure 3 It is a schematic diagram of the structure of the stacked fusion convolution block provided by the present invention; Figure 4 It is a structural diagram of the convolutional attention module provided by the present invention; Figure 5 It is a schematic diagram of the structure of cross-level feature fusion provided by the present invention; Figure 6 is a structural schematic diagram of the CARAFE module provided by the present invention; Figure 7It is a flow chart of a remote sensing image small target detection method based on improved YOLOv5s provided by the present invention; Figure 8 It is a structural schematic diagram of the image small target detection device provided by the present invention; Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Deep learning technology is one of the key solutions to the problem of small target detection in remote sensing images. Research in the field of deep learning target detection can be basically divided into two directions: one is a two-stage detector based on candidate regions, such as Faster-RCNN (Faster Region-based Convolutional Neural Network), and the other is a single target detector based on regression calculation, such as YOLO (You Only Look Once). The two directions have their own characteristics. Generally, the two-stage detector has higher accuracy and the single target detector has faster speed. Considering the real-time requirements of small target detection in remote sensing images, the YOLO algorithm with high detection efficiency is more suitable.

[0021] The YOLO series of cutting-edge algorithms have good real-time performance, such as the YOLO series cutting-edge version YOLOv5s, which is suitable for real-time detection of small targets in remote sensing images. Figure 1 It is a schematic diagram of the structure of the YOLOv5s network in the prior art, such as Figure 1As shown in the figure, YOLOv5s is a target detection algorithm based on deep learning. Its network structure is mainly composed of three core modules: backbone, neck and head. Among them, backbone is the backbone network of YOLOv5s, which is mainly responsible for extracting the features of the input image. It processes the input image through a series of convolutional layers and pooling layers, gradually reducing the size of the feature map and increasing the number of channels, thereby retaining and extracting important features in the image. The neck module is located between the backbone and the head. It is mainly responsible for multi-scale feature fusion of the features extracted by the backbone and passing these features to the prediction layer (i.e., the head). The head module is the detection head part of YOLOv5s, which is mainly responsible for the final regression prediction. It uses the feature maps extracted by the backbone and neck modules to detect the location and category of the target.

[0022] Specifically, the backbone module mainly includes structures such as Images, Focus, CBS, C3, and SPP. Among them, Images represents the original image input to the network, and Focus is used to slice the input image or feature map, split the high-resolution image or feature map into multiple low-resolution images or feature maps, and splice them to reduce the amount of calculation and parameters while retaining effective information. CBS refers to the combination of Conv + BN + SiLU, that is, the continuous operation of convolution (Conv), batch normalization (Batch Normalization) and SiLU activation function (a nonlinear activation function). The C3 structure is an important component of the YOLOv5 network. Its main function is to increase the depth and feature extraction ability of the network by adaptively aggregating receptive fields of different sizes. The SPP (Spatial Pyramid Pooling) module is used to improve the model's detection ability for objects of different scales. In the backbone of YOLOv5s, the SPP module can perform pooling operations of different sizes on the input feature map, and connect the pooled feature maps in series for subsequent processing.

[0023] The neck module includes multiple feature fusion layers, each of which can fuse feature maps of different scales through upsampling, downsampling, concatenation, convolution and other operations. The feature fusion layer can include structures such as Upsample, CBS, C3_1, and Concat, where Upsample represents an upsampling operation, C3_1 also represents a C3 structure, but it is somewhat different from the C3 structure in the backbone module, and Concat represents a concatenation operation.

[0024] The head module usually contains multiple detection heads, each of which predicts target information of a specific scale. Figure 1 The Detect shown in the figure represents the detection head. The scale of the target information output by each Detect is different. For example, for the first detection head (Detect: [1,3,80,80,85]), the first dimension represents the batch size, that is, how many images are processed at a time, the second dimension represents the number of anchor boxes predicted by each grid unit, the third and fourth dimensions together represent the size of the feature map (i.e., height and width), and the fifth dimension represents the number of predicted information for each anchor box. In YOLOv5s, the model improves the detection ability of objects of different sizes through multi-scale prediction. Figure 1 The three detection heads shown in the figure correspond to feature maps of different scales, so that objects of different sizes can be detected. Among them, larger feature maps (such as 80×80) are more suitable for detecting smaller objects, while smaller feature maps (such as 20×20) are more suitable for detecting larger objects.

[0025] However, when YOLOv5s is applied to small target detection in remote sensing images, it is found that the detection accuracy is not high. In this regard, the present invention provides a small target detection method for remote sensing images based on improved YOLOv5s, which can solve the problem of insufficient detection accuracy caused by complex backgrounds, dense targets, and large differences in target scales. The specific improvement strategies are as follows: First, the backbone network of the existing YOLOv5s only uses ordinary convolution to extract features from feature maps, so the extracted features are limited and the feature information is not fully extracted. The improved YOLOv5s of the present invention uses stacked fusion convolution blocks and lightweight convolutional attention modules to reconstruct the C3 structure in the backbone network, while reducing the number of parameters, maintaining the feature extraction capability, and splicing with the fusion layer features of the neck module to obtain more and more detailed feature information.

[0026] Second, in the original YOLOv5s, the feature fusion layer of the neck module will cause the loss of feature information and small target position information during downsampling and network forward propagation. The present invention introduces cross-level feature fusion to increase the feature information propagation path, and combines the upsampling operator CARAFE to enhance content perception, making full use of shallow information for feature fusion, thereby effectively avoiding feature loss.

[0027] Third, the positioning loss function in the original YOLOv5s cannot effectively handle the problem of class imbalance, it is difficult to focus on samples that are difficult to classify, and it is too sensitive to the geometric overlap between the predicted box and the real box, which affects the positioning accuracy. The present invention improves the positioning loss function and introduces a focused normalization loss function. It can solve the problem of class imbalance by adjusting the sample weights, improve the model's learning ability for difficult samples, and reduce the sensitivity to geometric overlap, thereby improving detection accuracy. The above three improvements will be specifically introduced below.

[0028] Figure 2 It is a flow chart of the image small target detection method provided by the present invention, such as Figure 2 As shown, the method includes: Step 210, obtaining an image to be detected; Specifically, the image to be detected refers to the original image that needs to be processed for target detection, which can contain multiple targets of different categories, including large targets and small targets. The image to be detected can be image data from various sources, such as video frames captured by surveillance cameras, satellite remote sensing images, etc.

[0029] Step 220, inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model; Among them, the target detection model includes a feature extraction network, which includes a stacked fusion convolution block and a convolution attention module. The stacked fusion convolution block is used to perform channel-by-channel and point-by-point convolution on the image to be detected to generate a first feature map. The convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map. The second feature map is obtained based on the first feature map.

[0030] Specifically, after obtaining the image to be detected, the image to be detected is input into the target detection model, and the target detection result can be obtained by performing target detection on the image to be detected through the model. Here, the target detection model is a machine learning model that is specifically used to identify the target object in the image and give the position and category of each target. For example, the target detection model can be an improved YOLOv5s model.

[0031] Specifically, the target detection model includes a feature extraction network, which is an important component of the target detection model and is responsible for extracting useful feature information from the input image. In an embodiment of the present invention, the feature extraction network is optimized by introducing stacked fusion convolution blocks and convolution attention modules to better meet the needs of small target detection. Here, the stacked fusion convolution block is a special network structure that gradually extracts feature information in the image by stacking multiple convolution layers (including channel-by-channel convolution and point-by-point convolution). This structure can capture features of different scales and types in the image and effectively integrate these features together through a fusion mechanism. In an embodiment of the present invention, the stacked fusion convolution block is used to generate a first feature map to provide rich feature information for subsequent target detection tasks. Here, the first feature map refers to the preliminary feature representation after processing by the stacked fusion convolution block.

[0032] It can be understood that channel-by-channel convolution is a special convolution method that performs convolution operations on each channel of the input feature map independently, instead of mixing the information of multiple channels together like standard convolution. This convolution method can reduce the amount of calculation and retain more channel information, which is conducive to subsequent feature extraction and fusion. Point-by-point convolution is also called 1×1 convolution. Its convolution kernel size is 1×1. It is used to change the number of channels of the feature map or perform linear combination of features to achieve the integration of different channel features.

[0033] The convolutional attention module is an attention mechanism for enhancing feature representation. It integrates the attention weights of the feature map in the channel dimension and the spatial dimension, so that the model can automatically learn and identify which channels and spatial positions in the feature map are more important for the target detection task, thereby giving higher weights to these important channels and spatial positions to enhance their representation capabilities. In an embodiment of the present invention, the convolutional attention module is used to process the second feature map to generate an enhanced feature map, thereby improving the accuracy of target detection. Here, the second feature map is obtained by performing corresponding processing on the first feature map output by the stacked fusion convolution block. It should be understood that the enhanced feature map refers to the feature map processed by the convolutional attention module, which is more prominent and accurate in the representation of key information than the original feature map, and is therefore more conducive to subsequent target detection tasks and can improve the accuracy and efficiency of detection.

[0034] The method provided by the embodiment of the present invention can more effectively extract features from the image to be detected by introducing stacked fusion convolution blocks and convolutional attention modules into the feature extraction network of the target detection model, so that the model can more accurately identify the target in the image. The stacked fusion convolution blocks can reduce model parameters and computational complexity while maintaining feature extraction capabilities and capturing feature information of different scales and types in the image, while the convolutional attention module further integrates attention weights in the channel dimension and the spatial dimension, so that the model can focus on more critical and useful features in the image, thereby generating enhanced features and improving the accuracy and efficiency of feature extraction. In addition, the design of the stacked fusion convolution blocks and the convolutional attention module enables the model to retain more detail information when extracting features, thereby more effectively capturing the features of small targets in the image and improving detection accuracy.

[0035] Based on the above embodiment, the feature extraction network includes a plurality of feature extraction modules connected in series. Accordingly, step 220 specifically includes: Step 221, based on each feature extraction module in the feature extraction network, feature extraction and enhancement are performed on the image to be detected to obtain an enhanced feature map; Step 222, based on the feature fusion network of the target detection model, performing multi-scale feature fusion on the enhanced feature map to obtain a fused feature map; Step 223, based on the detection module of the target detection model, the fused feature map is applied to perform target detection to obtain the target detection result.

[0036] It should be noted that the target detection model may include a feature extraction network, a feature fusion network and a detection module. For example, when the target detection model is an improved YOLOv5s model, the feature extraction network is the backbone module, the feature fusion network is the neck module, and the detection module is the head module.

[0037] Specifically, the feature extraction network may include multiple serially connected feature extraction modules (i.e., the reconstructed C3 structure), which are responsible for extracting useful feature information from the input image. Each feature extraction module includes a stacked fusion convolution block and a convolution attention module, which can extract and enhance features of the image to be detected to extract more detailed feature information, thereby facilitating subsequent target detection tasks.

[0038] After obtaining the enhanced feature maps output by the feature extraction network, these feature maps will be sent to the feature fusion network for multi-scale feature fusion. The feature fusion network will integrate feature maps from different levels, which have different receptive fields and resolutions, and can capture target information of different scales, providing richer information for subsequent target detection.

[0039] Finally, based on the fused feature map, the detection module applies a series of operations to detect the object in the image and outputs the object detection results, including the object category, location (such as bounding box coordinates), and confidence level. The detection module can include multiple detection heads, each of which is used to predict object information of a specific scale.

[0040] Based on any of the above embodiments, any feature extraction module includes a stacked fusion convolution block, a bottleneck module, a splicing module and a convolutional attention module.

[0041] It should be noted that in the original YOLOv5s backbone network, the C3 structure is an important feature extraction module, which usually includes an initial 1×1 convolution layer, multiple Bottleneck modules, residual connections, and a final 1×1 convolution layer. Among them, the initial 1×1 convolution layer is used to perform preliminary processing on the input feature map; multiple Bottleneck modules are arranged in sequence to perform multiple compression, feature extraction, and expansion operations on the feature map; in the C3 structure, the residual connection is used to add the input feature map to the feature map processed by multiple Bottleneck modules; the final 1×1 convolution layer is used to perform final processing on the output of the C3 structure to generate a feature map for subsequent detection or classification tasks. However, since the original C3 structure only uses ordinary convolution to extract features from feature maps, the extracted features are limited and feature information is not fully extracted.

[0042] In this regard, the embodiment of the present invention improves the original YOLOv5s backbone network and reconstructs the C3 structure of the backbone network, that is, stacked fused convolution blocks are used to replace all 1×1 convolutions in the original C3 structure, and at the same time, after splicing the Bottleneck modules of the original C3 structure, a convolutional block attention module (CBAM) is introduced. The improved C3 structure (i.e., feature extraction module) includes stacked fused convolution blocks, multiple bottleneck modules, splicing modules (i.e., residual connections), and convolutional attention modules.

[0043] Based on any of the above embodiments, the step of performing feature extraction by each feature extraction module (ie, the above step 221) includes: Step 2211, based on the stacked fusion convolution block of any of the feature extraction modules, performing channel-by-channel convolution and point-by-point convolution on the input feature map to obtain a first feature map, wherein the input feature map is determined based on the image to be detected or the enhanced feature map output by the previous feature extraction module; Specifically, the stacked fusion convolution block is formed by stacking channel-by-channel convolution and point-by-point convolution, and is used to perform channel-by-channel convolution and point-by-point convolution on the input feature map to obtain a first feature map. Here, the input feature map is determined based on the image to be detected or the enhanced feature map output by the previous feature extraction module. For example, Figure 1 As shown, in the feature extraction network backbone, it can include multiple feature extraction modules (i.e., the improved C3 structure). For the first feature extraction module, its input feature map is obtained after the corresponding processing of the image to be detected, and for the second feature extraction module, its input feature map can be obtained by processing the enhanced feature map output by the first feature extraction module through the CBS module.

[0044] Furthermore, step 2211 specifically includes: Based on the channel-by-channel convolution unit of the stacked fusion convolution block, a convolution operation is performed on each channel of the input feature map to obtain a feature map to be integrated; Based on the point-by-point convolution unit of the stacked fusion convolution block, information between channels of the feature map to be integrated is integrated to obtain the first feature map.

[0045] Figure 3 : is a schematic diagram of the structure of the stacked fusion convolution block provided by the present invention, such as Figure 3 As shown in the figure, the stacked fusion convolution block is composed of channel-by-channel convolution and point-by-point convolution. First, each channel of the input feature map is convolved separately to generate a feature map with the same number of input channels. Then, the point-by-point convolution stage uses a 1×1 convolution kernel to integrate the information between channels of the output of the channel-by-channel convolution to generate the final output feature map. This stacked fusion strategy reduces the model parameters and computational complexity while maintaining feature extraction capabilities by separating the calculation of spatial dimensions and channel dimensions.

[0046] Step 2212: based on the bottleneck module of any one of the feature extraction modules, sequentially perform compression, feature extraction and expansion processing on the first feature map to obtain a processed feature map; Specifically, each feature extraction module includes multiple Bottleneck modules, which are arranged in sequence and used to perform multiple compression, feature extraction and expansion operations on the feature map. Specifically, in the compression stage, the Bottleneck module first reduces the number of channels of the feature map through a 1×1 convolution layer, aiming to reduce the computational complexity of subsequent operations; then, in the transformation stage, the module uses a 3×3 convolution layer for feature extraction, which can capture local information in the image while keeping the size of the feature map unchanged; finally, in the expansion stage, the module again restores the number of channels of the feature map to the original size or the required size through a 1×1 convolution layer, which helps to maintain the expressive power of the network without increasing too much computation.

[0047] Step 2213, based on the splicing module of any feature extraction module, splice the input feature map and the processed feature map to obtain a second feature map; Specifically, the concatenation module here can be implemented by residual connection, through which the input feature map and the feature map processed by multiple Bottleneck modules can be concatenated to obtain a second feature map. Here, the second feature map is the feature map output by the concatenation module.

[0048] Step 2214: Based on the convolutional attention module of any of the feature extraction modules, the attention weights of the second feature map are integrated in the channel dimension and the spatial dimension to obtain the enhanced feature map.

[0049] Specifically, after the Bottleneck module is spliced, the convolutional attention module CBAM is introduced. This module combines the channel attention unit CAM (Channel Attention Module) and the spatial attention unit SAM (Spatial Attention Module). After that, the attention weights of the channel and spatial dimensions are integrated and multiplied with the original features to output the enhanced features. .

[0050] Furthermore, step 2214 specifically includes: Based on the channel attention unit of the convolutional attention module, applying the second feature map to generate each channel weight, and multiplying the each channel weight with the second feature map to obtain a channel weighted feature map; Based on the spatial attention unit of the convolutional attention module, the channel-weighted feature map is applied to generate each spatial position weight, and each spatial position weight is multiplied by the channel-weighted feature map to obtain the enhanced feature map.

[0051] Figure 4It is a schematic diagram of the structure of the convolutional attention module provided by the present invention. Figure 4 As shown in the figure, in the convolutional attention module CBAM, the channel attention unit CAM and the spatial attention unit SAM play different roles to enhance the convolutional neural network's ability to learn and express image features. The main function of the channel attention unit CAM is to enable the network to automatically select the feature channel that is most useful for the current task. By weighting different channels, CAM can emphasize important feature channels and suppress unimportant channels, thereby improving the network's ability to distinguish different categories and its ability to express features. Specifically, CAM performs a feature map on the input. Perform operations such as global pooling to extract the global information of each channel, and use this information to generate an attention weight for each channel. By multiplying these weights with the corresponding channels of the original feature map, weighting of different channels can be achieved, that is, feature enhancement in the channel dimension can be achieved.

[0052] The spatial attention unit SAM is responsible for enabling the network to automatically focus on the most important areas of the image for the current task. By weighting different spatial positions, SAM can emphasize important image areas and suppress unimportant areas, thereby improving the network's ability to locate targets and express areas of interest. Specifically, SAM first further processes the feature map weighted by channel attention to extract key information in spatial positions and generate a spatial attention map. By multiplying this attention map with the original input feature map (or the feature map weighted by channel attention), it is possible to weight different spatial positions, that is, to achieve feature enhancement in the spatial dimension.

[0053] In the embodiment of the present invention, by integrating attention weights in the channel dimension and the spatial dimension, the convolutional attention module can refine the expression of features and improve the prediction ability of the model. This attention mechanism enables the model to capture key information more accurately when dealing with tasks such as complex background and small target detection, thereby improving the accuracy and robustness of detection.

[0054] Based on any of the above embodiments, the present invention introduces cross-level feature fusion, and adds at least one additional upsampling link in the feature fusion network (ie, the neck module) to fuse features of different layers in order to more comprehensively understand the content. Figure 5 is a schematic diagram of the structure of the cross-level feature fusion provided by the present invention, such as Figure 5 As shown, B3, B4, and B5 in the figure are respectively Figure 1 The corresponding modules in the backbone shown in the figure correspond to each other, where B3 corresponds to the second C3 structure from top to bottom in the backbone module, B4 corresponds to the third C3 structure, and B5 corresponds to the SPP module. Figure 5L3, L4, and L5 in Figure 1 As shown in the figure, P3, P4, and P5 correspond to the modules on the right side of the neck. L3 corresponds to the Concat module at the top left of the neck module, L4 corresponds to the first CBS module from top to bottom on the left, and L5 corresponds to the CBS module at the bottom left. P3 corresponds to the C3_1 structure at the top right of the neck module, P4 corresponds to the C3_1 structure in the middle of the right side, and P5 corresponds to the C3_1 structure at the bottom right side. Figure 5 As shown, the embodiment of the present invention adds two additional upsampling links (shown by two dotted lines in the figure) to the neck module, and in the process of feature upsampling, uses the upsampling operator CARAFE to obtain an improved neck module.

[0055] Figure 6 : is a schematic diagram of the structure of the CARAFE module provided by the present invention, such as Figure 6 As shown, in the feature upsampling link, the upsampling operator CARAFE is applied, that is, the upsampling link includes a CARAFE module, which is composed of an upsampling prediction module and a feature reorganization module. Specifically, the steps of performing feature upsampling in the upsampling link include: Based on the upsampling prediction module, the enhanced feature map is compressed in number of channels to obtain a compressed feature map, the compressed feature map is encoded, and the encoded feature map is upsampled and predicted and feature expanded to obtain an extended feature map, and the extended feature map is normalized to obtain a channel upsampling kernel; Based on the feature recombination module, a regional feature map of a preset size is extracted from the enhanced feature map, and a dot product operation is performed between the regional feature map and the channel upsampling kernel to obtain an upsampling feature map.

[0056] Specifically, in the upsampling kernel prediction module, the amount of computation is first reduced by compressing the number of channels of the feature map, that is, the number of channels of the input feature map is compressed from C to C. 1 ; Then use the K×K size convolutional layer to encode the compressed feature map; then perform α times upsampling prediction on the encoded feature map and expand it to αH×αW×K 2 Finally, the upsampling kernels on all channels are normalized. In the feature reorganization module, a K×K region is taken from the feature map of the original input and a dot product operation is performed with the upsampling kernel of the channel to finally obtain the upsampled feature map.

[0057] In the embodiment of the present invention, the feature fusion network is improved, cross-level feature fusion is introduced to increase the feature information propagation path, and the upsampling operator CARAFE is combined to enhance content perception, and shallow information is fully utilized for feature fusion, thereby effectively avoiding feature loss.

[0058] Based on any of the above embodiments, the loss function of the target detection model is a focused normalized loss function, which is determined based on a focused mean square error loss function and a distance metric loss function, and the focused mean square error loss function is determined based on a modulation factor and a standard mean square error of each sample, and the distance metric loss function is determined based on the distance between bounding boxes.

[0059] It should be noted that in the original YOLOv5s model, its loss function is the positioning loss function CIOU, where CIOU (Complete Intersection over Union) is a metric used to evaluate the similarity between bounding boxes in target detection. It is an improved version of IOU (Intersection over Union). However, CIOU cannot effectively deal with the problem of category imbalance, it is difficult to focus on samples that are difficult to classify, and it is too sensitive to the geometric overlap between the predicted box and the real box, resulting in affected positioning accuracy. In this regard, the present invention uses a combination of Focal-MSE (i.e., focused mean square error) and NWD metrics as a metric for the allocation of positive and negative samples to obtain an improved positioning loss function to enhance the model's learning ability for difficult samples and reduce sensitivity to geometric overlap, thereby improving detection accuracy.

[0060] Specifically, in order to obtain an improved positioning loss function, it can be implemented in three steps, namely, first calculating the Focal-MSE, then obtaining the NWD metric, and finally constructing the focus normalization loss function. Specifically, first, considering that the number of high-quality anchor boxes in the remote sensing image dataset is small, the number of low-quality anchor boxes is large, and CIOU cannot directly solve the category imbalance problem. Therefore, Focal-MSE is used to solve the category imbalance problem, reduce the loss weight of simple and easy-to-classify samples, and focus on difficult sample learning.

[0061] The execution steps of Focal-MSE include: first, calculating the standard mean square error of each sample; then calculating the modulation factor to adjust the weight of the sample so that the difficult-to-predict samples get more attention; then combining the modulation factor with the mean square error to obtain Focal-MSE, and finally averaging the Focal-MSE of all samples to obtain the final loss value. It should be understood that the modulation factor is a coefficient related to the difficulty of sample prediction, and its value is dynamically adjusted according to the prediction result of the sample. When the prediction result of the sample is close to the true value (that is, the sample is a simple and easy-to-classify sample), the value of the modulation factor will tend to be smaller, thereby reducing the weight of the sample in the total loss; on the contrary, when the prediction result of the sample is greatly different from the true value (that is, the sample is a difficult sample that is difficult to predict), the value of the modulation factor will tend to be larger, thereby increasing the weight of the sample in the total loss. In this way, the Focal-MSE loss function can achieve differentiated attention to samples of different difficulty levels.

[0062] Secondly, the Wasserstein distance is used to calculate the Gaussian distribution distance between two bounding boxes (i.e., the predicted box and the true box) to obtain the distance metric; since the distance metric cannot directly measure the similarity between bounding boxes, the distance metric is normalized in exponential form to obtain a new metric NWD (New Wasserstein Distance). NWD can be used to measure the similarity between the predicted box and the true box, thereby reducing the sensitivity brought by the original CIOU metric.

[0063] Finally, the Focal-MSE is used to adjust the weights of the samples to solve the problem of class imbalance, making the model pay more attention to samples that are difficult to classify; at the same time, the NWD metric reduces the sensitivity of the CIOU metric to the geometric overlap between the predicted box and the true box, thereby providing a more comprehensive evaluation of the distribution differences between the boxes. Therefore, a focused normalized loss function suitable for small target detection in remote sensing images is proposed. :

[0064] in, , is the proportional fusion coefficient, which is most related to the image characteristics of the remote sensing image dataset, the number of small and medium-sized targets, and the complexity of the task. represents the focused mean square error loss function, represents the distance metric loss function.

[0065] Based on any of the above embodiments, Figure 7 : is a flow chart of a remote sensing image small target detection method based on improved YOLOv5s provided by the present invention, such as Figure 7 As shown, the method includes: Step S1, converting the data format of the original remote sensing data set into the YOLO format, and performing corresponding preprocessing on the image data in the remote sensing data set.

[0066] Step S2, divide the preprocessed images into a training set, a validation set, and a test set in a ratio of 6:3:1.

[0067] Step S3, improve the original YOLOv5s backbone network, reconstruct the backbone network C3 structure, extract the detailed features of the feature map while reducing the number of parameters, and obtain an improved backbone network.

[0068] In step S3.1, the stacking fusion strategy is used to replace the ordinary 1×1 convolution in the C3 structure. This module is composed of channel-by-channel convolution and point-by-point convolution stacked together. This stacking fusion strategy reduces the model parameters and computational complexity while maintaining feature extraction capabilities by separating the calculation of spatial dimensions and channel dimensions.

[0069] In step S3.2, after the Bottleneck module of the C3 structure is concatenated, the convolutional attention module CBAM is introduced.

[0070] In step S4, cross-level feature fusion is introduced, and two additional upsampling links are added to the feature fusion layer to fuse features from different layers for a more comprehensive understanding of the content. In addition, in the process of feature upsampling, the upsampling operator CARAFE is used to obtain an improved neck module.

[0071] Step S5, using the combination of Focal-MSE and NWD metrics as a metric for the distribution of positive and negative samples to obtain an improved positioning loss function.

[0072] Step S6, setting model training parameters, using the RSOD data set to iteratively train the model, and obtaining and saving weight data of the remote sensing image target detection model.

[0073] In order to comprehensively evaluate the performance of the improved model, a series of comparative experiments were conducted to compare and analyze the optimized model with the prior object detection model. In the experiment, the main indicator of detection accuracy (mAP) was evaluated, and the results are shown in Table 1.

[0074] Table 1 Experimental results of the improved YOLOv5s model in the RSOD dataset

[0075] It should be noted that the Faster R-CNN in the first column of Table 1 is a traditional two-stage detector, SSD (Single Shot MultiBox Detector), YOLOv4 and YOLOv5s are traditional single-stage detectors, and the improved YOLOv5s is an improved remote sensing image target detection model provided by an embodiment of the present invention. The airplane, oil drum, overpass, and playground in Table 1 are different detection targets, and the corresponding values ​​in each column represent the detection accuracy of each detector for the corresponding detection target. It can be seen from Table 1 that the improved YOLOv5s model is compared with the existing typical model on the RSOD dataset. Compared with the baseline YOLOv5s model, the mAP is improved by 2.46%. Compared with the traditional single-stage and two-stage detectors, the average precision of the improved algorithm has been significantly improved.

[0076] Step S7: After the training is completed, the model training weights with the smallest loss function of the validation set are selected and loaded into the network, and the network is tested with the test set to obtain the detection performance of the network.

[0077] The method provided by the embodiment of the present invention can improve the detection accuracy of small targets in remote sensing images. By improving the backbone network, lightweight convolutional attention CBAM and stacking fusion strategy are introduced to reconstruct the backbone C3 module to extract more and more detailed feature information. The network neck is improved by introducing cross-level feature fusion to increase the feature information propagation path, and combining the upsampling operator CARAFE to enhance content perception, and make full use of shallow information for feature fusion, thereby effectively avoiding feature loss. By improving the positioning loss function and introducing the focus normalization loss function, the category imbalance problem can be solved by adjusting the sample weights, the model's learning ability for difficult samples is enhanced, and the sensitivity to geometric overlap is reduced, thereby improving detection accuracy.

[0078] Based on any of the above embodiments, Figure 8 is a schematic diagram of the structure of the image small target detection device provided by the present invention, such as Figure 8 As shown, the device comprises: An acquisition unit 810 is used to acquire an image to be detected; A detection unit 820 is used to input the image to be detected into a target detection model to obtain a target detection result output by the target detection model; Among them, the target detection model includes a feature extraction network, which includes a stacked fusion convolution block and a convolution attention module. The stacked fusion convolution block is used to perform channel-by-channel and point-by-point convolution on the image to be detected to generate a first feature map. The convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map. The second feature map is obtained based on the first feature map.

[0079] The device provided by the embodiment of the present invention can more effectively extract features from the image to be detected by introducing stacked fusion convolution blocks and convolutional attention modules into the feature extraction network of the target detection model, so that the model can more accurately identify the target in the image. The stacked fusion convolution blocks can reduce the model parameters and the amount of calculation while maintaining the feature extraction capability, and capture feature information of different scales and types in the image, through the combination of channel-by-channel convolution and point-by-point convolution. The convolutional attention module further integrates the attention weights in the channel dimension and the spatial dimension, so that the model can focus on more critical and useful features in the image, thereby generating enhanced features and improving the accuracy and efficiency of feature extraction. In addition, the design of the stacked fusion convolution blocks and the convolutional attention module enables the model to retain more detail information when extracting features, thereby more effectively capturing the features of small targets in the image and improving detection accuracy.

[0080] Based on any of the above embodiments, the feature extraction network includes a plurality of feature extraction modules connected in series, and accordingly, the detection unit 820 includes: A feature extraction unit, used to extract and enhance features of the image to be detected based on each feature extraction module in the feature extraction network to obtain an enhanced feature map; A feature fusion unit, configured to perform multi-scale feature fusion on the enhanced feature map based on a feature fusion network of the target detection model to obtain a fused feature map; The target detection unit is used to perform target detection based on the detection module of the target detection model by applying the fused feature map to obtain the target detection result.

[0081] Based on any of the above embodiments, any feature extraction module includes a stacked fusion convolution block, a bottleneck module, a splicing module and a convolution attention module, and accordingly, the feature extraction unit includes: A stacking fusion subunit, configured to perform channel-by-channel convolution and point-by-point convolution on an input feature map based on a stacking fusion convolution block of any of the feature extraction modules to obtain a first feature map, wherein the input feature map is determined based on the image to be detected or an enhanced feature map output by a previous feature extraction module; A processing subunit, configured to sequentially perform compression, feature extraction and expansion processing on the first feature map based on a bottleneck module of any feature extraction module to obtain a processed feature map; a splicing subunit, configured to splice the input feature map and the processed feature map based on a splicing module of any feature extraction module to obtain a second feature map; An enhancement subunit is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension based on the convolutional attention module of any of the feature extraction modules to obtain the enhanced feature map.

[0082] Based on any of the above embodiments, the stacking fusion subunit is specifically used for: Based on the channel-by-channel convolution unit of the stacked fusion convolution block, a convolution operation is performed on each channel of the input feature map to obtain a feature map to be integrated; Based on the point-by-point convolution unit of the stacked fusion convolution block, information between channels of the feature map to be integrated is integrated to obtain the first feature map.

[0083] Based on any of the above embodiments, the enhancer unit is specifically used for: Based on the channel attention unit of the convolutional attention module, applying the second feature map to generate each channel weight, and multiplying the each channel weight with the second feature map to obtain a channel weighted feature map; Based on the spatial attention unit of the convolutional attention module, the channel-weighted feature map is applied to generate each spatial position weight, and each spatial position weight is multiplied by the channel-weighted feature map to obtain the enhanced feature map.

[0084] Based on any of the above embodiments, the feature fusion network includes at least one upsampling link, the upsampling link includes an upsampling prediction module and a feature recombination module, and the step of performing feature upsampling in the upsampling link includes: Based on the upsampling prediction module, the enhanced feature map is compressed in number of channels to obtain a compressed feature map, the compressed feature map is encoded, and the encoded feature map is upsampled and predicted and feature expanded to obtain an extended feature map, and the extended feature map is normalized to obtain a channel upsampling kernel; Based on the feature recombination module, a regional feature map of a preset size is extracted from the enhanced feature map, and a dot product operation is performed between the regional feature map and the channel upsampling kernel to obtain an upsampling feature map.

[0085] Based on any of the above embodiments, the loss function of the target detection model is a focused normalized loss function, which is determined based on a focused mean square error loss function and a distance metric loss function, and the focused mean square error loss function is determined based on a modulation factor and a standard mean square error of each sample, and the distance metric loss function is determined based on the distance between bounding boxes.

[0086] Fig. 9 An example of a physical structure diagram of an electronic device is shown in FIG. Fig. 9 As shown, the electronic device may include: a processor 910, a communication interface 920, a memory 930 and a communication bus 940, wherein the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logic instructions in the memory 930 to execute the image small target detection method, the method comprising: obtaining an image to be detected; inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model; wherein the target detection model comprises a feature extraction network, the feature extraction network comprises a stacked fusion convolution block and a convolution attention module, the stacked fusion convolution block is used to perform channel-by-channel convolution and point-by-point convolution on the image to be detected to generate a first feature map, the convolution attention module is used to perform attention weight integration on the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map, and the second feature map is obtained based on the first feature map.

[0087] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the relevant technology or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0088] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image small target detection method provided by the above methods, and the method includes: obtaining an image to be detected; inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model; wherein the target detection model includes a feature extraction network, and the feature extraction network includes a stacked fusion convolution block and a convolution attention module, the stacked fusion convolution block is used to perform channel-by-channel convolution and point-by-point convolution on the image to be detected to generate a first feature map, and the convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map, and the second feature map is obtained based on the first feature map.

[0089] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the image small target detection method provided by the above-mentioned methods, the method comprising: obtaining an image to be detected; inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model; wherein the target detection model comprises a feature extraction network, the feature extraction network comprises a stacked fusion convolution block and a convolution attention module, the stacked fusion convolution block is used to perform channel-by-channel convolution and point-by-point convolution on the image to be detected to generate a first feature map, the convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map, and the second feature map is obtained based on the first feature map.

[0090] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0091] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small objects in an image, characterized in that: include: Acquire the image to be detected; Inputting the image to be detected into a target detection model to obtain a target detection result output by the target detection model; Among them, the target detection model includes a feature extraction network, which includes a stacked fusion convolution block and a convolution attention module. The stacked fusion convolution block is used to perform channel-by-channel and point-by-point convolution on the image to be detected to generate a first feature map. The convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map. The second feature map is obtained based on the first feature map.

2. The image small target detection method according to claim 1, characterized in that: The feature extraction network includes a plurality of feature extraction modules connected in series, and the image to be detected is input into the target detection model to obtain the target detection result output by the target detection model, including: Based on each feature extraction module in the feature extraction network, feature extraction and enhancement are performed on the image to be detected to obtain an enhanced feature map; Based on the feature fusion network of the target detection model, multi-scale feature fusion is performed on the enhanced feature map to obtain a fused feature map; Based on the detection module of the target detection model, the fused feature map is applied to perform target detection to obtain the target detection result.

3. The image small target detection method according to claim 2, characterized in that: Any feature extraction module includes a stacked fusion convolution block, a bottleneck module, a splicing module and a convolution attention module. The feature extraction modules in the feature extraction network extract and enhance the image to be detected to obtain an enhanced feature map, including: Based on the stacked fusion convolution block of any one of the feature extraction modules, perform channel-by-channel convolution and point-by-point convolution on the input feature map to obtain a first feature map, where the input feature map is determined based on the image to be detected or an enhanced feature map output by a previous feature extraction module; Based on the bottleneck module of any one of the feature extraction modules, sequentially compress, extract features and expand the first feature map to obtain a processed feature map; Based on the splicing module of any one of the feature extraction modules, the input feature map and the processed feature map are spliced ​​to obtain a second feature map; Based on the convolutional attention module of any of the feature extraction modules, the attention weights of the second feature map are integrated in the channel dimension and the spatial dimension to obtain the enhanced feature map.

4. The image small target detection method according to claim 3, characterized in that: The stacked fusion convolution block based on any one of the feature extraction modules performs channel-by-channel convolution and point-by-point convolution on the input feature map to obtain a first feature map, including: Based on the channel-by-channel convolution unit of the stacked fusion convolution block, a convolution operation is performed on each channel of the input feature map to obtain a feature map to be integrated; Based on the point-by-point convolution unit of the stacked fusion convolution block, information between channels of the feature map to be integrated is integrated to obtain the first feature map.

5. The image small target detection method according to claim 3, characterized in that: The convolutional attention module based on any one of the feature extraction modules performs attention weight integration on the second feature map in the channel dimension and the spatial dimension to obtain the enhanced feature map, including: Based on the channel attention unit of the convolutional attention module, applying the second feature map to generate each channel weight, and multiplying the each channel weight with the second feature map to obtain a channel weighted feature map; Based on the spatial attention unit of the convolutional attention module, the channel-weighted feature map is applied to generate each spatial position weight, and each spatial position weight is multiplied by the channel-weighted feature map to obtain the enhanced feature map.

6. The image small target detection method according to claim 2, characterized in that: The feature fusion network includes at least one upsampling link, the upsampling link includes an upsampling prediction module and a feature reorganization module, and the step of performing feature upsampling in the upsampling link includes: Based on the upsampling prediction module, the enhanced feature map is compressed in number of channels to obtain a compressed feature map, the compressed feature map is encoded, and the encoded feature map is upsampled and predicted and feature expanded to obtain an extended feature map, and the extended feature map is normalized to obtain a channel upsampling kernel; Based on the feature recombination module, a regional feature map of a preset size is extracted from the enhanced feature map, and a dot product operation is performed between the regional feature map and the channel upsampling kernel to obtain an upsampling feature map.

7. The image small target detection method according to any one of claims 1 to 6, characterized in that: The loss function of the target detection model is a focused normalized loss function, which is determined based on a focused mean square error loss function and a distance metric loss function. The focused mean square error loss function is determined based on a modulation factor and a standard mean square error of each sample, and the distance metric loss function is determined based on the distance between bounding boxes.

8. An image small target detection device, characterized in that: An acquisition unit, used for acquiring an image to be detected; A detection unit, used to input the image to be detected into a target detection model to obtain a target detection result output by the target detection model; Among them, the target detection model includes a feature extraction network, which includes a stacked fusion convolution block and a convolution attention module. The stacked fusion convolution block is used to perform channel-by-channel and point-by-point convolution on the image to be detected to generate a first feature map. The convolution attention module is used to integrate the attention weights of the second feature map in the channel dimension and the spatial dimension to obtain an enhanced feature map. The second feature map is obtained based on the first feature map.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for detecting small objects in an image as described in any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting small objects in an image as claimed in any one of claims 1 to 7 is implemented.

Images