Small target detection method and device, electronic equipment, storage medium and program product

Abstract

The application provides a small target detection method and device, electronic equipment, storage medium and program product, and relates to the technical field of image processing. The method comprises the following steps: performing feature extraction on a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map; processing the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map; inputting the shallow enhanced feature map into a rear-end layer of the backbone network, continuing to forward propagate, and obtaining rear-end layer feature maps output by multiple network stages; performing weighted fusion on the shallow enhanced feature map and the rear-end layer feature maps through a multi-scale feature fusion network to obtain a multi-scale fusion feature map; and detecting the multi-scale fusion feature map through a detection head to obtain a small target detection result. The application can improve the accuracy of the small target detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a small target detection method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] Target detection is a core task in the field of computer vision, which aims to identify specific class object instances in images or videos and determine their positions and sizes. In recent years, with the development of deep learning technology, target detection algorithms based on convolutional neural networks (CNN) have become the mainstream solution.

[0003] In existing technical solutions, target detection usually includes two main stages: feature extraction and target positioning / classification, and the performance of the feature extraction stage has a decisive influence on the accuracy of the entire detection system. Current mainstream structural frameworks, such as the ResNet (Residual Network) series, ShuffleNet (Shuffle Network) series, etc., perform excellently in processing medium or large scale targets, but their extraction ability significantly decreases when facing small target detection, often failing to capture effective representation information, resulting in poor accuracy of small target detection results.

[0004] Therefore, how to effectively improve the accuracy of small target detection is a problem that needs to be solved at present. SUMMARY

[0005] The present application provides a small target detection method, device, electronic equipment, storage medium and program product to improve the accuracy of small target detection results.

[0006] The present application provides a small target detection method, comprising:

[0007] extracting features of a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map;

[0008] processing the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map;

[0009] inputting the shallow enhanced feature map into a back-end layer of the backbone network for further propagation to obtain back-end layer feature maps output by multiple network stages;

[0010] performing weighted fusion on the shallow enhanced feature map and the back-end layer feature maps through a multi-scale feature fusion network to obtain a multi-scale fusion feature map;

[0011] detecting the multi-scale fusion feature map through a detection head to obtain a small target detection result.

[0012] The small target detection method provided by the application comprises the following steps:

[0013] The initial feature map is obtained by performing feature extraction on the to-be-detected image through an initial convolutional layer of the backbone network.

[0014] The initial shallow layer feature map is obtained by performing feature extraction on the initial feature map through a first network stage of the backbone network; the first network stage is the first network stage after the first spatial down-sampling is completed in the backbone network.

[0015] The small target detection method provided by the application comprises the following steps:

[0016] The back-end layer feature map is up-sampled to have the same spatial size as the shallow layer enhanced feature map.

[0017] The shallow layer enhanced feature map and the back-end layer feature map after up-sampling are respectively adjusted in channels through a 1*1 convolutional layer to obtain a to-be-fused shallow layer enhanced feature map and a to-be-fused back-end layer feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map have the same number of channels.

[0018] An adaptive weight is generated by an attention module according to the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map.

[0019] The to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map are weighted and summed according to the adaptive weight to obtain the multi-scale fusion feature map.

[0020] The small target detection method provided by the application comprises the following steps:

[0021] The feature maps output by the last two network stages in the back-end layer are selected from the back-end layer feature map as target scale feature maps.

[0022] The shallow layer enhanced feature map and the target scale feature map are weighted and fused based on an attention mechanism through a multi-scale feature fusion network to obtain a multi-scale fusion feature map.

[0023] According to the small target detection method provided by the application, the initial shallow feature map is processed through an asymmetric convolution module to obtain a shallow enhanced feature map, which comprises:

[0024] A horizontal convolution kernel is applied to the initial shallow feature map to perform convolution operation to generate a horizontal direction feature map;

[0025] A vertical convolution kernel is applied to the initial shallow feature map to perform convolution operation to generate a vertical direction feature map;

[0026] The horizontal direction feature map and the vertical direction feature map are added and fused to obtain the shallow enhanced feature map.

[0027] According to the small target detection method provided by the application, the horizontal convolution kernel is a 1x m convolution kernel, and the vertical convolution kernel is a m x 1 convolution kernel, wherein m is an integer greater than 1.

[0028] The application further provides a small target detection device, comprising:

[0029] A feature extraction module is configured to extract features of a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map;

[0030] A first processing module is configured to process the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map;

[0031] A second processing module is configured to input the shallow enhanced feature map into a back-end layer of the backbone network and continue to propagate forward to obtain back-end layer feature maps output by multiple network stages;

[0032] A weighted fusion module is configured to perform weighted fusion on the shallow enhanced feature map and the back-end layer feature maps through a multi-scale feature fusion network to obtain a multi-scale fusion feature map;

[0033] A small target detection module is configured to detect the multi-scale fusion feature map through a detection head to obtain a small target detection result.

[0034] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the small target detection method according to any of the above when executing the computer program.

[0035] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the small target detection method according to any of the above.

[0036] The application further provides a computer program product comprising a computer program which, when executed by a processor, implements the small target detection method according to any one of the above.

[0037] The small target detection method, device, electronic equipment, storage medium and program product provided by the application first perform feature extraction on a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map. The initial shallow feature map contains rich detailed information while retaining a high spatial resolution. Then, the initial shallow feature map is processed through an asymmetric convolution module to enhance the directional details therein, and a shallow enhanced feature map is obtained, thereby significantly strengthening the detailed features that are crucial to small target detection. Next, the shallow enhanced feature map is input into a back-end layer of the backbone network for continued propagation to obtain back-end layer feature maps output by multiple network stages. The shallow enhanced feature map and the back-end layer feature maps are weighted and fused through a multi-scale feature fusion network to obtain a multi-scale fused feature map. The fusion process effectively retains key semantics and detailed information of different scales, so that the final feature map has both high-resolution details and high-level semantics. Finally, the multi-scale fused feature map is detected through a detection head to obtain a small target detection result. Through the cooperative mechanism of directional enhancement and multi-scale fusion, the asymmetric convolution and multi-layer feature fusion technologies are organically combined to improve the expression ability and discriminativeness of small target features, thereby significantly improving the accuracy of the small target detection result. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0039] Figure 1 is one of the flowcharts of the small target detection method provided by the application;

[0040] Figure 2 is another flowchart of the small target detection method provided by the application;

[0041] Figure 3 is a third flowchart of the small target detection method provided by the application;

[0042] Figure 4 is a fourth flowchart of the small target detection method provided by the application;

[0043] Figure 5 is a fifth flowchart of the small target detection method provided by the application;

[0044] Figure 6 is a structural schematic diagram of a small target detection device provided by the present application;

[0045] Figure 7 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0047] Target detection is a core task in the field of computer vision, which aims to identify specific class of object instances in images or videos and determine their location and size. In recent years, with the development of deep learning technology, target detection algorithms based on convolutional neural networks have become the mainstream solution.

[0048] In existing technical solutions, target detection usually includes two main stages: feature extraction and target positioning / classification, and the performance of the feature extraction stage has a decisive influence on the accuracy of the entire detection system. Current mainstream structural frameworks, such as ResNet series and ShuffleNet series, perform excellently in processing medium-scale or large-scale targets, but their extraction ability significantly decreases when facing small target detection, often failing to capture effective representation information, resulting in poor accuracy of small target detection results.

[0049] Therefore, how to effectively improve the accuracy of small target detection is a problem that needs to be solved at present.

[0050] Through analysis, these mainstream structural frameworks generally adopt a single and symmetric deep network structure, which has two key limitations:

[0051] First, the directional feature extraction ability is insufficient. Traditional convolution operations use square convolution kernels (such as 3x3 or 5x5), which use symmetric weight distribution in horizontal and vertical directions. This uniform attention to global features makes it difficult for the network to extract features with directional characteristics, such as horizontally oriented road markings or vertically distributed building edges. For small targets with limited pixel quantity, this symmetric design weakens their key discriminative features.

[0052] Second, the limitation of single-scale feature extraction. The existing single network architecture adopts a serial feature extraction mode. As the network layer deepens, the receptive field gradually expands. This design causes the effective features of small targets to be diluted in the deep network. For example, a 10x10 pixel target only occupies a very small proportion of the receptive field in the deep network.

[0053] Based on the above analysis, the present application provides a small target detection method, device, electronic equipment, storage medium and program product, which will be described below in combination with Figures 1-7 .

[0054] Figure 1 is one of the flowcharts of the small target detection method provided by the present application, as shown in Figure 1 , the small target detection method comprises steps S110, S120, S130, S140 and S150.

[0055] Step S110, feature extraction is performed on the to-be-detected image through the front-end layer of the backbone network to obtain an initial shallow feature map.

[0056] The backbone network can adopt a classic convolutional neural network structure, for example, ResNet series (such as ResNet-18, ResNet-50, ResNet-101), ShuffleNet series (such as ShuffleNet V1, ShuffleNet V2), GhostNet series, etc.

[0057] The front-end layer refers to the network layer close to the input image and located in the front part of the backbone network, which is the first layer level passed by the to-be-detected image after entering the network. Specifically, the front-end layer can include an initial convolutional layer and a first network stage, or an initial convolutional layer, a first network stage and a second network stage. Among them, the first network stage is the first network stage after completing the first spatial down-sampling in the backbone network, and the second network stage is the next network stage of the first network stage.

[0058] Taking ResNet-18 as the backbone network as an example, the ResNet-18 architecture includes a stem stage (initial convolutional layer), stage1 (stage 1, also known as the first residual stage), stage2, stage3 and stage4. Among them, the initial convolutional layer corresponds to the stem stage, the first spatial down-sampling is completed by the stem stage, the first network stage corresponds to stage1, and the second network stage corresponds to stage2.

[0059] Taking ShuffleNet V2 as an example, the ShuffleNet V2 architecture includes a Head (also referred to as an initial convolutional layer), a stage2, a stage3, and a stage4. The initial convolutional layer corresponds to the Head, the first spatial down-sampling is completed by the Head, the first network stage corresponds to the stage2, and the second network stage corresponds to the stage3.

[0060] The initial shallow feature map has a significantly reduced size compared to the original image to be detected, but still retains rich spatial position information, edges, textures, and other low-level but crucial details for small target positioning.

[0061] In an embodiment, the initial convolutional layer of the backbone network is used to extract features of the image to be detected to obtain an initial feature map; and the first network stage of the backbone network is used to extract features of the initial feature map to obtain an initial shallow feature map; wherein the first network stage is a network stage that completes the first spatial down-sampling in the backbone network. The specific execution process can refer to the following embodiments, which will not be repeated here.

[0062] In another embodiment, the initial convolutional layer of the backbone network is used to extract features of the image to be detected to obtain an initial feature map; and the first network stage and the second network stage of the backbone network are used to extract features of the initial feature map in sequence to obtain an initial shallow feature map; wherein the first network stage is a network stage that completes the first spatial down-sampling in the backbone network, and the second network stage is a network stage next to the first network stage.

[0063] It should be noted that the initial shallow feature map extracted by the first embodiment has a higher spatial resolution and richer low-level detail information than the initial shallow feature map extracted by the second embodiment, which can ensure that more rich detail sources can be obtained during subsequent direction enhancement and feature fusion, and is more helpful to improve the accuracy of small target detection results.

[0064] In step S120, the initial shallow feature map is processed by an asymmetric convolution module to obtain a shallow enhanced feature map.

[0065] In order to enhance the expression ability of the shallow feature without significantly increasing the calculation cost, an asymmetric convolution module is introduced in the embodiments of the present application. The initial shallow feature map is processed by the asymmetric convolution module to obtain an enhanced shallow feature map, which is denoted as a shallow enhanced feature map.

[0066] The asymmetric convolution module includes two parallel branches: a horizontal convolution branch and a vertical convolution branch. In the horizontal convolution branch, a horizontal convolution kernel is applied to the shallow feature map for convolution operation to generate a horizontal direction feature map; in the vertical convolution branch, a vertical convolution kernel is applied to the shallow feature map for convolution operation to generate a vertical direction feature map; the horizontal direction feature map and the vertical direction feature map are added and fused to obtain a shallow enhanced feature map. The specific execution process can refer to the following embodiments, which will not be described here.

[0067] Further, the asymmetric convolution module adopts a double-channel output structure, the front half channel of the output shallow enhanced feature map is dedicated to storing horizontal direction features, and the rear half channel is dedicated to storing vertical direction features, thereby maintaining the independence of the direction features and providing a structured input basis for subsequent fusion.

[0068] In step S130, the shallow enhanced feature map is input to the rear end layer of the backbone network, and continues to propagate forward to obtain a plurality of network stage output rear end layer feature maps.

[0069] The rear end layer refers to the network layer in the backbone network that is far away from the input image and is in the rear part of the backbone network, specifically, it refers to other network layers in the backbone network except the front end layer.

[0070] The shallow enhanced feature map is input to the rear end layer of the backbone network, and continues to propagate forward to obtain a plurality of network stage output rear end layer feature maps.

[0071] Taking ResNet-18 as the backbone network as an example, the ResNet-18 architecture includes a stem stage, a stage1, a stage2, a stage3 and a stage4. When the front end layer includes the stem stage and the stage1, the rear end layer includes the stage2, the stage3 and the stage4. Correspondingly, the rear end layer feature map includes a feature map processed by the stage2, a feature map processed by the stage3 and a feature map processed by the stage4.

[0072] Taking ShuffleNet V2 as the backbone network as an example, the ShuffleNet V2 architecture includes a Head, a stage2, a stage3 and a stage4. When the front end layer includes the Head and the stage2, the rear end layer includes the stage3 and the stage4. Correspondingly, the rear end layer feature map includes a feature map processed by the stage3 and a feature map processed by the stage4.

[0073] In step S140, the shallow enhanced feature map and the rear end layer feature map are weighted and fused by a multi-scale feature fusion network to obtain a multi-scale fusion feature map.

[0074] The multi-scale feature fusion network comprises a plurality of branches, each branch can independently process features on a feature map of a specific scale, in this way, the feature maps of multiple scales can be input in parallel, the protection of features of each scale, especially shallow layer features, is realized, thereby effectively preventing small target information from being diluted.

[0075] In an embodiment, the back-end layer feature map is up-sampled to have the same spatial size as the shallow layer enhanced feature map; a 1x1 convolution layer is used to adjust the channels of the shallow layer enhanced feature map and the up-sampled back-end layer feature map respectively, to obtain a to-be-fused shallow layer enhanced feature map and a to-be-fused back-end layer feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map have the same number of channels; the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map are weighted and summed according to a preset weight, to obtain a multi-scale fusion feature map.

[0076] In another embodiment, the feature maps output by the last two network stages in the back-end layer are selected from the back-end layer feature map as target scale feature maps; the target scale feature maps are up-sampled to have the same spatial size as the shallow layer enhanced feature map; a 1x1 convolution layer is used to adjust the channels of the shallow layer enhanced feature map and the up-sampled target scale feature map respectively, to obtain a to-be-fused shallow layer enhanced feature map and a to-be-fused target scale feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused target scale feature map have the same number of channels; the to-be-fused shallow layer enhanced feature map and the to-be-fused target scale feature map are weighted and summed according to a preset weight, to obtain a multi-scale fusion feature map.

[0077] Further, in the process of weighted fusion, the weighted fusion is performed based on an attention mechanism, that is, the shallow layer enhanced feature map and the back-end layer feature map are weighted and fused based on the attention mechanism by the multi-scale feature fusion network, to obtain a multi-scale fusion feature map.

[0078] Specifically, in an embodiment, the back-end layer feature map is up-sampled to have the same spatial size as the shallow layer enhanced feature map; a 1x1 convolution layer is used to adjust the channels of the shallow layer enhanced feature map and the up-sampled back-end layer feature map respectively, to obtain a to-be-fused shallow layer enhanced feature map and a to-be-fused back-end layer feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map have the same number of channels; an adaptive weight is generated by an attention module according to the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map are weighted and summed according to the adaptive weight, to obtain a multi-scale fusion feature map. The specific execution process can refer to the following embodiments, which will not be described here.

[0079] In another implementation, from the back-end layer feature maps, the feature maps output by the last two network stages in the back-end layer are screened as target scale feature maps; the target scale feature maps are up-sampled to have the same spatial size as the shallow layer enhanced feature maps; the shallow layer enhanced feature maps and the up-sampled target scale feature maps are respectively subjected to channel adjustment through a 1x1 convolutional layer to obtain to-be-fused shallow layer enhanced feature maps and to-be-fused target scale feature maps; the to-be-fused shallow layer enhanced feature maps and the to-be-fused target scale feature maps have the same number of channels; adaptive weights are generated from the to-be-fused shallow layer enhanced feature maps and the to-be-fused target scale feature maps through an attention module; and the to-be-fused shallow layer enhanced feature maps and the to-be-fused target scale feature maps are subjected to weighted summation according to the adaptive weights to obtain multi-scale fusion feature maps. The specific execution process can refer to the following embodiments, which will not be repeated here.

[0080] In step S150, the multi-scale fusion feature maps are detected by the detection head to obtain small target detection results.

[0081] The multi-scale fusion feature maps obtained above are input into the detection head to be detected by the detection head to obtain small target detection results.

[0082] The small target detection method provided by the embodiments of the present application first extracts features of a to-be-detected image through the front-end layer of the backbone network to obtain an initial shallow layer feature map, which contains rich detail information while retaining a high spatial resolution. Then, the initial shallow layer feature map is processed by the asymmetric convolution module to enhance the directional details therein, thereby obtaining a shallow layer enhanced feature map, which significantly strengthens the detail features that are crucial to small target detection. Next, the shallow layer enhanced feature map is input into the back-end layer of the backbone network for further propagation to obtain back-end layer feature maps output by multiple network stages, and the shallow layer enhanced feature map and the back-end layer feature maps are subjected to weighted fusion by the multi-scale feature fusion network to obtain multi-scale fusion feature maps. The fusion process effectively retains key semantics and detail information of different scales, so that the final feature map has both high-resolution details and high-level semantics. Finally, the multi-scale fusion feature maps are detected by the detection head to obtain small target detection results. Through the synergistic mechanism of directional enhancement and multi-scale fusion, the embodiments of the present application organically combine the asymmetric convolution and multi-layer feature fusion technology to improve the expression ability and discriminativeness of small target features, thereby significantly improving the accuracy of small target detection results.

[0083] Based on any of the above embodiments, Figure 2 is a flowchart of the small target detection method provided by the present application, as shown in Figure 2 Step S110 includes step S111 and step S112.

[0084] Step S111, performing feature extraction on the to-be-detected image through an initial convolutional layer of the backbone network to obtain an initial feature map.

[0085] The initial convolutional layer is used to perform preliminary feature extraction and substantial spatial down-sampling on the to-be-detected image, so as to reduce subsequent calculation amount and expand the receptive field.

[0086] Taking ResNet-18 as the backbone network as an example, in the ResNet-18 architecture, the initial convolutional layer is composed of a 7x7 convolutional layer (with a stride of 2) and a 3x3 max-pooling layer (with a stride of 2) in series. After processing an input to-be-detected image with a size of 224x224 through the initial convolutional layer, the spatial size is down-sampled to 56x56, and an initial feature map with a channel number of 64 is output.

[0087] Taking ShuffleNet V2 as the backbone network as an example, in the ShuffleNet V2 architecture, the initial convolutional layer is composed of a 3x3 convolutional layer (with a stride of 2) and a 3x3 max-pooling layer (with a stride of 2) in series. After processing an input to-be-detected image with a size of 224x224 through the initial convolutional layer, the spatial size is down-sampled to 56x56, and an initial feature map with a channel number of 24 is output.

[0088] Step S112, performing feature extraction on the initial feature map through a first network stage of the backbone network to obtain the initial shallow feature map; wherein the first network stage is the first network stage after the first spatial down-sampling in the backbone network.

[0089] The initial feature map is input into the first network stage of the backbone network, and the initial feature map is subjected to feature extraction through the first network stage to obtain the initial shallow feature map.

[0090] The first network stage is the first network stage after the first spatial down-sampling in the backbone network. For example, in the ResNet-18 architecture, the first network stage corresponds to the first group of residual modules (stage1), which specifically includes two basic residual blocks, each of which has two layers of 3x3 convolution, and the output channel number is 64. For another example, in the ShuffleNet V2 architecture, the first network stage corresponds to stage2, and specifically, the stage2 includes four sequentially connected ShuffleNet basic blocks (Block). The first basic block uses a depth convolution with a stride of 2 to realize spatial down-sampling, and the subsequent three basic blocks have a stride of 1 to keep the feature map size unchanged. Each basic block performs channel partitioning, grouped convolution, channel shuffling, and the like, and finally outputs a feature map with a channel number of 244.

[0091] Taking ResNet-18 as the backbone network as an example, after obtaining the initial feature map of 56x56, the initial feature map is input into the stage1 of ResNet-18, the spatial size of the input and output is kept unchanged (still 56x56), feature extraction is performed, and finally a feature map with a channel number of 64 is output, which is the initial shallow feature map.

[0092] Taking ShuffleNet V2 as the backbone network as an example, after obtaining the initial feature map of 56x56, the initial feature map is first input into the first basic block (stride=2) in the stage2 of ShuffleNet V2, and the spatial size is down-sampled to 28x28. Then, the subsequent three basic blocks are processed, and the size of 28x28 is kept unchanged, and finally a feature map with a channel number of 244 is output, which is the initial shallow feature map.

[0093] It should be noted that the output feature map of the first network stage is selected as the initial shallow feature map because it retains a very high spatial resolution while having undergone preliminary semantic information extraction, and contains rich low-level visual information such as edges, textures, and corner points. As the source of subsequent direction enhancement and feature fusion, it can ensure that these important information for small target detection can be effectively protected and utilized, thereby significantly overcoming the problem of detail loss caused by deepening of the network, and improving the recall rate and positioning accuracy of small targets.

[0094] The small target detection method provided by the embodiment of the application takes the feature map obtained through the initial convolution layer and the first network stage of the backbone network as the initial shallow feature map, which retains the maximum spatial resolution and the most original detail information, and takes it as the source of subsequent direction enhancement and feature fusion, which can fundamentally solve the problem of dilution or loss of small target information caused by deepening of the network.

[0095] Based on any of the above embodiments, Figure 3 is a flowchart of the third of the small target detection methods provided by the present application, as shown in Figure 3 Step S140 includes step S141, step S142, step S143 and step S144.

[0096] Step S141, up-sampling the back-end layer feature map to the same spatial size as the shallow layer enhanced feature map.

[0097] Step S142, adjusting the channels of the shallow layer enhanced feature map and the up-sampled back-end layer feature map respectively through a 1x1 convolution layer to obtain a to-be-fused shallow layer enhanced feature map and a to-be-fused back-end layer feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature map have the same number of channels.

[0098] Considering that although the existing standard feature pyramid attempts to fuse multi-scale features, the one-way fusion method still causes deep semantic information to be submerged in fine spatial details of the shallow layer. Therefore, in this embodiment, a progressive feature fusion architecture is provided, multiple branches are set, each branch can independently process features on a specific scale feature map, and the weights of each branch are automatically learned based on an attention mechanism to realize dynamic weighted fusion, so as to protect small target information from being lost while realizing multi-scale feature complementary enhancement, thereby further improving the accuracy of small target detection results.

[0099] First, a common target size and channel number are selected to process the feature maps of each branch. Specifically, the size of the highest resolution shallow layer enhanced feature map can be selected as the reference to maximize the preservation of detail information. The channel number can be selected as 256, which can balance the performance and expression ability and the computational feasibility.

[0100] Since the spatial size of the back-end layer feature map is small, the back-end layer feature map is up-sampled to the same spatial size as the shallow layer enhanced feature map. Then, the channels of the shallow layer enhanced feature map and the up-sampled back-end layer feature map are adjusted through a 1x1 convolution layer, so that the channel numbers of the feature maps are the same to facilitate subsequent fusion. The feature dimension is adjusted through a 1x1 convolution to avoid distortion in cross-layer fusion.

[0101] Taking ResNet-18 as the backbone network as an example, the spatial size of a shallow layer enhanced feature map (denoted as C1) is 56x56, and the number of channels is 64. After processing by the back-end layers stage2, stage3 and stage4, a plurality of back-end layer feature maps are obtained, which are denoted as C2, C3 and C4 respectively, wherein the spatial size of C2 is 28x28, and the number of channels is 128; the spatial size of C3 is 14x14, and the number of channels is 256; the spatial size of C4 is 7x7, and the number of channels is 512.

[0102] Correspondingly, at this time, the multi-scale feature fusion network includes four branches, denoted as a first branch, a second branch, a third branch and a fourth branch.

[0103] The C1 is adjusted in the number of channels through the first branch to obtain a to-be-fused shallow layer enhanced feature map (denoted as F1). When adjusting the number of channels, the number of channels of F1 is adjusted to 256 using a 1x1 convolution, so that the spatial size of F1 is 56x56, and the number of channels is 256.

[0104] The C2 is up-sampled and adjusted in the number of channels through the second branch to obtain a to-be-fused feature map (denoted as F2). When up-sampling, 2 times up-sampling is used, and then the number of channels of F2 is adjusted to 256 using a 1x1 convolution, so that the spatial size of F2 is 56x56, and the number of channels is 256.

[0105] The C3 is up-sampled through the third branch to obtain a to-be-fused feature map (denoted as F3). When up-sampling, 4 times up-sampling is used, so that the spatial size of F3 is 56x56, and the number of channels is 256.

[0106] The C4 is up-sampled and adjusted in the number of channels through the fourth branch to obtain a to-be-fused feature map (denoted as F4). When up-sampling, 8 times up-sampling is used, and then the number of channels of F4 is adjusted to 256 using a 1x1 convolution, so that the spatial size of F4 is 56x56, and the number of channels is 256.

[0107] At this time, the to-be-fused shallow layer enhanced feature map is F1, and the to-be-fused back-end layer feature maps include the above F2, F3 and F4.

[0108] In step S143, an attention module is used to generate adaptive weights according to the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature maps.

[0109] The to-be-fused shallow layer enhanced feature map and the plurality of to-be-fused back-end layer feature maps are input into the attention module to generate adaptive weights corresponding to each feature map.

[0110] Specifically, the element addition is first performed on the to-be-fused shallow layer enhanced feature map and the plurality of to-be-fused back-end layer feature maps to obtain an aggregated feature map, denoted as F', in the above example, F' = F1+F2+F3+F4, the spatial size of F' is 56x56, and the channel number is 256.

[0111] Global average pooling is applied to the aggregated feature map to compress the global information, which is then input into a fully connected neural network and normalized by a Softmax function to obtain adaptive weights corresponding to each feature map, denoted as α1, α2, α3 and α4, respectively.

[0112] In step S144, the adaptive weights are used to perform weighted summation on the to-be-fused shallow layer enhanced feature map and the to-be-fused back-end layer feature maps to obtain the multi-scale fusion feature map.

[0113] The adaptive weights corresponding to each feature map are used to perform weighted summation on each feature map to obtain the multi-scale fusion feature map.

[0114] In the above example, the multi-scale fusion feature map F = (α1xF1) + (α2xF2) + (α3xF3) + (α4xF4).

[0115] The small target detection method provided by the embodiment of the present application performs spatial size and channel number alignment processing on the shallow layer enhanced feature map and the plurality of back-end layer feature maps through each branch of the multi-scale feature fusion network. In the processing process, the spatial size of the shallow layer enhanced feature map with the highest resolution is selected as the reference, and the spatial scale and the channel are matched through upsampling and 1x1 convolution to maximize the retention of the detailed information which is crucial for small target detection. Then, the weights of each branch are automatically learned based on the attention mechanism to realize dynamic weighted fusion, which can automatically increase the weight of the feature map with strong discrimination for small targets. Based on the asymmetric convolution processing of the shallow layer feature to realize small target information enhancement extraction, the above multi-scale alignment and attention weighted fusion mechanism are combined to form a closed-loop optimization system of "extraction-protection-enhancement": the direction-sensitive feature extraction is realized through asymmetric convolution, the detailed information is protected without loss through scale alignment and feature fusion, and finally the adaptive enhancement in the fusion process is realized through the attention mechanism. While effectively maintaining the integrity of the small target structure, the system fully integrates the deep semantic and shallow details, significantly improves the discrimination ability of multi-scale feature expression, and further improves the accuracy of the small target detection result.

[0116] Based on any of the above embodiments, Figure 4 is a fourth flowchart of the small target detection method provided by the present application, as shown in Figure 4 step S140 further includes steps S145 and S146.

[0117] Step S145, filtering the feature maps output by the last two network stages in the backend layer from the backend layer feature map as the target scale feature map.

[0118] Considering different network architectures, the backend layer includes different numbers of network stages, and considering the accuracy and efficiency, in the embodiment of the application, the feature maps output by the last two network stages in the backend layer are selected as the target scale feature map. The two target scale feature maps have large receptive fields and can represent global and high-level information of the target, and then dynamically weighted fusion is performed with the shallow layer enhanced feature map, which can provide a feature representation with high resolution details and high-level semantics for small target detection, effectively solve the core problem of small target missing detection and inaccurate positioning, and improve the accuracy of small target detection results.

[0119] Taking the ResNet-18 architecture as an example, the last two network stages of the backend layer are Stage3 and Stage4, and the feature maps of these two network stages are selected as the target scale feature map, that is, C3 and C4.

[0120] Step S146, performing attention mechanism-based weighted fusion on the shallow layer enhanced feature map and the target scale feature map through a multi-scale feature fusion network to obtain a multi-scale fusion feature map.

[0121] Then, the shallow layer enhanced feature map and the target scale feature map are weighted fused based on the attention mechanism through the multi-scale feature fusion network to obtain a multi-scale fusion feature map.

[0122] Specifically, the above step S146 includes steps S1461, S1462, S1463 and S1464.

[0123] Step S1461, upsampling the target scale feature map to have the same spatial size as the shallow layer enhanced feature map.

[0124] Step S1462, adjusting the channels of the shallow layer enhanced feature map and the target scale feature map after upsampling through a 1x1 convolution layer to obtain a to-be-fused shallow layer enhanced feature map and a to-be-fused target scale feature map; the to-be-fused shallow layer enhanced feature map and the to-be-fused target scale feature map have the same number of channels.

[0125] Step S1463, generating adaptive weights according to the to-be-fused shallow layer enhanced feature map and the to-be-fused target scale feature map through an attention module.

[0126] Step S1464, according to the adaptive weight, weighted sum of the to-be-fused shallow layer enhanced feature map and the to-be-fused target scale feature map is performed to obtain the multi-scale fusion feature map.

[0127] At this time, the target scale feature map includes 2, taking ResNet-18 as the backbone network as an example, the target scale feature map includes the feature map C3 output by stage3 and the C4 output by stage4. Correspondingly, the multi-scale feature fusion network includes 3 branches, denoted as the first branch, the second branch and the third branch.

[0128] The C1 is adjusted in channel number through the first branch to obtain the to-be-fused shallow layer enhanced feature map (denoted as F1). When adjusting the channel number, the channel number is adjusted to 256 using 1x1 convolution, and the spatial size of F1 is 56x56 and the channel number is 256.

[0129] The C3 is up-sampled through the second branch to obtain the to-be-fused feature map (denoted as F3). When up-sampling, 4 times up-sampling is used, and the spatial size of F3 is 56x56 and the channel number is 256.

[0130] The C4 is up-sampled and adjusted in channel number through the third branch to obtain the to-be-fused feature map (denoted as F4). When up-sampling, 8 times up-sampling is used, and then the channel number is adjusted to 256 using 1x1 convolution, and the spatial size of F4 is 56x56 and the channel number is 256.

[0131] At this time, the to-be-fused shallow layer enhanced feature map is F1, and the to-be-fused back-end layer feature map includes the above F3 and F4.

[0132] Then, the adaptive weights corresponding to each to-be-fused feature map are generated through the attention module, and are denoted as β1, β2 and β3 respectively. At this time, the multi-scale fusion feature map F is: F=(β1xF1)+(β2xF3)+(β3xF4).

[0133] The small target detection method provided in the embodiments of the present application filters the feature maps output by the last two network stages in the backend layer as target scale feature maps, and performs weighted fusion based on the attention mechanism with the shallow layer enhanced feature maps. Compared with selecting all the backend layer feature maps and the shallow layer enhanced feature maps for fusion, the fusion object in the embodiments of the present application is more targeted, can focus on the most representative semantic level (i.e., the target scale feature map) and the best quality detail source (i.e., the shallow layer enhanced feature map), avoids the interference of redundant and noisy features, enables the attention mechanism to more effectively perform fusion, thereby improving the key indicators (especially the small target detection accuracy), and at the same time, can significantly reduce the calculation amount and memory occupation in the multi-scale fusion process, improve the detection efficiency, and make it more suitable for real-time applications and resource-limited platforms. In addition, compared with selecting only the feature map output by the last network stage and the shallow layer enhanced feature map for fusion, the embodiments of the present application select the feature maps of the last two layers, which can ensure the diversity and hierarchy of high-level semantic information, and further improve the accuracy of the small target detection result.

[0134] Based on any of the above embodiments, Figure 5 is a fifth flowchart of the small target detection method provided by the present application, as shown in Figure 5 Step S120 includes step S121, step S122 and step S123.

[0135] It should be noted that step S121 and step S122 are executed in parallel.

[0136] Step S121 applies a horizontal convolution kernel to the initial shallow layer feature map to perform convolution operation and generate a horizontal direction feature map.

[0137] Step S122 applies a vertical convolution kernel to the initial shallow layer feature map to perform convolution operation and generate a vertical direction feature map.

[0138] The traditional convolution operation adopts a square convolution kernel (such as 3x3 or 5x5), which adopts a symmetric weight distribution in the horizontal and vertical directions. This uniform attention to global features makes it difficult for the network to extract features with directional characteristics, such as horizontally oriented road markings or vertically distributed building edges. And for small targets with limited pixel amount, this symmetric design weakens their key discriminative features. Therefore, in the present embodiment, an asymmetric convolution kernel (including a horizontal convolution kernel and a vertical convolution kernel) is designed in the asymmetric convolution module, which can break through the symmetry limitation of the traditional convolution and enhance the discriminative features such as the edge direction of the small target.

[0139] The asymmetric convolution module includes two parallel convolution branches: a horizontal convolution branch and a vertical convolution branch.

[0140] wherein, the horizontal convolution branch, applying a horizontal convolution kernel (denoted as k1) to the input initial shallow feature map (denoted as input) to perform two-dimensional convolution operation to generate the horizontal direction feature map (k1 input), the convolution operation slides in the horizontal direction of the feature map, and is specifically used to capture and enhance the horizontal structural features. The vertical convolution branch, applying a vertical convolution kernel (denoted as k2) to the same input initial shallow feature map to perform two-dimensional convolution operation to obtain the vertical direction feature map (k2 input), the convolution operation slides in the vertical direction of the feature map, and is specifically used to capture and enhance the vertical structural features.

[0141] Further, the horizontal convolution kernel is a 1xm convolution kernel, and the vertical convolution kernel is a mx1 convolution kernel, where m is an integer greater than 1.

[0142] Correspondingly, the horizontal direction feature map and the vertical direction feature map are calculated as follows:

[0143] ;

[0144] ;

[0145] wherein, a represents the input initial shallow feature map; i and j represent the position coordinates of the output feature map, i is the row index, and j is the column index; v represents the offset of the convolution kernel in the horizontal direction, and the value range is -m / 2~m / 2; u represents the offset of the convolution kernel in the vertical direction, and the value range is -m / 2~m / 2; represents the horizontal convolution kernel weight parameter; represents the vertical convolution kernel weight parameter.

[0146] It should be noted that by setting the convolution kernel of the same length in the horizontal direction and the vertical direction, relatively balanced performance can be obtained on small targets of various shapes, avoiding missing detection due to favoring features in a specific direction, thereby further improving the overall accuracy of the small target detection result.

[0147] In step S123, the horizontal direction feature map and the vertical direction feature map are added and fused to obtain the shallow enhanced feature map.

[0148] The two feature maps output by the horizontal convolution branch and the vertical convolution branch are added element by element, and the added result is added to a learnable bias term to obtain the shallow enhanced feature map. Specifically as follows:

[0149] output= + +b;

[0150] wherein, output is a shallow layer enhanced feature map, b is a bias quantity, b is a scalar (a single numerical value) which is added to + The bias quantity b is a standard component in a neural network and is used to increase the expression capacity of the model.

[0151] By setting a 1xm convolution kernel and an mx1 convolution kernel, a horizontal direction feature map and a vertical direction feature map are respectively generated, the dimensions of the horizontal direction feature map and the vertical direction feature map are the same, and then the shallow layer enhanced feature map can be directly added and fused, and through this fusion mode, the enhanced feature information in the horizontal and vertical directions can be superimposed without increasing the size and channel number of the feature map, forming a comprehensive feature representation which is more sensitive to linear and structural features. In addition, compared with using a traditional symmetric convolution kernel, the parameter quantity and the calculation complexity can be greatly reduced.

[0152] The small target detection method provided by the embodiment of the present application uses an asymmetric convolution module, replaces the traditional symmetric convolution kernel with a horizontal direction convolution kernel and a vertical direction convolution kernel, greatly reduces the parameter quantity and the calculation complexity, and at the same time, through the special extraction and addition and fusion of horizontal and vertical features, a stronger representation capability for small target structures can be obtained, thereby improving the accuracy of the small target detection result.

[0153] Figure 6 is a structural schematic diagram of a small target detection device provided by the present application, as shown in Figure 6 The device comprises a feature extraction module 610, a first processing module 620, a second processing module 630, a weighted fusion module 640 and a small target detection module 650; wherein:

[0154] The feature extraction module 610 is used for performing feature extraction on a to-be-detected image through a front end layer of a backbone network to obtain an initial shallow layer feature map.

[0155] The first processing module 620 is used for processing the initial shallow layer feature map through an asymmetric convolution module to obtain a shallow layer enhanced feature map.

[0156] The second processing module 630 is used for inputting the shallow layer enhanced feature map to a back end layer of the backbone network, continuing to propagate forward, and obtaining a back end layer feature map output by a plurality of network stages.

[0157] The weighted fusion module 640 is used for performing weighted fusion on the shallow layer enhanced feature map and the back end layer feature map through a multi-scale feature fusion network to obtain a multi-scale fusion feature map.

[0158] The small target detection module 650 is configured to detect the multi-scale fusion feature map by using a detection head to obtain a small target detection result.

[0159] The small target detection device provided by the embodiment of the present application first extracts features of a to-be-detected image by using a front-end layer of a backbone network to obtain an initial shallow layer feature map. The initial shallow layer feature map contains rich detail information while retaining a high spatial resolution. Then, the initial shallow layer feature map is processed by using an asymmetric convolution module to enhance directional details in the initial shallow layer feature map, thereby obtaining a shallow layer enhanced feature map, and significantly strengthening the detail features that are crucial for small target detection. Next, the shallow layer enhanced feature map is input into a back-end layer of the backbone network for further propagation to obtain back-end layer feature maps output by multiple network stages. The shallow layer enhanced feature map and the back-end layer feature maps are weighted and fused by using a multi-scale feature fusion network to obtain a multi-scale fusion feature map. The fusion process effectively retains key semantics and detail information of different scales, so that the final feature map has both high-resolution details and high-level semantics. Finally, the multi-scale fusion feature map is detected by using a detection head to obtain a small target detection result. The embodiment of the present application uses a synergistic mechanism of directional enhancement and multi-scale fusion to organically combine the asymmetric convolution and multi-layer feature fusion technology, thereby improving the expression ability and discriminativeness of small target features, and significantly improving the accuracy of the small target detection result.

[0160] It should be noted that the small target detection device provided by the embodiment of the present application can implement all the method steps of the small target detection method embodiment and achieve the same technical effects. Therefore, the same parts and beneficial effects of the method embodiment will not be described in detail.

[0161] Figure 7 An example of a schematic diagram of a physical structure of an electronic device is shown in FIG. 1. Figure 7As shown, the electronic device can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740, wherein the processor 710, the communications interface 720, and the memory 730 complete mutual communication through the communications bus 740. The processor 710 can invoke a logic instruction in the memory 730 to execute a small target detection method, which includes: performing feature extraction on a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map; processing the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map; inputting the shallow enhanced feature map into a back-end layer of the backbone network, continuing to forward propagate, and obtaining a back-end layer feature map output by a plurality of network stages; performing weighted fusion on the shallow enhanced feature map and the back-end layer feature map through a multi-scale feature fusion network to obtain a multi-scale fusion feature map; and performing detection on the multi-scale fusion feature map through a detection head to obtain a small target detection result.

[0162] In addition, the logic instruction in the memory 730 described above can be implemented in the form of a software function unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0163] In another aspect, the present application also provides a computer program product comprising a computer program, which can be stored on a non-transitory computer readable storage medium, and the computer program is executable by a processor to enable a computer to perform the small target detection method provided by the above-mentioned methods, which comprises: performing feature extraction on a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map; processing the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map; inputting the shallow enhanced feature map into a back-end layer of the backbone network and continuing to forward propagate to obtain back-end layer feature maps output by multiple network stages; performing weighted fusion on the shallow enhanced feature map and the back-end layer feature maps through a multi-scale feature fusion network to obtain a multi-scale fusion feature map; and performing detection on the multi-scale fusion feature map through a detection head to obtain a small target detection result.

[0164] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the small target detection method provided by the above-mentioned methods, which comprises: performing feature extraction on a to-be-detected image through a front-end layer of a backbone network to obtain an initial shallow feature map; processing the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map; inputting the shallow enhanced feature map into a back-end layer of the backbone network and continuing to forward propagate to obtain back-end layer feature maps output by multiple network stages; performing weighted fusion on the shallow enhanced feature map and the back-end layer feature maps through a multi-scale feature fusion network to obtain a multi-scale fusion feature map; and performing detection on the multi-scale fusion feature map through a detection head to obtain a small target detection result.

[0165] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the present embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0166] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and necessary high-resource hardware platforms, and of course, the various embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that contributes to the present application can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0167] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A small object detection method characterized by, The method comprises the following steps: feature extraction is performed on the to-be-detected image by a front-end layer of a backbone network to obtain an initial shallow feature map; processing is performed on the initial shallow feature map by an asymmetric convolution module to obtain a shallow enhanced feature map; wherein the asymmetric convolution module comprises parallel horizontal convolution kernels and vertical convolution kernels; the shallow enhanced feature map is input into a back-end layer of the backbone network for further propagation to obtain back-end layer feature maps output by multiple network stages; weighting fusion is performed on the shallow enhanced feature map and the back-end layer feature maps by a multi-scale feature fusion network to obtain a multi-scale fusion feature map; detection is performed on the multi-scale fusion feature map by a detection head to obtain a small target detection result; the weighting fusion performed on the shallow enhanced feature map and the back-end layer feature maps by the multi-scale feature fusion network to obtain the multi-scale fusion feature map comprises the following steps: upsampling is performed on the back-end layer feature maps to have the same spatial size as the shallow enhanced feature map; channel adjustment is performed on the shallow enhanced feature map and the back-end layer feature maps after upsampling respectively by a 1x1 convolution layer to obtain a to-be-fused shallow enhanced feature map and a to-be-fused back-end layer feature map; the to-be-fused shallow enhanced feature map and the to-be-fused back-end layer feature map have the same number of channels; an adaptive weight is generated by an attention module according to the to-be-fused shallow enhanced feature map and the to-be-fused back-end layer feature map; the to-be-fused shallow enhanced feature map and the to-be-fused back-end layer feature map are weighted and summed according to the adaptive weight to obtain the multi-scale fusion feature map.

2. The small object detection method of claim 1, wherein the feature extraction performed on the to-be-detected image by the front-end layer of the backbone network to obtain the initial shallow feature map comprises the following steps: feature extraction is performed on the to-be-detected image by an initial convolution layer of the backbone network to obtain an initial feature map; feature extraction is performed on the initial feature map by a first network stage of the backbone network to obtain the initial shallow feature map; wherein the first network stage is the first network stage after the first spatial down-sampling in the backbone network.

3. The small object detection method according to any one of claims 1 to 2, characterized in that, the weighting fusion performed on the shallow enhanced feature map and the back-end layer feature maps by the multi-scale feature fusion network to obtain the multi-scale fusion feature map further comprises the following steps: feature maps output by the last two network stages in the back-end layer are selected from the back-end layer feature maps as target scale feature maps; weighting fusion is performed on the shallow enhanced feature map and the target scale feature maps based on an attention mechanism by the multi-scale feature fusion network to obtain the multi-scale fusion feature map.

4. The small object detection method according to any one of claims 1 to 2, characterized in that, the processing performed on the initial shallow feature map by the asymmetric convolution module to obtain the shallow enhanced feature map comprises the following steps: a horizontal convolution kernel is applied to the initial shallow feature map to perform convolution operation and generate a horizontal direction feature map; a vertical convolution kernel is applied to the initial shallow feature map to perform convolution operation and generate a vertical direction feature map; the horizontal direction feature map and the vertical direction feature map are added and fused to obtain the shallow enhanced feature map.

5. The small object detection method of claim 4, wherein The horizontal convolution kernel is a 1*m convolution kernel, and the vertical convolution kernel is an m*1 convolution kernel, where m is an integer greater than 1.

6. A small object detection apparatus, characterized by comprising: Comprise: The feature extraction module is configured to extract features of the to-be-detected image through a front-end layer of the backbone network to obtain an initial shallow feature map; The first processing module is configured to process the initial shallow feature map through an asymmetric convolution module to obtain a shallow enhanced feature map; wherein the asymmetric convolution module comprises a horizontal convolution kernel and a vertical convolution kernel in parallel; The second processing module is configured to input the shallow enhanced feature map into a back-end layer of the backbone network and continue to propagate forward to obtain back-end layer feature maps output by multiple network stages; The weighted fusion module is configured to perform weighted fusion on the shallow enhanced feature map and the back-end layer feature maps through a multi-scale feature fusion network to obtain a multi-scale fusion feature map; The small target detection module is configured to detect the multi-scale fusion feature map through a detection head to obtain a small target detection result. The weighted fusion module is specifically configured to: Up-sample the back-end layer feature maps to have the same spatial size as the shallow enhanced feature map; Adjust channels of the shallow enhanced feature map and the back-end layer feature map after up-sampling through a 1*1 convolution layer to obtain a to-be-fused shallow enhanced feature map and a to-be-fused back-end layer feature map; the to-be-fused shallow enhanced feature map and the to-be-fused back-end layer feature map have the same number of channels; Generate adaptive weights according to the to-be-fused shallow enhanced feature map and the to-be-fused back-end layer feature map through an attention module; Perform weighted summation on the to-be-fused shallow enhanced feature map and the to-be-fused back-end layer feature map according to the adaptive weights to obtain the multi-scale fusion feature map.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the computer program to implement the small target detection method in any one of claims 1-5.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the small target detection method in any one of claims 1-5.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the small target detection method in any one of claims 1-5.

Images