A lightweight small target feature enhancement method and device based on background contrast

Abstract

The application provides a lightweight small target feature enhancement method and device based on background contrast, which enhances the shallow feature map through a feature enhancement module, expands the receptive field and captures more context information while maintaining a low parameter amount, and uses the context information to enhance the contrast between the small target object and the background to enhance the expression ability of the feature, which helps to achieve better small target detection performance. Through the ingenious design of channel dilation convolution and channel attention mechanism, the complexity of the network is reduced, the network is lightweight, and is suitable for mobile devices and resource-limited scenes. Since the application uses channel dilation convolution for dilation, compared with ordinary dilation convolution or depth separable dilation convolution, the parameter amount can be greatly reduced, thereby improving the calculation efficiency of the model, thereby improving the effect of the enhanced feature map, and further improving the accuracy of detecting small targets.

Description

Technical Field

[0001] This invention belongs to the field of small target detection technology, specifically relating to a lightweight small target feature enhancement method and device based on background contrast. Background Technology

[0002] The purpose of object detection is to accurately pinpoint the location of specific objects in images or videos using automatic recognition techniques. Traditional object detection methods face challenges in practical applications, including high computational complexity, large model size, and slow processing speed. Their performance and efficiency are limited, especially in mobile devices, embedded systems, or resource-constrained scenarios, where they face numerous challenges, particularly in small object detection tasks.

[0003] Because small targets typically occupy only a few pixels in an image, they have low spatial resolution, resulting in limited detail information and potentially preventing existing detection methods from accurately detecting or classifying them. The low contrast between small targets and their surrounding background also makes them difficult to distinguish in an image, hindering the correct identification of targets by existing methods. Small targets may be obscured by other objects, occlusions, or noise, further complicating detection. Incomplete information about occluded parts can lead to erroneous detection results from existing methods. Summary of the Invention

[0004] To address the aforementioned problems in the prior art, this invention provides a lightweight small target feature enhancement method and apparatus based on background contrast. The technical problem to be solved by this invention is achieved through the following technical solution:

[0005] In a first aspect, the present invention provides a lightweight small target feature enhancement method based on background contrast, comprising:

[0006] S100: Obtain the image to be enhanced containing small targets, and perform feature extraction on the image to be enhanced to obtain a shallow feature map;

[0007] S200, the shallow feature map is input into the constructed feature enhancement module, and the operation of extracting the channel weight parameters of the shallow feature map and performing multiple dilation convolution operations on the shallow feature map are performed to obtain the background feature map; the background feature map is weighted using the channel weight parameters, and the enhanced feature map is obtained by combining the shallow feature map.

[0008] Secondly, the present invention provides a lightweight small target feature enhancement device based on background contrast, comprising:

[0009] The acquisition module is configured to acquire an image to be enhanced containing small targets, and to perform feature extraction on the image to be enhanced to obtain a shallow feature map;

[0010] The feature enhancement module is configured to input the shallow feature map into the constructed feature enhancement module, perform operations to extract the channel weight parameters of the shallow feature map and perform multiple dilation convolution operations on the shallow feature map to obtain a background feature map; use the channel weight parameters to assign weights to the background feature map, and combine the shallow feature map to obtain the enhanced feature map.

[0011] Thirdly, the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0012] Memory, used to store computer programs;

[0013] When a processor executes a program stored in memory, it implements the steps of the lightweight small target feature enhancement method based on background contrast as described in the first aspect.

[0014] Beneficial effects:

[0015] This invention provides a lightweight small target feature enhancement method and apparatus based on background contrast. The feature enhancement module enhances shallow feature maps, expanding the receptive field and capturing more contextual information while maintaining a low parameter count. This contextual information is used to strengthen the contrast between the small target and the background, enhancing the expressive power of the features and contributing to better small target detection performance. Through the ingenious design of channel dilated convolution and channel attention mechanisms, the complexity of the network is reduced, making the network lightweight and suitable for mobile devices and resource-constrained scenarios. Because this invention uses channel dilated convolution for dilation, compared to ordinary dilated convolution and depthwise separable dilated convolution, the number of parameters is significantly reduced, thereby improving the computational efficiency of the model and enhancing the feature map enhancement effect, further improving the accuracy of small target detection.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a lightweight small target feature enhancement method based on background contrast provided by the present invention.

[0018] Figure 2 This is a structural diagram of the lightweight small target feature enhancement method based on background contrast provided by the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0020] Combination Figure 1 and Figure 2 This invention provides a lightweight small target feature enhancement method based on background contrast, comprising:

[0021] S100: Obtain the image to be enhanced containing small targets, and extract features from the image to be enhanced to obtain a shallow feature map;

[0022] This step involves extracting shallow feature maps of the image to be enhanced from the backbone network, which are then input into the feature enhancement module.

[0023] S200, the shallow feature map is input into the constructed feature enhancement module, and the operation of extracting the channel weight parameters of the shallow feature map and performing multiple dilation convolution operations on the shallow feature map are performed to obtain the background feature map; the background feature map is weighted using the channel weight parameters, and the enhanced feature map is obtained by combining the shallow feature map.

[0024] For details, please refer to Figure 2 The feature enhancement module of the present invention includes a channel attention mechanism module, three channels dilated convolutional layers, a summation layer, a weight allocation layer, a concatenation layer and an output convolutional layer;

[0025] The process involves the following steps: the image to be enhanced is input to a first dilated convolutional layer, the stitching layer, and the channel attention mechanism module; the channel weight parameters output by the channel attention mechanism module are input to the weight allocation layer; the first dilated feature map output by the first channel dilated convolutional layer is input to a second channel dilated convolutional layer and the summing layer; the second dilated feature map output by the second channel dilated convolutional layer is input to a third channel dilated convolutional layer and the summing layer; the third dilated feature map output by the third channel dilated convolutional layer is input to the summing layer; the feature map output by the summing layer is input to the weight allocation layer; the weight allocation layer outputs the enhanced feature map to the stitching layer; the stitching layer outputs the stitched image to the output convolutional layer; and the output convolutional layer outputs the enhanced image of the image to be enhanced.

[0026] This invention provides a lightweight small target feature enhancement method based on background contrast. This method employs global average pooling and one-dimensional convolution to obtain the weights for each channel, thereby guiding the model to focus more on feature channels containing rich information and complex backgrounds. Next, the shallow feature map is processed sequentially through three layers of channel-wise dilated convolutions. To increase the receptive field and avoid the checkerboard effect, different dilation rates of 1, 2, and 3 are set. This design allows the invention to capture broader contextual information and obtain smoother feature maps with richer background information. After the outputs of the three-layer channel-wise dilated convolutions, the outputs of each layer are summed. These layers have different scales and semantic information; summing them fuses these features at different scales, helping to enhance the expressive power of the features and imbue them with background information from different scenes. Next, the new feature map is multiplied by the channel weights of the original feature map and concatenated with the original feature map along the channel dimension. To enhance the interaction between channels and strengthen the contrast between small targets and background information, this invention uses a single-layer convolution with a kernel size of 1 for channel interaction and reduces the number of channels to the same size as the original feature map.

[0027] This invention can be embedded in any target detection network and improve the network's small target detection performance.

[0028] For example, the present invention uses EtinyNet as the backbone network for feature extraction to obtain shallow feature maps with an eight-fold downsampling.

[0029] Specifically, S200 of the present invention includes:

[0030] S210, the channel weight parameters of each channel are obtained by global average pooling and one-dimensional convolution of the shallow feature map;

[0031] This step processes the input shallow feature map through global average pooling and one-dimensional convolution, and then uses the sigmoid function to calculate the channel weight parameters for each channel. The channel weight parameters are expressed as follows:

[0032] ω=σ(C1D(AG(X)))(1);

[0033] Where σ represents the sigmoid function, C1D represents one-dimensional convolution, AG represents global average pooling, and X represents shallow feature map.

[0034] S220, the first dilated feature map is obtained through the first channel dilated convolutional layer, the second dilated feature map is obtained through the second channel dilated convolutional layer, and the third dilated feature map is obtained through the third channel dilated convolutional layer;

[0035] The dilation rates of the three channel-by-channel dilated convolutional layers are 1, 2, and 3, respectively.

[0036] S230, the first dilated feature map, the second dilated feature map, and the third dilated feature map are added together by the summation layer to obtain the background feature map;

[0037] This invention takes the original shallow feature map and inputs it into dilated convolutions with dilation rates of 1, 2, and 3, respectively. The outputs of each convolution are summed to obtain a new feature map.

[0038] Each channel-dilated convolutional layer has a different receptive field for shallow feature maps, and the formula for calculating the receptive field is as follows:

[0039] RF i+1 =RF i +(k-1)×S i (2);

[0040] k`=k+(k-1)×(r-1) (3);

[0041] Among them, RF i+1 Represents the receptive field of the current channel-dilated convolutional layer, RF i The receptive field of the previous channel dilated convolutional layer is represented by k', where k' represents the equivalent kernel size of the dilated convolution, k is the actual kernel size, r is the dilation rate, and S... i S represents the product of the strides of all previous channel dilated convolutional layers. i =ПiStride i Stride i This indicates the stride of the channel dilation convolution layer.

[0042] The receptive field of each dilated convolution layer relative to the original shallow feature map can be calculated using the above formula, as shown in Table 1.

[0043] Table 1. Receptive field parameters for each layer of dilated convolution relative to the original shallow feature map.

[0044] R＝1 R＝2 R＝3 Receptive Field 3 7 13

[0045] By summing the outputs of the three dilated convolution layers, since each convolution layer has a receptive field of 3, 7, and 13 for the original image, and the features extracted by each layer have different scales and semantic information, summing their outputs can fuse these features of different scales, which helps to enhance the expressive power of the features and give them background information for different scenes.

[0046] S240, The background feature map is multiplied by the channel weight parameters through the enhancement layer to obtain a new feature map;

[0047] This invention multiplies the obtained background feature map by the channel weights. Since only channel dilation convolution is used to minimize the number of parameters, without inter-channel interaction, scaling the background feature map using the original feature map channel weights enhances the feature channels with richer information and more complex backgrounds. This helps the network focus more intently on the important features of the target, thereby improving the accuracy of target detection. This operation achieves performance similar to using depthwise separable dilated convolution, but significantly reduces the number of parameters.

[0048] The number of parameters used in the three-channel progressive convolutional layers is:

[0049] Params = C × k × k × 3 +ε (4);

[0050] Where C is the number of channels, k is the kernel size, and ε is the number of parameters in a one-dimensional convolution.

[0051] The number of parameters used in existing techniques for separable dilated convolution is:

[0052] Params = ( C × k × k + C × C ) × 3 (5);

[0053] Where C is the number of channels, k is the kernel size, and ε in equation (1) is the number of one-dimensional convolution parameters, which can be ignored. In this example, when the kernel size is 3 and the number of channels is 128, the number of parameters of the three-layer channel-wise convolution is only 6.5% of that of the depthwise separable convolution, which can greatly reduce the number of parameters and the amount of computation.

[0054] S250, the background feature map and the shallow feature map are spliced ​​together through the splicing layer to obtain a spliced ​​feature map;

[0055] S260, the enhanced feature map is obtained by performing channel interaction on the concatenated feature map through the output convolutional layer.

[0056] This invention concatenates the obtained background feature map with the original feature map in the channel dimension, and then uses a single ordinary convolutional layer with a kernel size of 1 to enhance the contrast between small target objects and background information, and reduces the number of channels to the number of channels in the original shallow feature map.

[0057] Specifically, following S200, the lightweight small target feature enhancement method based on background contrast of the present invention further includes:

[0058] The location of the small target is obtained by detecting the enhanced feature map.

[0059] The final result obtained by this invention, namely the enhanced feature map, can be output to a subsequent network or detection head for processing to obtain the location of small targets.

[0060] This invention provides a lightweight small target feature enhancement device based on background contrast, comprising:

[0061] The acquisition module is configured to acquire an image to be enhanced containing small targets, and to perform feature extraction on the image to be enhanced to obtain a shallow feature map;

[0062] The feature enhancement module is configured to input the shallow feature map into the constructed feature enhancement module, perform operations to extract the channel weight parameters of the shallow feature map and perform multiple dilation convolution operations on the shallow feature map to obtain a background feature map; use the channel weight parameters to assign weights to the background feature map, and combine the shallow feature map to obtain the enhanced feature map.

[0063] This invention provides a lightweight small-object feature enhancement device based on background contrast. The feature enhancement module employs global average pooling and one-dimensional convolution to obtain the weights for each channel, thereby better utilizing the information in the feature map and guiding the model to focus more on feature channels containing rich information and complex backgrounds. Next, the feature map is processed sequentially through three layers of channel-wise dilated convolutions. To increase the receptive field and avoid the checkerboard effect, different dilation rates of 1, 2, and 3 are set. This design allows the invention to capture broader contextual information and obtain smoother feature maps with richer background information. After the outputs of the three-layer channel-wise dilated convolutions, the outputs of each layer are summed. These layers have different scales and semantic information; summing them fuses features of different scales and imbues them with background information from different scenes, enhancing the expressive power of the features. Finally, the invention multiplies the new feature map by the channel weights of the original feature map and concatenates it with the original feature map along the channel dimension. To enhance the interaction between channels and strengthen the contrast between small target objects and background information, this invention uses a single convolutional layer with a kernel size of 1 for channel interaction and reduces the number of channels to the same size as the original feature map.

[0064] This invention provides an electronic device including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0065] Memory, used to store computer programs;

[0066] When a processor executes a program stored in memory, it implements the steps of the present invention's lightweight small target feature enhancement method based on background contrast.

[0067] The effectiveness of this invention can be further illustrated by the following example experimental data.

[0068] This invention validates the performance of EtinyNet and compares it on the visdrone small object dataset. Training was performed using the AdamW optimizer and a cosine annealing learning rate strategy, with an initial learning rate of 0.001, for a total of 300 epochs. Table 1 lists the comparison results, showing the performance of this invention compared to the original EtinyNet.

[0069] Table 2 Comparison of the effects of the present invention and the original EtinyNet

[0070] Parameters mAP EtinyNet 594k 20.32％ EtinyNet+ This invention 630k 22.94％

[0071] As shown in Table 2, by improving EtinyNet using the method of this invention and training it on the Visdrone dataset, the number of parameters increased by only 36k, but the detection accuracy improved by 2.62%. This indicates that the present invention achieves a significant improvement in small object detection and has advantages over the original EtinyNet. Such experimental data further demonstrates the effectiveness and potential of the present invention in object detection tasks.

[0072] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0073] Although this application has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings, the disclosure, and the appended claims in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality.

[0074] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A lightweight small target feature enhancement method based on background contrast, characterized in that, include: S100: Obtain the image to be enhanced containing small targets, and perform feature extraction on the image to be enhanced to obtain a shallow feature map; S200, the shallow feature map is input into the constructed feature enhancement module, and the operation of extracting the channel weight parameters of the shallow feature map and performing multiple dilation convolution operations on the shallow feature map are performed to obtain the background feature map; the background feature map is weighted using the channel weight parameters, and the shallow feature map is combined to obtain the enhanced feature map; The feature enhancement module includes a channel attention mechanism module, three channels dilated convolutional layers, a summation layer, a weight allocation layer, a concatenation layer, and an output convolutional layer; The shallow feature map is input to the first dilated convolutional layer, the stitching layer, and the channel attention mechanism module; the channel weight parameters output by the channel attention mechanism module are input to the weight allocation layer; the first dilated feature map output by the first channel dilated convolutional layer is input to the second channel dilated convolutional layer and the summing layer; the second dilated feature map output by the second channel dilated convolutional layer is input to the third channel dilated convolutional layer and the summing layer; the third dilated feature map output by the third channel dilated convolutional layer is input to the summing layer; the feature map output by the summing layer is input to the weight allocation layer; the weight allocation layer outputs the enhanced feature map to the stitching layer; the stitching layer outputs the stitched image to the output convolutional layer; and the output convolutional layer outputs the enhanced feature map.

2. The lightweight small target feature enhancement method based on background contrast according to claim 1, characterized in that, S200 includes: S210, the channel weight parameters of each channel are obtained by global average pooling and one-dimensional convolution of the shallow feature map; S220, the first dilated feature map is obtained through the first channel dilated convolutional layer, the second dilated feature map is obtained through the second channel dilated convolutional layer, and the third dilated feature map is obtained through the third channel dilated convolutional layer; S230, the first dilated feature map, the second dilated feature map, and the third dilated feature map are added together by the summation layer to obtain the background feature map; S240, a new feature map is obtained by multiplying the background feature map with the channel weight parameters through the enhancement layer; S250, the background feature map and the shallow feature map are spliced ​​together through the splicing layer to obtain a spliced ​​feature map; S260, the enhanced feature map is obtained by performing channel interaction on the concatenated feature map through the output convolutional layer.

3. The lightweight small target feature enhancement method based on background contrast according to claim 1, characterized in that, The dilation rates of the three-channel dilated convolutional layers are 1, 2, and 3, respectively.

4. The lightweight small target feature enhancement method based on background contrast according to claim 1, characterized in that, The channel weight parameter is expressed as follows: ω=σ(C1D(AG(X))) （1）； in, σ express sigmoid function, C1D Represents one-dimensional convolution. AG This represents global average pooling, and X represents the shallow feature map.

5. The lightweight small target feature enhancement method based on background contrast according to claim 1, characterized in that, Each channel-dilated convolutional layer has a different receptive field for shallow feature maps, and the formula for calculating the receptive field is as follows: RF i+1 =RF i + (k - 1) × S i （2）； k` = k + (k - 1) × (r - 1) （3）； in, RF i+1 This represents the receptive field of the current channel dilated convolutional layer. RF i This represents the receptive field of the channel-dilated convolutional layer above. k` This represents the equivalent kernel size of dilated convolution. k This is the actual kernel size. r For expansion rate, S i This represents the product of the strides of all previous channel-dilated convolutional layers. ，S i =П i Stride i Stride i This indicates the stride of the channel dilation convolution layer.

6. The lightweight small target feature enhancement method based on background contrast according to claim 1, characterized in that, The number of parameters used in the three-channel dilated convolutional layers is: Params = C × k × k × 3 +ε （4）； in, C For the number of channels, k The kernel size is [size]. ε The number of parameters for a one-dimensional convolution.

7. The lightweight small target feature enhancement method based on background contrast according to claim 1, characterized in that, Following S200, the lightweight small target feature enhancement method based on background contrast further includes: The location of the small target is obtained by detecting the enhanced feature map.

8. A lightweight small target feature enhancement device based on background contrast, characterized in that, include: The acquisition module is configured to acquire an image to be enhanced containing small targets, and to perform feature extraction on the image to be enhanced to obtain a shallow feature map; The feature enhancement module is configured to input the shallow feature map into the constructed feature enhancement module, perform operations to extract the channel weight parameters of the shallow feature map and perform multiple dilation convolution operations on the shallow feature map to obtain a background feature map; use the channel weight parameters to assign weights to the background feature map, and combine the shallow feature map to obtain an enhanced feature map; The feature enhancement module includes a channel attention mechanism module, three channels dilated convolutional layers, a summation layer, a weight allocation layer, a concatenation layer, and an output convolutional layer; The shallow feature map is input to the first dilated convolutional layer, the stitching layer, and the channel attention mechanism module; the channel weight parameters output by the channel attention mechanism module are input to the weight allocation layer; the first dilated feature map output by the first channel dilated convolutional layer is input to the second channel dilated convolutional layer and the summing layer; the second dilated feature map output by the second channel dilated convolutional layer is input to the third channel dilated convolutional layer and the summing layer; the third dilated feature map output by the third channel dilated convolutional layer is input to the summing layer; the feature map output by the summing layer is input to the weight allocation layer; the weight allocation layer outputs the enhanced feature map to the stitching layer; the stitching layer outputs the stitched image to the output convolutional layer; and the output convolutional layer outputs the enhanced feature map.

9. An electronic device, characterized in that... It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the lightweight small target feature enhancement method based on background contrast as described in any one of claims 1-7.

Images