Multi-scale receptive field low-illumination target detection method under adaptive enhancement

Abstract

The invention provides a low-illumination target detection method for a multi-scale receptive field under adaptive enhancement, and the method comprises the steps: 1, introducing a Global module, an RCSOBEL module and an LSKADCNN module to optimize a yolov8 target detection algorithm, and constructing a low-illumination target detection model based on image adaptive enhancement and the multi-scale receptive field; 2, acquiring an image data set under a low-illumination condition, and processing the image data set; training the low-illumination target detection model by using the processed data set to obtain a trained low-illumination target detection model; and 3, inputting a to-be-detected low-illumination image into the trained low-illumination target detection model, and outputting a low-illumination target detection result. The method can achieve a good detection effect in a low-illumination image, and is suitable for target detection of images generated under various conditions of insufficient illumination, such as night monitoring and unmanned driving.

Description

Technical Field

[0001] The present invention relates to the technical field of low-illumination image processing, and in particular to a low-illumination target detection method with a multi-scale receptive field under adaptive enhancement. Background Art

[0002] Object detection is a technology that automatically identifies and locates target objects in images or videos through computer algorithms. It is an important research direction in the field of computer vision. At present, object detection technology has been widely used in many fields, including intelligent transportation, security monitoring, medical image analysis, agriculture, etc. With the continuous development of technology, object detection technology will further improve accuracy and efficiency, and bring innovation and progress to more fields. However, lighting conditions are one of the important factors affecting the accuracy of object detection. In practical applications, such as full-time monitoring and unmanned driving, object detection is required under low illumination. In this special environment, the performance of existing deep learning-based object detection algorithms will be greatly reduced. Most of the current object detectors are studied in high-quality images under normal conditions. In real environments, there are often many poor lighting conditions such as night, dark light, and exposure, which reduce image quality and affect the performance of detectors.

[0003] At present, although the target detection technology represented by Yolov8 can achieve good accuracy under normal lighting conditions, it still has defects in low-light conditions. Existing low-light target detection methods often use the method of first using the low-light image enhancement module as a preprocessing module before sending it to the target detector. This method usually ignores the difference between image enhancement and target detection tasks. The enhanced image is more suitable for human eye observation rather than target detection, and does not fully utilize the feature information of the image. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention aims to provide a low-illumination target detection method with multi-scale receptive field under adaptive enhancement, which can solve the existing problems.

[0005] In order to achieve the above object, the technical solution of the present invention is as follows:

[0006] The present invention is realized by the following technical scheme: a low-light target detection method with multi-scale receptive field under adaptive enhancement, comprising the following steps:

[0007] Step 1: By optimizing the Yolov8 target detection algorithm, a low-light target detection model based on image adaptive enhancement and multi-scale receptive field is constructed; including:

[0008] The Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection; the RCSOBEL module is designed in Yolov8 to learn richer feature representations from features and strengthen the extraction of edge information and spatial information; the deformable large kernel separable attention LSKADCN module is designed in Yolov8 to transform the SPPF module in Yolov8 to obtain the LKSPPF module, so that the network focuses on key feature information;

[0009] Step 2: Obtain an image data set under low illumination conditions and process it; use the processed data set to train the low illumination target detection model to obtain a trained low illumination target detection model;

[0010] Step 3: Input the low-light image to be tested into the trained low-light target detection model, and output the low-light target detection result.

[0011] Furthermore, the low-light target detection model includes a backbone network, a neck network and a detection head; wherein

[0012] The backbone network includes a Global module, a convolution module 1, a convolution module 2, a RCSOBEL module 1, a convolution module 3, a C2f module 1, a convolution module 4, a C2f module 2, a RCSOBEL module 2, and a SPPF_LSKADCN module;

[0013] The neck network includes upsampling module 1, fusion module 1, C2f module 1, upsampling module 2, fusion module 2, C2f module 2, convolution module 1, fusion module 3, C2f module 3, convolution module 2, fusion module 4, and C2f module 4;

[0014] The detection head includes three decoupling heads, and the C2f module 2, C2f module 3 and C2f module 4 of the neck network are respectively input into the three decoupling heads.

[0015] Furthermore, the Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection, including:

[0016] After two 3×3 convolution expansion channels, high-dimensional features are encoded at low resolution to capture global information in the image;

[0017] The cross-attention module obtains a 3×3 color matrix and γ value from the high-dimensional features;

[0018] The color matrix and γ are subjected to two 1×1 convolutions respectively to further process and optimize the parameters;

[0019] Perform matrix product of the original image and the color matrix, and perform γ power calculation on the result to obtain the final processed image, which is input into the subsequent network.

[0020] Furthermore, the cross-attention module obtains a 3×3 color matrix and a γ value from the high-dimensional features, including:

[0021] First, the input feature map is convolved with a depth-wise convolution of 3×3 size, and then passed through two fully connected layers to generate the value (V) and key (K) required in the self-attention mechanism. The query (Q) is randomly initialized, and the attention score matrix is ​​calculated using Q and K. The final output result of the Attention layer is calculated using V and the attention score matrix, which can be expressed as follows:

[0022]

[0023] Where Q (Query): query matrix; K (Key): key matrix; V (Value): value matrix; d k : Dimension of the key matrix;

[0024] After passing through a multi-layer perceptron, ten parameters are finally output, representing a 3x3 color matrix and a 1-dimensional gamma (γ) value.

[0025] Furthermore, the matrix product of the original image and the color matrix is ​​performed, and the result is subjected to γ ​​power calculation, including: color correction and gamma correction of the image, which is expressed as follows:

[0026]

[0027] In the formula is a 3x3 color matrix, γ is the power value of gamma correction, and the value of ε is 10 -6 , to prevent the pixel value from being set to 0.

[0028] Furthermore, the RCSOBEL module is designed in Yolov8 to learn richer feature representations from features and enhance the extraction of edge information and spatial information; including:

[0029] The input feature map passes through the RepConv module; passes through the EIESA module twice; the three output channels are spliced ​​together; and passes through a RepConv module to obtain the final feature map.

[0030] Furthermore, the RepConv module is composed of three parallel direct outputs, 3×3 convolutions and 1×1 convolutions during training. Finally, the obtained feature maps are added and passed through the activation function SiLu. During inference, the above module is re-parameterized into a 3×3 convolution plus SiLu activation function, reducing parameters while ensuring accuracy.

[0031] Furthermore, the EIESA module workflow includes:

[0032] The input feature map passes through the convolution layer and the pooling layer, and then the feature map is flattened to (batch_size, feature_dim) size, where batch_size refers to the number of samples in each iteration, and feature_dim is the size obtained after flattening the feature map. Then it passes through the fully connected layer and the softmax normalized weight to obtain four weight values;

[0033] The input feature map is evenly divided into four feature maps in the channel direction, each of which passes through the SobelConv module, the RepConv module, the maximum pooling module and the direct output channel to obtain four feature maps;

[0034] The obtained feature maps are multiplied by the weight values ​​respectively, and after residual connection, they are fused in the channel direction, and finally passed through the channel shuffle module to shuffle the channels to obtain the final feature map.

[0035] Furthermore, the LSKADNC module is composed of LSKA and deformable convolution, and the workflow includes:

[0036] The feature map undergoes a 3×3 deformable group convolution, which allows attention to change adaptively according to data features and focus on key information hidden in the dark. It then passes through two cascaded 1D separable weighted convolution kernels composed of deep hole group convolutions, where the sizes of the convolution kernels are (1, 5) and (5, 1), respectively. In addition, each channel in the feature map is convolved with the corresponding channel in the convolution kernel.

[0037] After 1×1 convolution, cross-channel feature fusion is performed, and finally the attention map is inferred;

[0038] Multiply the output attention map and the input features for adaptive feature refinement;

[0039] The output of LSKADNC is expressed as:

[0040] Attention=Conv 1×1 (DW 5×1 (DW 1×5 (DCN 3×3(F))))

[0041]

[0042] Where F is the input feature map, DCN is the deformable convolution, and DW is the 1D separable weight convolution kernel.

[0043] The present invention also protects a low-illumination target detection device with multi-scale receptive field under adaptive enhancement, comprising a processor and a memory; the memory is used to store programs; the processor executes the program to implement any of the methods described above.

[0044] Compared with the prior art, the beneficial effects of the present invention include:

[0045] The present invention proposes a low-illumination target detection method with multi-scale receptive field under adaptive enhancement, which can achieve good detection effect in low-illumination images. The present invention is suitable for image target detection generated under various insufficient lighting conditions, such as night monitoring and unmanned driving.

[0046] The present invention adds a Globel module for image adaptive enhancement to the backbone network, uses a self-attention mechanism, utilizes image global information to generate parameters for adjusting image information, and performs end-to-end training to make the generated image more conducive to target detection.

[0047] The present invention designs an EIESA module for feature extraction that can learn richer feature representations. The module combines the SobelConv branch for extracting edge information, the repVGG branch for extracting diverse spatial information, and the maximum pooling Maxpool branch for extracting key features to obtain more diverse information, which can effectively utilize the limited information of low-light images and improve detection accuracy.

[0048] This paper designs a variable large-kernel separable attention structure LSKADCN module, which uses LSKA to obtain a richer field of view at a lower cost, has spatial and channel adaptability, effectively solves the problem of incorrect allocation of model attention, and filters noise. At the same time, variable convolution is introduced to obtain more information that is beneficial to the target itself and reduce useless computational consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0050] Figure 1 A schematic diagram of a low-light target detection method with multi-scale receptive field under adaptive enhancement according to the present invention;

[0051] Figure 2 is a structural diagram of a Globle module in an embodiment of the present invention;

[0052] Figure 3 It is a structural diagram of the RCSOBEL module in an embodiment of the present invention;

[0053] Figure 4 It is a structural diagram of the EIESA module in an embodiment of the present invention;

[0054] Figure 5 : is a structural diagram of a RepConv module in an embodiment of the present invention;

[0055] Figure 6 It is a structural diagram of the SPPF_LSKADCN module in an embodiment of the present invention;

[0056] Figure 7 It is a structural diagram of the LSKADCN module in an embodiment of the present invention;

[0057] Figure 8 This is a module structure diagram of a low-light image detection network model in an implementation case of the present invention;

[0058] Fig. 9 It is a rendering of the detection in the implementation case of the present invention. DETAILED DESCRIPTION

[0059] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific implementation modes and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or as a limitation or restriction to the technical solution of the present invention.

[0060] This paper proposes a low-light target detection method with multi-scale receptive field under adaptive enhancement, which is improved on the basis of Yolov8. First, the image adaptive enhancement module is used to adjust the image through the global information of the image to generate an image that is more suitable for target detection, and the feature information of each dimension is more fully extracted and utilized by using multi-scale receptive field and the attention mechanism of large-scale convolution kernel; finally, the proposed method can greatly improve the detection accuracy.

[0061] The low-light target detection method of the present invention with multi-scale receptive field under adaptive enhancement specifically comprises the following steps:

[0062] Step 1: Optimize the Yolov8 target detection algorithm to build a low-light target detection model based on image adaptive enhancement and multi-scale receptive field; specifically include:

[0063] (i) The Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection;

[0064] (ii) Design the RCSOBEL module in Yolov8 to learn richer feature representations from features and enhance the extraction of edge and spatial information;

[0065] (3) Design a deformable large-kernel separable attention LSKADCN module in Yolov8 to transform the SPPF module in Yolov8 to obtain the LKSPPF module, so that the network focuses on key feature information;

[0066] The following is a detailed introduction based on specific implementation cases:

[0067] Among them, (1) the Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection; and the model's ability to detect targets in low light is improved.

[0068] The principle of the Global module is that various poor lighting scenes in natural scenes will affect the obtained images when performing some visual tasks, and low-light images are usually more suitable for human observation rather than target detection. In order to solve this problem, the model of the present invention introduces the Global module to predict the parameters that need to be adjusted for detection based on the global information of the image, so that the image can be adaptively adjusted to overcome the impact of poor lighting and be more suitable for detection.

[0069] The workflow of the Global module is as follows Figure 2 As shown, it includes the following four steps:

[0070] S1. After two 3×3 convolution expansion channels, high-dimensional features are encoded at low resolution to capture global information in the image; specifically: first, the image is downsampled, two 3×3 convolution layers (3×3Conv) are used, the step size is 2, the activation function is the GELU function, and batch normalization is used. The number of channels output by the two convolution layers is 23 and 64 respectively. This step can convert the image into a low resolution and encode high-level features at a low resolution. On the one hand, it can save computing costs and help lightweight design, and on the other hand, it helps to extract global features.

[0071] S2. Obtain a 3×3 color matrix and γ value from the high-dimensional features through the cross-attention module. Specifically, the generated features are passed to the subsequent parameter prediction module. First, the features are subjected to a 3×3 depthwise convolution (3×3WConv). Its main feature is that the convolution operation is applied independently to each input channel, which can significantly reduce the number of parameters and the amount of calculation of the model. Given an input tensor X of shape (B, H, W, C), where B is the size, H is the height, W is the width, and C is the number of channels. For each channel C, the depthwise convolution operation can be expressed as: Y c =X c *K c .Y c is the result of output channel C, X c is the input channel C, K c It is the convolution kernel for channel C. The residual module is subsequently connected to ensure the stability of the model while performing feature extraction;

[0072] The obtained image features are self-attention calculated.

[0073]

[0074] Where Q (Query): query matrix; K (Key): key matrix; V (Value): value matrix; d k : Dimension of the key matrix;

[0075] According to the above formula, the following steps are specifically included:

[0076] 1. The image features are fed into two fully connected layers (FC) and the dimensions are adjusted to obtain keys and values ​​of size 1 × 10. The query is initialized as a 1 × 10 all-zero vector.

[0077] 2. Perform a dot product of the query and key to get the attention score, which is then multiplied by the scaling factor to alleviate numerical instability and improve gradient flow, making the model more robust and efficient during training and inference.

[0078] 3. Perform softmax normalization (LN) on the attention scores to obtain the attention weights. In addition, a dropout operation is applied to the attention weights to improve the generalization ability of the model.

[0079] 4. Apply the attention weight to the value, perform weighted summation, and get the attention output.

[0080] Finally, a linear layer is used to project the obtained attention output to enhance the expressiveness and capture more complex patterns and features. Finally, a dropout operation is performed to improve the robustness of the model and obtain the processed output. The above steps implement the self-attention mechanism, which can capture long-distance dependencies in the input sequence and capture global information. Through multi-head attention, the model can learn multiple different attention distributions at the same time.

[0081] S3. The color matrix and γ are subjected to two 1×1 convolutions respectively, and the parameters are further processed and optimized. Specifically, the obtained output features are passed through a simple multi-layer perceptron, including two 1×1 convolution layers, each of which is followed by an activation function (GELU function) and a Dropout layer. The 1×1 convolution layer can increase the number of channels and the complexity of feature representation while maintaining the same resolution in the spatial dimension. The activation function used is the GELU function, which increases nonlinearity and enhances the expressiveness of the model. Finally, ten parameters are output, representing the 3x3 color matrix and the 1-dimensional gamma (γ) value.

[0082] S4. Perform matrix product of the original image and the obtained color matrix, and then perform γ power calculation on the obtained result. The purpose of the color matrix is ​​to ensure that the color of the object is constant under different light, and at the same time convert the image color into pixel color suitable for detection. The obtained gamma value is used for gamma correction, which is a nonlinear transformation. By adjusting the gamma value, the brightness and contrast of the image can be flexibly controlled to achieve a more ideal effect. When the gamma value is greater than 1, the highlight area (highlight part) of the image will be compressed, so that the details of these areas are reduced, while the low-brightness area (shadow part) will be expanded to increase the visibility of the details. In this case, the image as a whole will appear darker and the shadow part will be emphasized more; when the gamma value is less than 1, the highlight part of the image will be expanded, which means that the highlight area will become brighter and clearer, while the low-brightness area will be compressed to reduce the visibility of the details. This will make the image as a whole appear brighter and enhance the expressiveness of the highlight area.

[0083] Expressed as follows:

[0084] In the formula, I in Represents the brightness value of the input pixel, usually between 0 and 1 (when processing 8-bit images, the range is 0 to 255). out Represents the output pixel brightness value after gamma correction.

[0085] The matrix product of the original image and the color matrix is ​​performed, and the result is subjected to γ ​​power calculation, including: color correction and gamma correction of the image, which is expressed as follows:

[0086]

[0087] In the formula is a 3x3 color matrix, γ is the power value of gamma correction, and the value of ε is 10 -6 , to prevent the pixel value from being 0. Finally, the processed image is obtained.

[0088] Among them, (ii) is to design the RCSOBEL module in the Yolov8 backbone network, which draws on the RCS-OSA module and replaces the original C2f module in the backbone network. This module can improve the efficiency of the network when processing dense connections. By aggregating the diverse features of all multi-scale receptive fields in the last feature map, it can overcome the low efficiency problem of dense connections. The workflow of the RCSOBEL module is as follows Figure 3 As shown. It includes the following three steps:

[0089] S1. Pass the obtained feature map through the RepConv module. The principle of the RepConv module is as follows Figure 5 As shown:

[0090] During the training phase, the model Figure 5 As shown in Figure a, it has a multi-branch topology, including multiple branches, 1×1 convolution layer (1×1Conv), 3×3 convolution layer (3×3Conv) and identity mapping layer. These branches can learn different features. The feature maps obtained by the three branches are fused by adding and passing through the activation function SiLu to form the final output feature map.

[0091] In the inference phase, the model Figure 5 As shown in Figure b, in order to improve computational efficiency and speed, the multi-branch structure is converted to a single-branch structure. The parameters of the multi-branch used in training are fused into a single 3×3 convolution kernel (3×3Conv) plus SiLu activation function, which can significantly improve the computational speed and reduce latency. The number of output channels of the feature map is 1 / 2 of the number of input channels. The idea of ​​reparameterization is to convert the 1×1 convolution in the training phase and the two paths that are not processed into a 3×3 convolution and then fuse them.

[0092] 1×1 to 3×3: Add a circle of 0s outside the 1×1 convolution kernel to form a 3×3 convolution kernel.

[0093] 0×0 is converted to 3×3: A circle of 0 is added outside the weight size of BN to form a 3×3 convolution kernel.

[0094] The fusion process is expressed as:

[0095]

[0096] Where fuse represents the convolution operation after convolution and BN fusion.

[0097] The multi-branch topology can learn rich feature information during training, and the simplified single-branch structure during inference can save memory consumption for inference and achieve fast inference.

[0098] S2: The feature map obtained by RepConv passes through two stacked EIESA modules to ensure feature reuse and strengthen the information flow between different layers through feature cascading.

[0099] The workflow of the EIESA module is as follows Figure 4 As shown, the process is as follows:

[0100] 1. Split the input feature map in the channel direction into four equal parts.

[0101] 2. The segmented feature maps are passed through the SobelConv module, RepConv module, MaxPool module and identity mapping respectively; specifically:

[0102] 2.1. The working principle of SobelConv is as follows: Sobel filter is an edge detection method that can detect sudden changes in intensity in an image, so it can capture the edge features of the image well and obtain important edge information. SobelConv efficiently implements the Sobel operator through 3D group convolution, and the group size is the number of input channels.

[0103] The 3D group convolution uses two 3×3 convolution kernels to calculate the horizontal and vertical gradients, respectively:

[0104] Horizontal gradient kernel Vertical gradient kernel

[0105] For each feature point, applying the above convolution kernel, we can get the horizontal and vertical gradients, which can be expressed as:

[0106]

[0107] Wherein, horizontal gradient: Gx(x,y); I represents the image, Gx represents the horizontal convolution kernel, Gy represents the vertical convolution kernel; vertical gradient: Gy(x,y).

[0108] For ease of calculation, the gradient magnitude uses the approximate gradient magnitude: G≈|G x |+|G y |.

[0109] Finally, SobelConv outputs edge features of the same size as the input features.

[0110] 2.2. The MaxPool module process is as follows:

[0111] First, it goes through a zero-padding layer, filling one zero on the right and bottom;

[0112] Then after maximum pooling, a 2×2 window is used for maximum pooling with a stride of 1.

[0113] The MaxPool module combines zero padding and maximum pooling to help reduce complexity and control overfitting while extracting key features.

[0114] 3. The process of obtaining weights W1, W2, W3, and W4 is as follows:

[0115] First, a 1×1 convolution is performed on the input feature map to reduce the number of feature map channels while maintaining the resolution, and the network's expressiveness is enhanced by combining and reorganizing features.

[0116] Perform global average pooling on the feature map to convert the feature map into the average value of each channel and retain the global information;

[0117] Flatten the feature map into a one-dimensional vector to facilitate input into the fully connected layer;

[0118] Finally, the final weights W1, W2, W3, and W4 are generated through the fully connected layer.

[0119] 4. Perform weighted fusion on the feature map:

[0120] The feature maps generated by the four branches are multiplied by the four weights obtained in the previous step, and multiple feature maps are fused into a new feature map along the channel dimension. This can effectively utilize features at different levels, highlight the most relevant features, and improve the expressiveness of the spliced ​​feature maps.

[0121] 5. Residual Connection

[0122] The feature map obtained in the previous step is residually connected with the model input feature map to promote feature reuse and improve model performance.

[0123] 6. Channel Shuffler

[0124] Divide the feature tensor obtained in the previous step into four groups along the channel, with the number of channels in each group being num_channels / / 4;

[0125] Reshape the feature tensor into a five-dimensional tensor (batchsize, 4, num_channels / / 4, height, width);

[0126] Use the torch.transpose function to swap dimensions 1 and 2 (i.e. 4 and num_channels / / 4). This allows for the exchange of channels within a group;

[0127] Reshape the tensor again into a 4D tensor (batchsize, num_channels, height, width), and the channel order is disrupted.

[0128] The shuffle operation can enhance the representation ability of the feature map. The shuffled feature map can promote information exchange between different channels in the network, thereby improving the performance of the model and introducing more flexibility and expressiveness to the model. The above is the workflow of the EIESA module.

[0129] S3: The three outputs obtained by passing through the RepConv module in step S1 and the two stacked EIESA modules in step S2 are merged after channel shuffling, and the model's ability to process features is improved through one-time aggregation while maintaining computational efficiency. Finally, another RepConv module is passed to further extract features.

[0130] Among them, (3) is to design a deformable large-core separable attention LSKADCN module in the Yolov8 backbone network, and introduce this module into the SPPF module, such as Figure 6 shown.

[0131] SPPF_LSKADCN, which consists of SPPF and LSKADCN, has the following workflow:

[0132] After a 1×1 deep convolution (Conv) and three maximum pooling (Maxpool2d), the feature maps output by these four steps are concatenated by channel, and then passed through an LSKADCN variable large core separable attention module to obtain the final output.

[0133] The LSKADCN module introduces the LSKA large kernel separable attention and DCN deformable convolution.

[0134] For low-light images, local feature information is often scarce, and useful information is usually hidden in dark scenes, making it difficult to extract local features. Therefore, the extraction of global context information is extremely important. By establishing long-distance relationships between pixels, the model can more easily extract global information and make up for the lack of local feature information.

[0135] Therefore, we introduce large kernel separable attention (LSKA). Ordinary CNNs can obtain a large receptive field by stacking many convolution operations or using larger convolution kernels, thereby establishing long-distance relationships between pixels, but this will lead to excessive computation and increase costs. However, research has found that large convolution kernels in CNNs can be simulated by cascading deep convolutions and deep dilated convolutions without increasing excessive computation.

[0136] LSKA further divides the k×k convolution kernel of the above-mentioned deep dilated convolution into cascaded 1×k and k×1 convolution kernels, and introduces an attention module. Through this step, the growth of the number of parameters can be significantly reduced, and long-distance dependence and spatial and channel adaptability can be achieved.

[0137] The above large-core separable attention can infer target information from contextual information, thereby improving the ability to recognize targets under low-light conditions. Finally, the input feature map is weighted by the attention weight to make the network pay more attention to useful information. Given an input feature map Where C is the number of input channels, H and W represent the height and width of the feature map respectively. The output of LSKA is as follows:

[0138]

[0139] Where * and Represents the convolution and the multiplication of the corresponding elements of the feature tensor. C Represents the output of the convolution of the input feature tensor F and the cascade of a depthwise convolution kernel of size 1×(2d-1) and a depthwise convolution kernel of size (2d-1)×1. Each channel C in F is convolved with the corresponding channel in kernel W. Z C represent The output of the convolution with the cascaded 1×(k / d) depth dilated convolution kernel and (k / d)×1 depth dilated convolution kernel, where [.] represents the rounding down operation and d is the dilation rate of the dilated convolution. Then the 1×1 convolution kernel is used to convolve to obtain the attention map A. C .

[0140] is the input feature map F C After attention map A C The result after element-wise weighting.

[0141] Since the outline of the object to be detected in the low-light image is not clear enough and is relatively fuzzy. Therefore, the deformable convolution is introduced to dynamically adjust the position and shape of the convolution kernel, so that the model can better capture the shape and outline of important features in the image and capture more subtle local changes. By replacing the deep convolution in LSKA with a deformable convolution, the size and position of the convolution kernel can be adaptively adjusted according to the feature map, which can adapt to complex backgrounds and changing object shapes, and selectively focus on more important local areas, thereby improving the robustness of the model.

[0142] The implementation process of deformable convolution is:

[0143] First, the input feature map is passed through the convolution layer to generate an offset. The total size of the offset is (B, 2×k×k, H, W), where k represents the size of the convolution kernel. Since the deformable convolution does not change the size of the input feature map, (H, W) is the feature map size of each channel on the output feature map. Each feature map point corresponds to k×k convolution sampling points, each sampling point corresponds to an offset, and each offset consists of a coordinate (x, y). The same offset is used on all channels in a batch.

[0144] The generated offset is used to adjust the sampling points of the standard convolution so that the model focuses on more important areas. The adjusted deformable convolution operation formula is:

[0145]

[0146] In the formula, w(p n ) represents the weight of the corresponding position of the convolution kernel, x(p0+p n +Δp n ) represents p0+p on the input feature map n +Δp n The element value at position y(p0) represents the element value at position p0 on the output feature map, Δp n Indicates the offset.

[0147] After adding the offset, the position is usually a decimal, and bilinear interpolation is used to obtain the offset pixel value.

[0148] The specific operation process of LSKADCN is as follows Figure 7 As shown:

[0149] 1. The input feature map is convolved by a deformable group convolution (DCN) with a kernel size of 3×3, and zero-padded with padding=1 and a stride of 1;

[0150] 2. After the dilated group convolution (1×dD-conv, d×1D-conv, where d is 5) of size 1×5 and 5×1 cascade, it is used for horizontal feature extraction and vertical feature extraction, with the padding size of (0, 4), only padding four pixels on the left and right, and size (4, 0), padding four pixels on the top and bottom. The stride is (1, 1), moving 1 pixel each time in the horizontal and vertical directions, and the dilation rate is set to 2.

[0151] 3. After a 1×1 convolution (1×1conv), the obtained feature map is mixed with information between different channels, so that the network can better learn the relationship between different features and obtain the corresponding attention map.

[0152] 4. Multiply the obtained attention map by the input feature map element by element to obtain the final output feature map. This module enables the model to focus more on the target itself, effectively eliminate background noise, and reduce unnecessary computational consumption.

[0153] The output of LSKADNC is expressed as:

[0154] Attention=Conv 1×1 (DW 5×1 (DW 1×5 (DCN 3×3 (F))))

[0155]

[0156] Where F is the input feature map, DCN is the deformable convolution, and DW is the 1D separable weight convolution kernel.

[0157] Through the above three steps, the Yolov8 model is optimized to obtain a low-light target detection model. The improved network structure is as follows: Figure 8 As shown, it includes the backbone network Backbone, the neck network Neck and the detection head Head.

[0158] The backbone network includes a Global module, a convolution module 1, a convolution module 2, a RCSOBEL module 1, a convolution module 3, a C2f module 1, a convolution module 4, a C2f module 2, a RCSOBEL module 2, and a SPPF_LSKADCN module;

[0159] The neck network includes upsampling module 1, fusion module 1, C2f module 1, upsampling module 2, fusion module 2, C2f module 2, convolution module 1, fusion module 3, C2f module 3, convolution module 2, fusion module 4, and C2f module 4;

[0160] The detection head includes three decoupling heads, and the C2f module 2, C2f module 3 and C2f module 4 of the neck network are respectively input into the three decoupling heads.

[0161] Step 2: Obtain an image data set under low illumination conditions and process it; use the processed data set to train the low illumination target detection model to obtain a trained low illumination target detection model;

[0162] The dataset is selected from the Exdark dataset and converted into the YOLO format using the code and divided into training and test sets;

[0163] The target detection image data in the embodiment of the present invention is selected from the Exdark public dataset, which contains a total of 7,363 low-light images in 12 categories. The images contain 10 different lighting conditions from super low-light environments to ordinary low-light environments. The 12 categories include: bicycles, boats, cars, buses, people, motorcycles, etc.

[0164] First, we use our own python code to convert the images into the YOLO training format. A total of 5896 images are randomly selected from 12 categories as the training set, and the remaining 1467 images are used as the test set. Before the dataset is input into the network, the images in the dataset are preprocessed by rotation, scaling, inversion, random cropping, etc. to achieve data enhancement.

[0165] The low-light target detection model is trained based on the training set to obtain the optimal low-light target detection model.

[0166] The network training was carried out in the pytorch2.0.0 environment, using an RTX 3090 for training. The optimizer was set to SGD, the batch size was 16, and the number of training times was 200. The image size was 640, and the initial learning rate was set to 0.01.

[0167] Step 3: Input the low-light image to be tested into the trained low-light target detection model and output the low-light target detection result. Output the low-light target detection result map, such as Fig. 9 shown.

[0168] Step 4: Verify module validity.

[0169] This experiment uses mAP50 and mAP50:95 as performance reference indicators. mAP50 is the AP of all images in each category when loU is set to 0.5, and then the average of all categories is calculated, that is, mAP. mAP50:95 is the average mAP at different loU thresholds (from 0.5 to 0.95, step size 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). The calculation formulas for each indicator are as follows:

[0170]

[0171] Where TP is the true positive, FP is the false positive, FN is the false negative, P is the precision, R is the recall, and AP is the average precision.

[0172] From the experimental results in Table 1, we can see that the addition of each module helps improve the model performance to a certain extent. This proves that the proposed module is effective in improving the performance of the low-light target detection model. The overall mAP50 in low-light target detection is improved by 2.4%.

[0173] Table 1 Comparison of adding experiments in each module

[0174] Experimental content mAP50 mAP50:95 Yolov8n 0.652 0.397 Yolov8n+Global 0.661 0.406 Yolov8n+RCSOBEL 0.658 0.643 Yolov8n+LSKADCN 0.660 0.404 Yolov8n+Global+LSKADCN 0.671 0.41 Yolov8n+Global+RCSOBEL+LSKADCN 0.676 0.411

[0175] Under the premise of keeping the experimental environment unchanged, the model of the present invention is compared with the currently commonly used target detection algorithms: YOLOv5n, Yolov8n, YOLOv10n, and RT-DETR. The performance comparison is shown in Figure 2:

[0176] Table 2 Comparison of target detection algorithms

[0177]

[0178]

[0179] The improved method proposed by the model of the present invention has certain advantages, which is 2.4% higher than the original Yolov8n in mAP50, 4.8% higher than YOLOv10n in mAP50, higher than YOLOv5 in mAP50, and higher than RT-DETR in mAP50. Because the algorithm model of the present invention adopts image adaptive processing to convert low-light images into a form that is easier to detect targets, and uses a feature extraction module with a multi-scale receptive field to fully extract edge feature information and spatial information lacking in low-light images, and also uses a deformable large-core separable attention module to reduce the defect of unclear target contours caused by low light, so that the model's attention is more focused on useful information, and fully utilizes the surrounding context information to make up for the lack of local detail features, thereby enhancing the model's ability to detect low-light targets.

[0180] The present invention proposes a low-illumination target detection method with multi-scale receptive field under adaptive enhancement, which can achieve good detection effect in low-illumination images. The present invention is suitable for image target detection generated under various insufficient lighting conditions, such as night monitoring and unmanned driving.

[0181] The present invention adds a Globel module for image adaptive enhancement to the backbone network, uses a self-attention mechanism, utilizes image global information to generate parameters for adjusting image information, and performs end-to-end training to make the generated image more conducive to target detection.

[0182] The present invention designs an EIESA module for feature extraction that can learn richer feature representations. The module combines the SobelConv branch for extracting edge information, the repVGG branch for extracting diverse spatial information, and the maximum pooling Maxpool branch for extracting key features to obtain more diverse information, which can effectively utilize the limited information of low-light images and improve detection accuracy.

[0183] This paper designs a variable large-kernel separable attention structure LSKADCN module, which uses LSKA to obtain a richer field of view at a lower cost, has spatial and channel adaptability, effectively solves the problem of incorrect allocation of model attention, and filters noise. At the same time, variable convolution is introduced to obtain more information that is beneficial to the target itself and reduce useless computational consumption.

[0184] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this article.

[0185] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0186] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0187] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0188] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0190] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.

Claims

1. A low-light target detection method with multi-scale receptive field under adaptive enhancement, characterized by: The following steps are involved: Step 1: By optimizing the Yolov8 target detection algorithm, a low-light target detection model based on image adaptive enhancement and multi-scale receptive field is constructed; including: The Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection; the RCSOBEL module is designed in Yolov8 to learn richer feature representations from features and strengthen the extraction of edge information and spatial information; the deformable large kernel separable attention LSKADCN module is designed in Yolov8 to transform the SPPF module in Yolov8 to obtain the LKSPPF module, so that the network focuses on key feature information; Step 2: Obtain an image data set under low illumination conditions and process it; use the processed data set to train the low illumination target detection model to obtain a trained low illumination target detection model; Step 3: Input the low-light image to be tested into the trained low-light target detection model, and output the low-light target detection result.

2. According to claim 1, the method for low-light target detection with multi-scale receptive field under adaptive enhancement is characterized by: The low-light target detection model includes a backbone network, a neck network and a detection head; wherein The backbone network includes a Global module, a convolution module 1, a convolution module 2, a RCSOBEL module 1, a convolution module 3, a C2f module 1, a convolution module 4, a C2f module 2, a RCSOBEL module 2, and a SPPF_LSKADCN module; The neck network includes upsampling module 1, fusion module 1, C2f module 1, upsampling module 2, fusion module 2, C2f module 2, convolution module 1, fusion module 3, C2f module 3, convolution module 2, fusion module 4, and C2f module 4; The detection head includes three decoupling heads, and the C2f module 2, C2f module 3 and C2f module 4 of the neck network are respectively input into the three decoupling heads.

3. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 1, characterized in that: The Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection, including: After two 3×3 convolution expansion channels, high-dimensional features are encoded at low resolution to capture global information in the image; The cross-attention module obtains a 3×3 color matrix and γ value from the high-dimensional features; The color matrix and γ are subjected to two 1×1 convolutions respectively to further process and optimize the parameters; Perform matrix product of the original image and the color matrix, and perform γ power calculation on the result to obtain the final processed image, which is input into the subsequent network.

4. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 3, characterized in that: The cross-attention module obtains a 3×3 color matrix and γ value from high-dimensional features, including: First, the input feature map is convolved with a depth-wise convolution of 3×3 size, and then passed through two fully connected layers to generate the value (V) and key (K) required in the self-attention mechanism. The query (Q) is randomly initialized, and the attention score matrix is ​​calculated using Q and K. The final output result of the Attention layer is calculated using V and the attention score matrix, which can be expressed as follows: Where Q (Query): query matrix; K (Key): key matrix; V (Value): value matrix; d k : Dimension of the key matrix; After passing through a multi-layer perceptron, ten parameters are finally output, representing a 3x3 color matrix and a 1-dimensional gamma (γ) value.

5. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 3, characterized in that: The matrix product of the original image and the color matrix is ​​performed, and the result is subjected to γ ​​power calculation, including: color correction and gamma correction of the image, which is expressed as follows: In the formula is a 3x3 color matrix, γ is the power value of gamma correction, and the value of ε is 10 -6 , to prevent the pixel value from being set to 0.

6. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 1, characterized in that: The RCSOBEL module is designed in Yolov8 to learn richer feature representations from features and enhance the extraction of edge information and spatial information; including: The input feature map passes through the RepConv module; passes through the EIESA module twice; the three output channels are spliced ​​together; and passes through a RepConv module to obtain the final feature map.

7. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 6, characterized in that: The RepConv module is composed of three parallel direct outputs, 3×3 convolutions and 1×1 convolutions during training. Finally, the obtained feature maps are added and passed through the activation function SiLu. During reasoning, the above module is re-parameterized into a 3×3 convolution plus SiLu activation function, reducing parameters while ensuring accuracy.

8. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 6, characterized in that: The EIESA module workflow includes: The input feature map passes through the convolution layer and the pooling layer, and then the feature map is flattened to (batch_size, feature_dim) size, where batch_size refers to the number of samples in each iteration, and feature_dim is the size obtained after flattening the feature map. Then it passes through the fully connected layer and the softmax normalized weight to obtain four weight values; The input feature map is evenly divided into four feature maps in the channel direction, each of which passes through the SobelConv module, the RepConv module, the maximum pooling module and the direct output channel to obtain four feature maps; The obtained feature maps are multiplied by the weight values ​​respectively, and after residual connection, they are fused in the channel direction, and finally passed through the channel shuffle module to shuffle the channels to obtain the final feature map.

9. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 1, characterized in that: The LSKADNC module consists of LSKA and deformable convolution, and the workflow includes: The feature map undergoes a 3×3 deformable group convolution, which allows attention to change adaptively according to data features and focus on key information hidden in the dark. It then passes through two cascaded 1D separable weighted convolution kernels composed of deep hole group convolutions, where the sizes of the convolution kernels are (1, 5) and (5, 1), respectively. In addition, each channel in the feature map is convolved with the corresponding channel in the convolution kernel. After 1×1 convolution, cross-channel feature fusion is performed, and finally the attention map is inferred; Multiply the output attention map and the input features for adaptive feature refinement; The output of LSKADNC is expressed as: Attention=Conv 1×1 (DW 5×1 (DW 1×5 (DCN 3×3 (F)))) Where F is the input feature map, DCN is the deformable convolution, and DW is the 1D separable weight convolution kernel.

10. A low-light target detection device with multi-scale receptive field under adaptive enhancement, characterized in that: It comprises a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method as described in any one of claims 1 to 9.

Images