A low-illumination target detection method of self-adaptive enhancement of multi-scale receptive field

Abstract

The application provides a low-illumination target detection method of self-adaptive enhancement of multi-scale receptive field, comprising the following steps: step one: introducing a Global module, an RCSOBEL module and an LSKADCN module to optimize a yolov8 target detection algorithm, and constructing a low-illumination target detection model based on image self-adaptive enhancement and multi-scale receptive field; step two: acquiring an image dataset under low-illumination conditions and processing the dataset; training the low-illumination target detection model using the processed dataset to obtain a trained low-illumination target detection model; and step three: inputting a low-illumination image to be detected into the trained low-illumination target detection model to output a low-illumination target detection result. The application can achieve good detection effect in low-illumination images, and is suitable for image target detection generated under various insufficient light conditions, such as night monitoring and unmanned driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of low-illumination image processing, in particular to a low-illumination target detection method for adaptively enhancing a multi-scale receptive field. BACKGROUND

[0002] Target detection is a technology that automatically identifies and locates target objects in images or videos through computer algorithms, and is an important research direction in the field of computer vision. Currently, target detection technology has been widely applied in many fields, including intelligent transportation, security monitoring, medical image analysis, agriculture, etc. With the continuous development of technology, target detection technology will further improve accuracy and efficiency, bringing innovation and progress to more fields. However, lighting conditions are one of the important factors affecting the accuracy of target detection. In practical applications, such as all-time monitoring, unmanned driving, etc., target detection needs to be performed under low illumination. In this special environment, the performance of existing deep learning-based target detection algorithms will be greatly reduced. Most current target detectors are studied in high-quality images under normal conditions. In real environments, there are often night, dim light, exposure, and many other poor lighting conditions, which degrade image quality and affect the performance of the detector.

[0003] Currently, target detection technology represented by Yolov8 can achieve good accuracy under normal lighting conditions, but still has defects under low illumination. Existing low-illumination target detection methods often use a low-illumination image enhancement module as a preprocessing module before sending it to the target detector. This method usually ignores the differences between image enhancement and target detection tasks, and the enhanced image is more suitable for human eye observation rather than target detection, and does not fully utilize the feature information of the image. SUMMARY

[0004] In view of the deficiencies in the prior art, the present application aims to provide a low-illumination target detection method for adaptively enhancing a multi-scale receptive field, which can solve the existing problems.

[0005] In order to achieve the above-mentioned purpose, the technical solution of the present application is as follows:

[0006] The present application is realized by the following technical solution: a low-illumination target detection method for adaptively enhancing a multi-scale receptive field, comprising the following steps:

[0007] Step 1: optimize the Yolov8 target detection algorithm to construct a low-illumination target detection model based on image adaptive enhancement and multi-scale receptive field; including:

[0008] A Global module is introduced before Yolov8 to generate a color matrix and a gamma value of global information of a predicted control image, so that the image is converted into a form more suitable for target detection; an RCSOBEL module is designed in Yolov8 to learn more rich feature representations from features, so as to strengthen the extraction of edge information and spatial information; a LSKADCN module of deformable large kernel separable attention is designed in Yolov8 to transform the SPPF module in Yolov8 to obtain an LKSPPF module, so that the network focuses on key feature information;

[0009] Step two: obtaining an image data set under low-illumination conditions and processing; training the low-illumination target detection model using the processed data set to obtain a trained low-illumination target detection model;

[0010] Step three: inputting a low-illumination image to be tested into the trained low-illumination target detection model to output a low-illumination target detection result.

[0011] Further, the low-illumination target detection model comprises a backbone network, a neck network and a detection head; wherein

[0012] The backbone network comprises a Global module, a convolution module one, a convolution module two, an RCSOBEL module one, a convolution module three, a C2f module one, a convolution module four, a C2f module two, an RCSOBEL module two and an SPPF_LSKADCN module.

[0013] The neck network comprises an up-sampling module one, a fusion module one, a C2f module one, an up-sampling module two, a fusion module two, a C2f module two, a convolution module one, a fusion module three, a C2f module three, a convolution module two, a fusion module four and a C2f module four.

[0014] The detection head comprises three decoupling heads, and the C2f module two, the C2f module three and the C2f module four of the neck network are respectively input into the three decoupling heads.

[0015] Further, the Global module is introduced before Yolov8 to generate a color matrix and a gamma value of global information of a predicted control image, so that the image is converted into a form more suitable for target detection, comprising:

[0016] After two 3x3 convolution expansion channels, low-resolution high-dimensional features are encoded to capture global information in the image;

[0017] After passing through a cross-attention module, a 3x3 color matrix and a gamma value are obtained from high-dimensional features;

[0018] The color matrix and the gamma are respectively processed and optimized by two 1x1 convolutions.

[0019] The original image and the color matrix are multiplied, and the obtained result is calculated by gamma power to obtain a final processing image, which is input into a subsequent network.

[0020] Further, the cross-attention module obtains a 3x3 color matrix and a gamma value from the high-dimensional feature, including:

[0021] First, the input feature map is subjected to a 3x3 size depth-wise convolution, and then subjected to two fully connected layers to generate the value (V) and key (K) required in the self-attention mechanism, respectively, and the query (Q) is randomly initialized, and the attention score matrix is calculated using Q and K, and the final output result of the Attention layer is calculated using V and the attention score matrix, which is expressed by the formula:

[0022] ;

[0023] In the formula, Q: query matrix; K: key matrix; V: value matrix; d k : the dimension of the key matrix;

[0024] Then, a multi-layer perceptron is passed through, and finally ten parameters are output, representing a 3x3 color matrix and a 1-dimensional gamma (γ) value.

[0025] Further, the original image and the color matrix are multiplied, and the obtained result is calculated by gamma power, including color correction and gamma correction of the image, which is expressed by the formula:

[0026]

[0027] In the formula, is a 3x3 color matrix, and γ is the power value of gamma correction, The value of , to prevent the pixel value from being 0.

[0028] Further, the RCSOBEL module is designed in Yolov8 to learn more rich feature representations from the features and strengthen the extraction of edge information and spatial information, including:

[0029] The input feature map is subjected to a RepConv module, subjected to two EIESA modules respectively, and the three output channels are spliced together, and subjected to a RepConv module to obtain a final feature map.

[0030] Furthermore, the RepConv module is composed of three parallel direct outputs, 3×3 convolutions, and 1×1 convolutions during training. Finally, the obtained feature maps are added and passed through the activation function SiLu. During inference, the above module is re-parameterized into a 3×3 convolution plus SiLu activation function, reducing parameters while ensuring accuracy.

[0031] Furthermore, the EIESA module workflow includes:

[0032] The input feature map passes through the convolution layer and the pooling layer, and then the feature map is flattened to the size of (batch_size, feature_dim), where batch_size refers to the number of samples in each iteration, and feature_dim is the size obtained after flattening the feature map. It then passes through the fully connected layer and the softmax normalization weight to obtain four weight values;

[0033] The input feature map is evenly divided into four feature maps along the channel direction, where each feature map passes through the SobelConv module, the RepConv module, the maximum pooling module and the direct output channel to obtain four feature maps;

[0034] The obtained feature maps are multiplied by the weight values, respectively, and after the residual connection, they are fused in the channel direction. Finally, the channels are shuffled by the channel shuffle module to obtain the final feature map.

[0035] Furthermore, the LSKADCN module consists of LSKA and deformable convolution, and the workflow includes:

[0036] The feature map undergoes a 3×3 deformable group convolution, which allows attention to adapt to data features and focus on key information hidden in the dark. It then passes through two cascaded 1D separable weighted convolution kernels composed of deep hole group convolution, where the convolution kernel sizes are (1, 5) and (5, 1), respectively. In addition, each channel in the feature map is convolved with the corresponding channel in the convolution kernel.

[0037] After 1×1 convolution, cross-channel feature fusion is performed and the attention map is finally inferred;

[0038] Multiply the output attention map and input features for adaptive feature refinement;

[0039] The output of LSKADCN is expressed as:

[0040]

[0041] ;

[0042] In the formula, F is an input feature map, DCN is a deformable convolution, and DW is a 1D separable weight convolution kernel.

[0043] The application also protects a low-illumination target detection device for adaptively enhancing a multi-scale receptive field, comprising a processor and a memory; the memory is used for storing a program; and the processor executes the program to realize the method in any one of the above.

[0044] Compared with the prior art, the beneficial effects of the application include:

[0045] The application proposes a low-illumination target detection method for adaptively enhancing a multi-scale receptive field, which can achieve good detection effect in a low-illumination image, and is suitable for image target detection generated under various insufficient light conditions, such as night monitoring and unmanned driving.

[0046] The application adds a Globel module for image adaptive enhancement to a backbone network, uses a self-attention mechanism, generates parameters for adjusting image information by using global image information, and performs end-to-end training, so that the generated image is more conducive to target detection.

[0047] The application designs an EIESA module for feature extraction that can learn more rich feature representations, which combines a SobelConv branch for extracting edge information, a repVGG branch for extracting diverse spatial information, and a Maxpool branch for extracting key features, obtains more diverse information, can effectively utilize limited information of a low-light image, and improves detection accuracy.

[0048] The application designs a variable large kernel separable attention structure LSKADCN module, uses LSKA to obtain more rich visual fields at a smaller cost, has spatial and channel adaptability, effectively solves the problem of attention error distribution of the model, and filters noise. Meanwhile, variable convolution is introduced to obtain more information conducive to the target itself and reduce useless calculation consumption. BRIEF DESCRIPTION OF DRAWINGS

[0049] The disclosure of the application will be described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the application. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0050] Figure 1 It is a flowchart of the low-illumination target detection method of the application for adaptively enhancing a multi-scale receptive field;

[0051] Figure 2 It is a structure diagram of the Globle module in the embodiment of the application;

[0052] Figure 3 A structure diagram of the RCSOBEL module in the embodiment of the application;

[0053] Figure 4 A structure diagram of the EIESA module in the embodiment of the application;

[0054] Figure 5 A structure diagram of the RepConv module in the embodiment of the application;

[0055] Figure 6 A structure diagram of the SPPF_LSKADCN module in the embodiment of the application;

[0056] Figure 7 A structure diagram of the LSKADCN module in the embodiment of the application;

[0057] Figure 8 A structure diagram of the low-illumination image detection network model in the embodiment of the application;

[0058] Figure 9 An effect diagram of detection in the embodiment of the application. DETAILED DESCRIPTION

[0059] It is easy to understand that, according to the technical solution of the application, a person skilled in the art can propose a plurality of structure modes and implementation modes that can be replaced with each other without changing the essential spirit of the application. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the application, and should not be regarded as the whole or as a limitation or restriction on the technical solution of the application.

[0060] A low-illumination target detection method based on adaptive enhancement and multi-scale receptive field is proposed in this paper. First, an image adaptive enhancement module is used to adjust the image based on global information, generating an image more suitable for target detection. Then, multi-scale receptive field and attention mechanism using large-scale convolution kernel are used to more fully extract and utilize feature information in each dimension. Finally, the proposed method can greatly improve the detection accuracy.

[0061] The low-illumination target detection method based on adaptive enhancement and multi-scale receptive field of the application specifically includes the following steps:

[0062] Step one: optimize the Yolov8 target detection algorithm to construct a low-illumination target detection model based on image adaptive enhancement and multi-scale receptive field; specifically including:

[0063] (1) Introduce a Global module before Yolov8 to generate a color matrix and gamma value of the global information of the prediction control image, so that the image is converted into a form more suitable for target detection;

[0064] (ii) Design an RCSOBEL module in Yolov8 to learn richer feature representations from features, strengthen the extraction of edge information and spatial information;

[0065] (iii) Design a deformable large kernel separable attention LSKADCN module in Yolov8 to modify the SPPF module in Yolov8 to obtain an LKSPPF module, so that the network focuses on key feature information;

[0066] The following will be described in detail in combination with specific implementation cases:

[0067] Among them, (i) is to introduce a Global module before Yolov8 to generate a color matrix and a gamma value of the predicted control image global information, so that the image is converted into a form more suitable for target detection; improve the ability of the model in low light target detection.

[0068] The principle of the Global module is that various bad lighting scenes existing in natural scenes will affect the pictures obtained when performing visual tasks, and low-light images are usually more suitable for human eye observation than target detection. In order to solve this problem, the model of the present application introduces a Global module to predict the parameters required for image detection according to the global information of the image, so that the image is adaptively adjusted to overcome the influence of bad lighting and is more suitable for detection.

[0069] The working process of the Global module is shown in Figure 2 , which includes the following four steps:

[0070] S1, expand the channel by two 3x3 convolutions, encode high-dimensional features at low resolution, and capture global information in the image; specifically: first, perform down-sampling processing on the image, use two 3x3 convolution layers (3x3 Conv), the step is 2, the activation function is GELU function, and batch normalization is used, and the channel numbers of the two convolution layers are 23 and 64 respectively. This step can convert the image into low resolution, encode high-level features at low resolution. On the one hand, it can save computing cost and help lightweight design, on the other hand, it is helpful to extract global features.

[0071] S2, obtain a 3x3 color matrix and a gamma value from the high-dimensional feature through the cross-attention module, specifically: pass the generated feature to the subsequent parameter prediction module, first pass the feature through a 3x3 depthwise convolution (3x3WConv) with a size of 3x3, the main feature of which is to apply convolution operation independently for each input channel, which can significantly reduce the number of parameters and computational complexity of the model. Given the shape of the input tensor X as (B, H, W, C), where B is the size, H is the height, W is the width, and C is the number of channels. For each channel C, the depthwise convolution operation can be represented as: . is the result of the output channel C, is the input channel C, is the convolution kernel for channel C. The subsequent connection residual module ensures the stability of the model while extracting features;

[0072] perform self-attention calculation on the obtained image features,

[0073] ;

[0074] where Q: query matrix; K: key matrix; V: value matrix; d k : the dimension of the key matrix;

[0075] According to the above formula, the following steps are included:

[0076] 1. Send the image features into two fully connected layers (FC) respectively, and adjust the dimensions to obtain a key and value with a size of 1x10. The query is initialized as a zero vector with a size of 1x10.

[0077] 2. Perform dot product of query and key to obtain attention score, then multiply by scaling factor to alleviate numerical instability and improve gradient flow, so that the model performs more stably and efficiently during training and inference.

[0078] 3. Perform softmax normalization (LN) on the attention score to obtain attention weight. And apply dropout operation to the attention weight to improve the generalization ability of the model.

[0079] 4. Apply attention weight to value to perform weighted summation to obtain attention output.

[0080] Finally, a linear layer is used to project the attention output to enhance the expressive power, capture more complex patterns and features. Finally, a dropout operation is performed to improve the robustness of the model, and the processed output is obtained. The above steps implement the self-attention mechanism, which can capture long-range dependencies and global information in the input sequence. Through multi-head attention, the model can simultaneously learn multiple different attention distributions.

[0081] S3, the color matrix and gamma are respectively subjected to two 1x1 convolutions, and the parameters are further processed and optimized. Specifically, the obtained output features are further subjected to a simple multi-layer perceptron, including two 1x1 convolution layers, each of which is followed by an activation function (GELU function) and a Dropout layer. The 1x1 convolution layer can maintain the same resolution in the spatial dimension while increasing the number of channels and the complexity of feature representation. The activation function used is the GELU function, which increases nonlinearity and enhances the expressive power of the model. Finally, ten parameters are output, representing the 3x3 color matrix and the 1-dimensional gamma (gamma) value.

[0082] S4, matrix multiplication is performed on the original image and the obtained color matrix, and the result is calculated by gamma power. The purpose of the color matrix is to ensure that the color of the object is constant under different lights, and at the same time, the image color is converted to a pixel color suitable for detection. The obtained gamma value is used for gamma correction, which is a non-linear transformation. By adjusting the Gamma value, the brightness and contrast of the image can be flexibly controlled to achieve a more ideal effect. When the gamma value is greater than 1, the highlight area (high light part) of the image will be compressed, reducing the details of these areas, while the low brightness area (shadow part) will be expanded, increasing the visibility of the details. In this case, the overall image will appear darker, emphasizing the shadow part more; when the gamma value is less than 1, the highlight part of the image will be expanded, meaning that the highlight area will become brighter and clearer, while the low brightness area will be compressed, reducing the visibility of the details. This will make the overall image appear brighter, enhancing the expressiveness of the highlight area.

[0083] which is expressed by the formula, ;

[0084] wherein, represents the brightness value of the input pixel, usually between 0 and 1 (when processing 8-bit images, the range is 0 to 255). represents the output pixel brightness value after gamma correction.

[0085] The matrix multiplication of the original image and the color matrix, and the gamma power calculation of the obtained result, include color correction and gamma correction of the image, which is expressed by the formula:

[0086]

[0087] wherein is a 3x3 color matrix, and γ is a power value of gamma correction, The value of is , to prevent the pixel value from being 0. Finally, the processed image is obtained.

[0088] wherein (ii) is to design an RCSOBEL module in the Yolov8 backbone network, which is inspired by the RCS-OSA module and replaces the original C2f module in the backbone network. This module can improve the efficiency of the network in processing dense connections. By aggregating all the diversified features of multi-scale receptive fields in the last feature mapping, the problem of low efficiency of dense connections can be overcome. The workflow of the RCSOBEL module is shown in Figure 3 , which includes the following three steps:

[0089] S1, the obtained feature map is subjected to a RepConv module, the principle of which is shown in Figure 5 .

[0090] In the training stage, the model is shown in Figure 5 a figure, which has a multi-branch topology structure and contains multiple branches, 1x1 convolution layers (1x1Conv), 3x3 convolution layers (3x3Conv) and identity mapping layers. These branches can learn different features. The feature maps obtained by the three branches are fused by addition, and then passed through the activation function SiLu to form the final output feature map.

[0091] In the inference stage, the model is shown in Figure 5 b figure, in order to improve the calculation efficiency and speed, the multi-branch structure is converted into a single branch structure. The parameters of the multi-branch used in training are fused into a single 3x3 convolution kernel (3x3Conv) plus SiLu activation function, which can significantly improve the calculation speed and reduce the delay. The output channel number of the feature map is 1 / 2 of the input channel number. The idea of reparameterization is: both the 1x1 convolution in the training stage and the two paths that do not undergo processing are converted into a 3x3 convolution, and then fused.

[0092] 1x1 to 3x3: supplement a circle of 0 outside the 1x1 convolution kernel to form a 3x3 convolution kernel.

[0093] 0x0 to 3x3: supplement a circle of 0 outside the weight size of BN to form a 3x3 convolution kernel.

[0094] The fusion process is expressed by the formula as:

[0095]

[0096] where fuse represents the convolution operation after the convolution and BN fusion.

[0097] The multi-branch topology can learn rich feature information during training, and the simplified single-branch structure during inference can save memory consumption of inference, and realize fast inference.

[0098] S2, the feature map obtained through RepConv is subjected to two stacked EIESA modules, and the features are cascaded to ensure feature reuse and strengthen information flow between different layers.

[0099] The workflow of the EIESA module is as shown in Figure 4 The process is as follows:

[0100] 1. Split the input feature map in the channel direction, and divide it into four equal parts.

[0101] 2. The obtained split feature map is respectively subjected to a SobelConv module, a RepConv module, a MaxPool module and an identity mapping; specifically

[0102] 2.1. The working principle of SobelConv is as follows: Sobel filter is an edge detection method that can detect sudden changes in intensity in an image, so it can well capture the edge features of the image, thereby obtaining important edge information. SobelConv efficiently implements the Sobel operator through 3D group convolution, and the group size is the input channel number.

[0103] 3D group convolution uses two 3x3 convolution kernels to calculate the horizontal and vertical gradients, respectively:

[0104] Horizontal gradient kernel , vertical gradient kernel ;

[0105] For each feature point, the horizontal and vertical gradients can be obtained by applying the above convolution kernel, and the formula is:

[0106]

[0107] ;

[0108] In the formula, the horizontal direction gradient: Gx(x,y); I represents the image, Gx represents the horizontal convolution kernel, and Gy represents the vertical convolution kernel; The vertical direction gradient: Gy(x,y).

[0109] In order to facilitate calculation, the gradient amplitude uses the approximate gradient amplitude: .

[0110] Finally, SobelConv outputs edge features of the same size as the input features.

[0111] 2.2. The MaxPool module process is as follows:

[0112] First, it passes through a zero-padding layer, filling one zero on the right and one zero on the bottom;

[0113] Then, after max pooling, a 2×2 window is used for max pooling with a stride of 1.

[0114] The MaxPool module combines zero padding and maximum pooling to help reduce complexity and control overfitting while extracting key features.

[0115] 3. The process of obtaining weights W1, W2, W3, and W4 is as follows:

[0116] First, a 1×1 convolution is performed on the input feature map to reduce the number of feature map channels while maintaining the resolution, and enhance the network's expressive power by combining and recombining features.

[0117] Perform global average pooling on the feature map to convert the feature map into the average value of each channel and retain global information;

[0118] Flatten the feature map into a one-dimensional vector to facilitate input into the fully connected layer;

[0119] Finally, the final weights W1, W2, W3, and W4 are generated through the fully connected layer.

[0120] 4. Perform weighted fusion on the feature map:

[0121] The feature maps generated by the four branches are multiplied by the four weights obtained in the previous step, and multiple feature maps are fused into a new feature map along the channel dimension. This can effectively utilize features at different levels, highlight the most relevant features, and improve the expressiveness of the spliced ​​feature map.

[0122] 5. Residual Connection

[0123] Perform a residual connection between the feature map obtained in the previous step and the model input feature map to promote feature reuse and improve model performance.

[0124] 6. Channel Shuffler

[0125] Divide the feature tensor obtained in the previous step into four groups along the channel, with the number of channels in each group being num_channels / / 4;

[0126] Reshape the feature tensor to a five-dimensional tensor (batchsize, 4, num_channels / / 4, height, width);

[0127] Use the torch.transpose function to exchange dimensions 1 and 2 (i.e., 4 and num_channels / / 4). This allows for the exchange of channels within groups;

[0128] Reshape the tensor again to a four-dimensional tensor (batchsize, num_channels, height, width), and the channel order is now scrambled.

[0129] Shuffling enhances the representational capacity of the feature map. The shuffled feature map can facilitate information exchange between different channels in the network, thereby improving the performance of the model and introducing more flexibility and expressiveness into the model. The above is the workflow of the EIESA module.

[0130] S3, merge the three outputs obtained by the RepConv module in S1 and the two stacked EIESA modules in S2 after channel shuffling, improve the model's ability to process features through one-time aggregation, while maintaining computational efficiency. Finally, pass it through another RepConv module to further extract features.

[0131] Among them, (three) is to design a deformable large kernel separable attention LSKADCN module in the Yolov8 backbone network, and introduce this module in the SPPF module, as shown in Figure 6 .

[0132] SPPF_LSKADCN, composed of SPPF and LSKADCN, its workflow is as follows:

[0133] After a 1x1 size deep convolution (Conv), three max pooling (Maxpool2d), the feature map output after these four steps is concatenated by channel (concat), and then passed through an LSKADCN deformable large kernel separable attention module to get the final output.

[0134] The LSKADCN module introduces LSKA large kernel separable attention and DCN deformable convolution.

[0135] For low-light images, local feature information is often scarce, and useful information is often hidden in dark scenes, making it difficult to extract local features. Therefore, the extraction of global context information is extremely important. By establishing long-distance relationships between pixels, the model can more easily extract global information and make up for the lack of local feature information.

[0136] Therefore, we introduce large kernel separable attention LSKA. Common CNN can obtain large receptive field by stacking many convolution operations or using large convolution kernel, thereby establishing long-distance relationship between pixels, but it will cause high computational cost and increase cost. But research shows that large convolution kernel in CNN can be simulated by cascading deep convolution and deep dilated convolution, and will not increase high computational cost.

[0137] LSKA further divides the k x k convolution kernel of the above deep convolution kernel deep dilated convolution into cascaded 1 x k and k x 1 size convolution kernels, and introduces an attention module. Through this operation, the parameter quantity can be significantly reduced, and long-distance dependence and spatial and channel adaptability can be realized.

[0138] Using the above large kernel separable attention, the target information can be inferred by the context information, thereby improving the recognition ability of the target under low light conditions. Finally, the input feature map is weighted by the attention weight to make the network pay more attention to useful information. Given an input feature map , where C is the input channel number, H and W represent the height and width of the feature map respectively, and the output result of LSKA is as follows:

[0139]

[0140]

[0141]

[0142]

[0143] In the formula, * and respectively represent the multiplication of the corresponding elements of the convolution and the feature tensor. represents the output of the convolution of the input feature tensor F and the cascaded deep convolution kernel with a size of 1 x (2d-1) and the deep convolution kernel with a size of (2d-1) x 1. Each channel C in F is convolved with the corresponding channel in the kernel W. represents and the cascaded deep dilated convolution kernel with a size of 1 x (k / d) and the deep dilated convolution kernel with a size of (k / d) x 1, where [.] represents the down rounding operation, and d is the expansion rate of the dilated convolution. Then, a 1 x 1 convolution kernel is used for convolution to obtain an attention map . is the input feature map after the attention map is weighted element by element.

[0144] Because the outline of the object to be detected in the low-light image is not clear enough, it is relatively blurred. Therefore, deformable convolution is introduced to dynamically adjust the position and shape of the convolution kernel, so that the model can better capture the shape and outline of important features in the image and capture more subtle local changes. By replacing the deep convolution in LSKA with deformable convolution, the size and position of the convolution kernel can be adaptively adjusted according to the feature map, which can adapt to complex backgrounds and changing object shapes, selectively focus on more important local areas, and thus improve the robustness of the model.

[0145] The implementation process of deformable convolution is as follows:

[0146] First, the input feature map is passed through the convolution layer to generate an offset, and the total size of the offset is (B, 2xkxk, H, W), k represents the size of the convolution kernel, and since deformable convolution does not change the size of the input feature map, (H, W) is the feature map size of each channel on the output feature map. Each feature map point corresponds to kxk convolution sampling points, and each sampling point corresponds to an offset, and each offset consists of a coordinate (x, y). The same offset is used for all channels in a batch.

[0147] The generated offset is used to adjust the sampling points of the standard convolution, so that the model focuses attention on more important areas. The adjusted deformable convolution operation formula is as follows:

[0148]

[0149] In the formula, represents the weight of the corresponding position of the convolution kernel, represents the element value of the input feature map at the position, represents the element value of the output feature map at the position, represents the offset. The position after adding the offset is usually a decimal number, and the pixel value after the offset is obtained using bilinear interpolation.

[0150] The specific operation process of LSKADCN is shown in .

[0151] Figure 7 1. The input feature map is passed through the deformable group convolution (DCN) with a convolution kernel size of 3x3, and zero padding with padding=1 and a stride of 1 is performed;

[0152] 1. The input feature map is passed through the deformable group convolution (DCN) with a convolution kernel size of 3x3, and zero padding with padding=1 and a stride of 1 is performed;

[0153] ​2. Dilated group convolution (1xd D-conv, dx1 D-conv, where d is 5) with size 1x5 and 5x1 cascaded, for horizontal feature extraction and vertical feature extraction, padding size (0, 4) and (4, 0) respectively, padding four pixels on the left and right and four pixels on the top and bottom respectively. The strides are both (1, 1), moving 1 pixel in the horizontal and vertical directions each time, and the dilation rate is set to 2.

[0154] 3. A 1x1 convolution (1x1conv) is performed on the obtained feature map to mix the information between different channels, so that the network can better learn the relationship between different features, and the corresponding attention map is obtained.

[0155] 4. The obtained attention map is multiplied element by element with the input feature map to obtain the final output feature map. This module makes the model focus more on the target itself, effectively eliminates background noise, and reduces unnecessary computational consumption.

[0156] The output of LSKADCN is expressed by the formula:

[0157]

[0158]

[0159] In the formula, F is the input feature map, DCN is deformable convolution, and DW is 1D separable weight convolution kernel.

[0160] Through the above three steps, a low-light target detection model is obtained by optimizing the Yolov8 model, and the improved network structure is as shown in Figure 8 , which includes a backbone network, a neck network, and a head.

[0161] The backbone network includes a Global module, a convolution module one, a convolution module two, an RCSOBEL module one, a convolution module three, a C2f module one, a convolution module four, a C2f module two, an RCSOBEL module two, and an SPPF_LSKADCN module.

[0162] The neck network includes an upsampling module one, a fusion module one, a C2f module one, an upsampling module two, a fusion module two, a C2f module two, a convolution module one, a fusion module three, a C2f module three, a convolution module two, a fusion module four, and a C2f module four.

[0163] The head includes three decoupled heads, and the C2f module two, the C2f module three, and the C2f module four of the neck network are respectively input to the three decoupled heads.

[0164] Step two: obtain an image dataset under low-illumination conditions and process it; use the processed dataset to train the low-illumination target detection model to obtain a trained low-illumination target detection model;

[0165] The dataset is selected from the Exdark dataset, and the code is used to convert it into the YOLO format, and it is divided into a training set and a test set;

[0166] In the embodiment of the application, the target detection image data is selected from the Exdark public dataset, which contains a total of 7363 low-light images of 12 categories. The images contain a total of 10 different light conditions from super-weak light environment to ordinary weak light environment. The 12 categories include: bicycle, ship, car, bus, person, motorcycle and the like.

[0167] First, the self-made python code is used to convert the image into the YOLO training format. A total of 5896 pictures are randomly selected from the 12 categories as the training set, and the remaining 1467 pictures are used as the test set. Before the dataset is input into the network, the pictures in the dataset are preprocessed, such as rotation, scaling, inversion, random cropping, etc. to realize data enhancement.

[0168] Based on the training set, a low-light target detection model is trained to obtain an optimal low-light target detection model.

[0169] The training of the network is carried out in the pytorch2.0.0 environment, and an RTX 3090 is used for training. The optimizer is set to SGD, the batch size is 16, the training times are 200, the image size is 640, and the initial learning rate is set to 0.01.

[0170] Step three: input the low-illumination image to be tested into the trained low-illumination target detection model, and output the low-illumination target detection result. The low-light target detection result image is output, as shown in Figure 9 .

[0171] Step four: verify the effectiveness of the module.

[0172] In this experiment, mAP50 and mAP50:95 are used as performance reference indexes. mAP50 is the average of all pictures of each class when loU is set to 0.5, and then all classes are averaged, that is, mAP. mAP50:95 is the average mAP on different loU thresholds (from 0.5 to 0.95, step 0.05) (0.5, 0.55, 0.6, 0.65, 0.7, 0.75, 0.8, 0.85, 0.9, 0.95). The calculation formula of each index is as follows:

[0173]

[0174]

[0175]

[0176]

[0177] In the formula, TP is a true example, FP is a false positive, FN is a false negative, P is precision, R is recall, and AP is average precision.

[0178] From the experimental results in Table 1, the addition of each module helps improve the performance of the model. This proves that the proposed module is effective in improving the performance of the low-light target detection model. The overall mAP50 of the low-light target detection is improved by 2.4%.

[0179] Table 1 Comparison of module addition experiments

[0180]

[0181] Under the premise of keeping the experimental environment unchanged, the model of the application is compared with the commonly used target detection algorithms: YOLOv5n, Yolov8n, YOLOv10n, and RT-DETR. The performance comparison is shown in Table 2:

[0182] Table 2 Comparison of target detection algorithms

[0183]

[0184] The improved method of the model of the application has certain advantages. It is 2.4% higher than the original Yolov8n in mAP50, 4.8% higher than YOLOv10n in mAP50, higher than YOLOv5 in mAP50, and higher than RT-DETR in mAP50. Because the algorithm model of the application uses image adaptive processing to convert low-light images into a form that is easier to detect targets, and uses a feature extraction module with a multi-scale receptive field to fully extract the edge feature information and spatial information lacking in low-light images, and also uses a deformable large kernel separable attention module to reduce the defects of unclear target outlines caused by low light, making the model focus more on useful information, fully utilizing the surrounding context information to make up for the lack of local detail features, and enhancing the model's ability to detect low-light targets.

[0185] The application proposes a low-illumination target detection method with adaptive enhancement of multi-scale receptive field, which can achieve good detection effect in low-illumination images. The application is suitable for image target detection generated under various insufficient light conditions, such as night monitoring and unmanned driving.

[0186] The application adds a Globel module of image adaptive enhancement to the backbone network, uses a self-attention mechanism, generates parameters for adjusting image information by using global image information, and performs end-to-end training, so that the generated image is more conducive to target detection.

[0187] The application designs a feature extraction EIESA module capable of learning more rich feature representations, which combines a SobelConv branch for extracting edge information, a repVGG branch for extracting diverse spatial information and a Maxpool branch for extracting key features, obtains more diverse information, effectively utilizes limited information of low-light images and improves detection accuracy.

[0188] The application designs a variable large kernel separable attention structure LSKADCN module, uses LSKA to obtain a richer field of view at a smaller cost, has spatial and channel adaptability, effectively solves the problem of attention error distribution of the model and filters noise.

[0189] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present text can be realized in electronic hardware, computer software or a combination of both, and the components and steps of the examples have been described in general in the above description in order to clearly show the interchangeability of hardware and software. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present text.

[0190] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0191] In several embodiments provided in the present text, it should be understood that the disclosed system, device and method can be realized in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can also be electrical, mechanical or other forms of connection.

[0192] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0193] In addition, each functional unit in each embodiment herein can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0194] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions herein, essentially or the part of the prior art that contributes to the technical solutions, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment herein. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0195] The principles and implementation manners of the present application are described in the specific embodiments herein, and the above embodiment descriptions are only used to help understand the methods and core ideas thereof; meanwhile, for those skilled in the art, according to the ideas herein, the specific implementation manners and application ranges will have changes, and the above description of the present application should not be understood as a limitation.

Claims

1. A low-light target detection method with multi-scale receptive field under adaptive enhancement, characterized by: The following steps are involved: Step 1: Optimize the Yolov8 target detection algorithm and build a low-light target detection model based on image adaptive enhancement and multi-scale receptive field; including: A Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection. The RCSOBEL module is designed in Yolov8 to learn richer feature representations from features and enhance the extraction of edge and spatial information. The LSKADCN module with deformable large kernel and separable attention is designed in Yolov8 to transform the SPPF module in Yolov8 to obtain the LKSPPF module, which enables the network to focus on key feature information. Specifically, the RCSOBEL module is designed in Yolov8 to learn richer feature representations from features and enhance the extraction of edge information and spatial information; including: The input feature map passes through the RepConv module; passes through the EIESA module twice; splices the three output channels together; passes through a RepConv module to obtain the final feature map; The RepConv module is composed of three parallel direct outputs, 3×3 convolutions, and 1×1 convolutions during training. The resulting feature maps are then summed and activated using the SiLu activation function. During inference, the module is reparameterized into a 3×3 convolution plus a SiLu activation function, reducing parameters while ensuring accuracy. The EIESA module workflow includes: The input feature map passes through the convolution layer and the pooling layer, and then the feature map is flattened to (batch_size, feature_dim) size, where batch_size refers to the number of samples in each iteration, and feature_dim is the size obtained after flattening the feature map. It then passes through the fully connected layer and the softmax normalization weight to obtain four weight values; The input feature map is evenly divided into four feature maps along the channel direction, where each feature map passes through the SobelConv module, the RepConv module, the maximum pooling module and the direct output channel to obtain four feature maps; The obtained feature maps are multiplied by the weight values, and after residual connection, they are fused in the channel direction. Finally, the channels are shuffled by the channel shuffle module to obtain the final feature map. The LSKADCN module consists of LSKA and deformable convolution, and the workflow includes: The feature map undergoes a 3×3 deformable group convolution, which allows attention to adapt to data features and focus on key information hidden in the dark. It then passes through two cascaded 1D separable weighted convolution kernels composed of deep hole group convolution, where the convolution kernel sizes are (1, 5) and (5, 1), respectively. In addition, each channel in the feature map is convolved with the corresponding channel in the convolution kernel. After 1×1 convolution, cross-channel feature fusion is performed and the attention map is finally inferred; Multiply the output attention map and input features for adaptive feature refinement; The output of LSKADCN is expressed as: ； ； Where F is the input feature map, DCN is the deformable convolution, and DW is the 1D separable weight convolution kernel; Step 2: Obtain an image dataset under low illumination conditions and process it; use the processed dataset to train the low illumination target detection model to obtain a trained low illumination target detection model; Step 3: Input the low-light image to be tested into the trained low-light target detection model and output the low-light target detection result.

2. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 1, characterized in that: The low-light target detection model includes a backbone network, a neck network and a detection head; wherein The backbone network includes a Global module, a convolution module 1, a convolution module 2, a RCSOBEL module 1, a convolution module 3, a C2f module 1, a convolution module 4, a C2f module 2, a RCSOBEL module 2, and a SPPF_LSKADCN module; The neck network includes upsampling module 1, fusion module 1, C2f module 1, upsampling module 2, fusion module 2, C2f module 2, convolution module 1, fusion module 3, C2f module 3, convolution module 2, fusion module 4, and C2f module 4; The detection head includes three decoupling heads, and the C2f module 2, C2f module 3, and C2f module 4 of the neck network are input to the three decoupling heads respectively.

3. The low-light target detection method with multi-scale receptive field under adaptive enhancement according to claim 1, characterized in that: The Global module is introduced before Yolov8 to generate the color matrix and γ value of the global information of the predicted control image, so that the image is transformed into a form more suitable for target detection, including: After two 3×3 convolution expansion channels, high-dimensional features are encoded at low resolution to capture global information in the image; The cross-attention module obtains a 3×3 color matrix and γ value from the high-dimensional features; The color matrix and γ are subjected to two 1×1 convolutions respectively to further process and optimize the parameters; Perform matrix product of the original image and the color matrix, and perform γ power calculation on the result to obtain the final processed image, which is input into the subsequent network.

4. The method for low-light target detection with adaptively enhanced multi-scale receptive field according to claim 3, characterized in that: The cross-attention module obtains a 3×3 color matrix and γ value from the high-dimensional features, including: First, the input feature map undergoes a 3×3 depthwise convolution, and then passes through two fully connected layers to generate the value (V) and key (K) required by the self-attention mechanism. The query (Q) is randomly initialized, and the attention score matrix is ​​calculated using Q and K. The final output of the attention layer is calculated using V and the attention score matrix, which can be expressed as follows: ； Where Q: query matrix; K: key matrix; V: value matrix; d k : Dimension of the bond matrix; After passing through a multi-layer perceptron, it finally outputs ten parameters, representing a 3x3 color matrix and a 1-dimensional gamma (γ) value.

5. The method for low-light target detection with multi-scale receptive field under adaptive enhancement according to claim 3, characterized in that: The matrix product of the original image and the color matrix is ​​performed, and the result is subjected to γ ​​power calculation, including: color correction and gamma correction of the image, which can be expressed as follows: ； In the formula is a 3x3 color matrix, γ is the power value of gamma correction, The value of , to prevent the pixel value from being 0.

6. A low-light target detection device with multi-scale receptive field under adaptive enhancement, characterized by: The method comprises a processor and a memory; the memory is used to store a program; the processor executes the program to implement the method according to any one of claims 1 to 5.

Images