Pyramid edge enhancement-based self-correction low-illumination target detection method

Abstract

The invention discloses a pyramid edge enhancement-based self-correction low-illumination target detection method, which relates to the technical field of image processing, and designs a low-illumination image enhancement network GENet by using a Gaussian pyramid, Retinex and a dark channel defogging algorithm, and adds edge features into the network, so as to improve the detection accuracy of a low-illumination target. The GENet enhances the overall brightness contrast and the target contour feature of the low-illumination image, and improves the overall quality of the image. Besides, a lightweight self-correction detection method LSC-RTDETR is designed, and the accuracy of the whole low-illumination target detection is improved by correcting a feature map output by a feature extraction network.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a self-correcting low-light target detection method based on pyramid edge enhancement. Background Art

[0002] Object detection is a basic task in the field of computer vision and is widely used in many fields such as smart city construction, medical diagnosis, agricultural monitoring, video surveillance, and autonomous driving. Common object detection algorithms are divided into two types according to the implementation method: anchor box object detection methods are divided into second-order and first-order object detection algorithms according to the implementation method of the detection box. Second-order object detection algorithms include RCNN, Fast RCNN, and Faster RCNN; first-order object detection algorithms include SSD and YOLO. The anchor box-free object detection method removes the use of anchor boxes and realizes object detection through the combination and positioning of key points, which simplifies the detection process, reduces redundant anchor boxes, and has better generalization ability. The anchor box-free object detection methods include CenterNet, FCOS (fully convolutional one-stage object detection) and Transformer-based object detection algorithms. Among them, the anchor box-free object detection algorithm based on Transformer has the advantages of global information capture, anchor box-free selection, and parallel fast calculation, and has been widely used in the field of object detection.

[0003] However, although these algorithms can achieve satisfactory detection results in conventional environments with sufficient lighting and clear images, in complex low-light environments, such as weak light or even darkness, scenes with light sources and uneven lighting, shadows and color distortion, the collected images have difficulties such as insufficient lighting and uneven brightness, low contrast between targets and backgrounds, severe noise interference and color distortion. Directly applying general target detection algorithms to low-light target detection will result in low detection accuracy, inaccurate target positioning, and easy false detection and missed detection. Therefore, how to achieve efficient and accurate target detection in complex low-light environments has become an important issue that needs to be urgently solved in this field. Summary of the invention

[0004] The purpose of the present invention is to solve the above-mentioned problems and to provide a self-correcting low-light target detection method based on pyramid edge enhancement.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] According to one aspect of the present invention, a self-corrected low-light target detection method based on pyramid edge enhancement is provided, including two parts: low-light image enhancement network design and lightweight self-corrected target detection network improvement, characterized in that:

[0007] Low-light image enhancement network design part:

[0008] GL-RTDETR first inputs the low-light image into the GENet network to enhance the low-light image. GENet first decomposes the input image using a Gaussian pyramid to obtain four images of different sizes. Then, the dark channel algorithm DNet based on edge prior is used to enhance the texture and edge contour features of the low-light target and improve the contrast between the foreground target and the background. Then, the MSR algorithm is used to enhance the overall brightness contrast of the image to obtain a clearer image. Finally, the processed lower-level image is upsampled and fused to output the enhanced low-light image.

[0009] Improvements to lightweight self-correcting target detection network:

[0010] The enhanced low-light image is input into the LSC-RTDETR target detection network, and a lightweight self-correcting Backbone module is designed. While enhancing the feature extraction capability, the correlation between adjacent pixels is established, the feature map output by the self-correcting Backbone is highlighted, and the contour features of the target are further improved. The detection and positioning accuracy of low-light targets is further improved, the network parameter calculation amount is reduced, and the reasoning speed is accelerated.

[0011] Furthermore, the low-light image enhancement network design combines the advantages of Gaussian pyramid, dark channel dehazing algorithm and Retinex algorithm, and combines the characteristics of low-light images to design a network GENet for low-light image enhancement. GENet first performs a Perform Gaussian pyramid downsampling;

[0012] (1)

[0013] in are images of different scales output by the Gaussian pyramid. In order to perform a downsampling operation on the input image based on the Gaussian pyramid, in this application, Set it to 4, that is, downsampling is performed four times, and four images of different scales are output. Represents Gaussian filtering. After each Gaussian pyramid operation, the width and height of the image are halved, and the resolution is 0.25 times that of the input image. According to the size of the input image, a total of four layers of Gaussian pyramid images are generated;

[0014] In order to highlight the texture, contour and edge detail features of the target in the low-light image, a dark channel enhancement algorithm based on edge features is used to enhance the contour features of the target and calculate the output image of the four-layer Gaussian pyramid. The dark channel value

[0015] (2)

[0016] in is the input image In Location Color channel on The strength of So The convolution window centered on is the dark channel value, which is the intensity of the minimum RGB channel in a certain area of ​​the input image. To calculate the minimum pixel value of the three RGB channels, this minimum value represents the lowest level of "darkness" or "brightness" of the low-light image pixel in the three color channels. The low-light intensity in each area is calculated as:

[0017] (3)

[0018] in, is the low light intensity in the selected area, The maximum value of the pixel in the selected area indicates the pixel value with the highest brightness in the selected area, and the transmission map of the low-light image is roughly estimated:

[0019] (4)

[0020] in, It is a rough estimated projection map of the low-light image;

[0021] In order to highlight the edge details and contour features of low-light targets, a dark channel dehazing algorithm based on edge feature prior is used to adjust and correct the rough estimation result of the transmission map. The adjusted transmission map is estimated as follows:

[0022] (5)

[0023] in is a preset increment used to enhance the transmittance at the edge. For the input image The edge strength is 1 if it is the target edge, otherwise it is 0. Calculated as:

[0024] (6)

[0025] Among them, the edge strength is obtained by the Canny operator. To ensure that the transmittance is in the range of [0,1], the transmission map of formula (5) is adjusted and corrected as follows:

[0026] (7)

[0027] in is the minimum value of the projection map. By correcting the projection map, the projection map is mapped between [0,1], and the correction and dynamic adjustment of the projection rate are realized. The image enhanced by the dark channel enhancement algorithm based on edge features is:

[0028] (8)

[0029] The dark channel enhancement algorithm based on edge features is used to enhance the contour and edge detail features of low-light targets. Then, MSR is used to enhance the overall brightness and contrast of low-light images. The single-scale Retinex (SSR) is calculated as:

[0030] (9)

[0031] in So is the Gaussian function of the standard deviation, is the output image after single SSR enhancement, is the convolution operation, is the input image after the dark channel enhancement algorithm based on edge features. The MSR output is a linear combination of the SSR results calculated at multiple scales:

[0032] (10)

[0033] in is the output image after MSR enhancement, is the first in the selected scale set This application selects RGB as three color scales, namely .

[0034] Furthermore, in the improvement of the lightweight self-calibrated target detection network, Backbone uses ResNet50 as the backbone feature extraction network of RTDETR to improve the contextual connection between targets, output more accurate feature maps, and improve the performance of the target detection network. Inspired by meta-mobile block and self-calibrated convolutions, the meta-mobile block module is embedded in the P3, P4, and P5 output layers of ResNet50, and the last layer of 1×1 convolution in the meta-mobileblock structure is replaced with self-calibrated convolutions to design a lightweight ResNet50 (SCC-ResNet50). The meta-mobile block is a network module designed for lightweight and efficient feature extraction. It cleverly combines the residual learning concept of the residual network (ResNet) with the optimization strategy of the mobile network. Based on residual connections, this module only contains convolution operations and an efficient multi-head self-attention mechanism (MHSA), abandoning other complex structures to ensure the simplicity of calculation. Inside the module, the input features are first processed by 1×1 convolution, and then the multi-head self-attention mechanism with efficient extended window is used to calculate the position information of the key, query and value, and the features generated in these two steps are weighted and fused to enhance the expressiveness of the features. Next, deep separable convolution (DW-Conv) is used to achieve downsampling through stride adaptation, which not only effectively reduces the computational complexity, but also takes into account the advantages of dynamic global feature fusion, significantly improving the feature extraction of low-light targets. The meta-mobile block does not need to introduce additional position embedding to provide inductive bias for MHSA in design, which further simplifies the model structure. Compared with conventional convolution, DW-Conv has only 36% of the parameters of conventional convolution when the input and output feature maps are the same. This feature enables the meta-mobile block to achieve efficient feature extraction while also realizing a lightweight design of the network structure.

[0035] Furthermore, the last layer of 1×1 convolution in the meta-mobile block structure is replaced with a self-correcting convolution. The self-correcting convolution decomposes the feature map and expands the receptive field of each convolution layer internally. The self-correcting convolution uses a new self-correcting operation to adaptively build long-range spatial and channel dependencies around each spatial position, thereby enhancing contextual information and enriching output features. The self-correcting convolution is simple and universal, without introducing additional parameters and computational complexity.

[0036] The self-rectified convolution splits the input feature map into two scale spaces, namely the original scale feature space and self-correcting scale space In the self-correcting scale space In the example, we first use the convolution kernel as Perform average pooling on the input features to get the input Weights at different scales, added to the input. The features extracted by convolution are corrected by the Sigmoid activation function to generate a corrected weight map. Finally, this weight map is used to correct the feature map in the original scale feature space to achieve the self-correction function. Convolution resizes the feature map and extracts the features extracted by the rectified convolution At the same time, in the original scale feature space, use Convolution on feature map Perform convolution operation to extract original scale features Finally, the features of the self-corrected scale space output And the features output in the original scale feature space Perform a splicing operation to obtain the final output feature containing information of two scale spaces The final output feature It contains both the spatial detail information of the original scale features and the contextual information of the self-corrected scale features, thereby improving the model's ability to extract and understand image semantic information.

[0037] The whole process of self-corrected convolution is calculated as

[0038] (11)

[0039] (12)

[0040] (13)

[0041] (14)

[0042] in, Upsample for nearest neighbor interpolation to restore the image size dimension, For average pooling, the pixels and target spatial positions in each feature map are constructed to construct long-range spatial and channel dependencies, thus obtaining richer and more accurate feature maps. is the convolution operation, is the convolution kernel, For Hadamard, is the activation function Sigmoid.

[0043] The meta-mobile block module is embedded into the P3, P4 and P5 output layers of ResNet50, and the last layer of 1×1 convolution in the meta-mobile block structure is replaced with self-correcting convolution. The designed SCC-ResNet50 can reduce redundant calculations while making full use of the characteristics of self-correcting convolution, deeply explore the dependencies between pixels and target spatial positions in each feature map, obtain richer and more accurate feature maps, and improve classification accuracy.

[0044] Corrected Conv takes the input features Calculated as:

[0045] (15)

[0046] (16)

[0047] (17)

[0048] in It is the Concate feature concatenation operation. is the input feature, and To input features The two feature spaces are split. and They are the features extracted by self-correction convolution and original scale convolution respectively. is the output feature of the SCC-ResNet50 network.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] 1. Low-light image enhancement network design: In order to improve the overall brightness of low-light images and the contrast between the target outline and the background, and to improve the recognition effect of the target detection algorithm in low-light scenes, this application innovatively designs a low-light image enhancement network called GENet by analyzing the multi-scale processing capabilities of the Gaussian pyramid, the advantages of the dark channel dehazing algorithm in ambient light adjustment, and the characteristics of the Retinex algorithm in color restoration.

[0051] 2. Improvement of lightweight self-calibrated target detection network: In order to improve the contextual connection between targets, output more accurate feature maps, and improve the performance of the target detection network, this application chooses ResNet50 as the backbone feature extraction network of RTDETR. Inspired by meta-mobile block and self-calibrated convolutions, the meta-mobile block module is embedded in the P3, P4, and P5 output layers of ResNet50, and the last layer of 1×1 convolution in the meta-mobile block structure is replaced with self-calibrated convolution. A lightweight ResNet50 (SCC-ResNet50) is designed, and the LSC-RTDETR low-light target detection algorithm is proposed. While achieving lightweight, the dependencies between each feature map, the long-range space of the target spatial position, and the channels are adaptively established to obtain richer and more accurate feature maps, and then encoding and decoding are performed according to the feature maps, which improves the performance of target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is the overall framework diagram;

[0053] Figure 2 This is the network structure diagram of the meta-mobile block;

[0054] Figure 3 This is the network structure diagram of Self-calibrated convolutions;

[0055] Figure 4 This is the SCC-ResNet50 network structure diagram;

[0056] Figure 5 is the RGB channel intensity and feature map;

[0057] Figure 6 This is the result of GENet enhancing low-light data;

[0058] Figure 7 This is a thermal comparison diagram of the ablation experiment;

[0059] Figure 8 This is a comparison chart of the visualization results of different algorithms. DETAILED DESCRIPTION

[0060] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0061] The present invention provides a self-corrected low-light target detection method based on pyramid edge enhancement. At present, the performance of the target detection algorithm in low-light scenes is improved mainly by designing a low-light image enhancement model, optimizing the model structure, and preprocessing enhancement. The purpose of low-light image enhancement is to improve the visual perception of low-light images by restoring image details and correcting color distortion, and to provide high-quality images for other visual tasks such as object detection. The low-light image enhancement model is divided into three methods in terms of implementation: traditional low-light image enhancement network, CNN network (convolutional neural network) and GAN (generative adversarial networks).

[0062] Based on the anchor-free target detection method RTDETR (real time detection transformer), this application designs a low-light target detection network GL-RTDETR (low-light object detection networkbased on GENet and LSC-RTDETR), and proposes a low-light image enhancement network GENet (gaussian pyramid-based edge feature low-light image enhancement network) based on Gaussian pyramid edge features. First, the input image is downsampled using the image Gaussian pyramid. The sampled image uses multi-scale Retinex and edge feature-based dark channel dehazing algorithms to enhance the overall brightness and target contour edge features of the low-light image, respectively. Finally, after upsampling and fusion, the low-light enhanced image is obtained. Secondly, a lightweight self-correcting object detection network LSC-RTDETR (a lightweight self-correcting object detection networkbased on RTDETR) is proposed. By designing a self-correcting feature extraction network, the features output by the feature extraction network are corrected, the accuracy of network feature extraction is improved, and better detection results are obtained.

[0063] The overall network structure of the GL-RTDETR method proposed in this application is as follows Figure 1 shown.

[0064] GL-RTDETR inputs low-light images into the GENet network for image enhancement. GENet first decomposes the input image using a Gaussian pyramid to obtain four images of different scales, and then passes through the edge prior-based dark channel algorithm DNet (Dark Channel Prior Network) to enhance the texture and edge contour features of low-light targets and improve the contrast between foreground targets and backgrounds. Then, the MSR (Multi-Scale Retinex Network) algorithm is used to enhance the overall brightness contrast of the image to obtain a clearer image. Finally, the processed lower-level images are upsampled and fused to output the enhanced low-light image.

[0065] Then, the enhanced low-light image is input into the LSC-RTDETR target detection network, and a lightweight self-correcting Backbone module (SCC-RseNest50) is designed. While enhancing the feature extraction capability, the correlation between adjacent pixels is established, the feature map output by the self-correcting Backbone is highlighted, and the contour features of the target are highlighted, which further improves the detection and positioning accuracy of low-light targets, reduces the network parameter calculation amount, and speeds up the reasoning speed.

[0066] Low-light image enhancement network design

[0067] like Figure 1 As shown in the figure, in order to improve the overall brightness of low-light images and the contrast between the target outline and the background, and to improve the recognition effect of the target detection algorithm in low-light scenes, this application fully combines the advantages of Gaussian pyramid, dark channel defogging algorithm and Retinex algorithm, and combines the characteristics of low-light images to design a network GENet for low-light image enhancement. GENet first performs a Perform Gaussian pyramid downsampling.

[0068] (1)

[0069] in are images of different scales output by the Gaussian pyramid. In order to perform a downsampling operation on the input image based on the Gaussian pyramid, in this application, Set it to 4, that is, downsampling is performed four times, and four images of different scales are output. Represents Gaussian filtering. After each Gaussian pyramid operation, the width and height of the image are halved, and the resolution is 0.25 times that of the input image. This application generates a total of four layers of Gaussian pyramid images according to the size of the input image.

[0070] In order to highlight the texture, contour and edge detail features of the target in low-light images, this application designs a dark channel enhancement algorithm based on edge features to enhance the contour features of the target and calculates the output image of the four-layer Gaussian pyramid. The dark channel value

[0071] (2)

[0072] in is the input image In Location Color channel on The strength of So The convolution window centered on is the dark channel value, which is the intensity of the minimum RGB channel in a certain area of ​​the input image. To calculate the minimum pixel value of the three RGB channels, this minimum value represents the lowest level of "darkness" or "brightness" of the low-light image pixel in the three color channels. The low-light intensity in each area is calculated as

[0073] (3)

[0074] in, is the low light intensity in the selected area, It is the maximum value of the pixel in the selected area, indicating the pixel value with the highest brightness in the selected area. A rough estimate of the transmission map of the low-light image

[0075] (4)

[0076] in, A rough estimated projection map for low-light images.

[0077] In order to highlight the edge details and contour features of low-light targets, this application proposes a dark channel defogging algorithm based on edge feature priors. Specifically, the rough estimation result of the transmission map is adjusted and corrected. The adjusted transmission map is estimated to be

[0078] (5)

[0079] in is a preset increment used to enhance the transmittance at the edge. For the input image The edge strength of the target edge is 1, otherwise it is 0. Calculated as

[0080] (6)

[0081] The edge strength is obtained by the Canny operator. To ensure that the transmittance is in the range of [0,1], the transmission map of equation (5) is adjusted and corrected as follows:

[0082] (7)

[0083] in is the minimum value of the projection map. By correcting the projection map, the projection map is mapped between [0,1], realizing the correction and dynamic adjustment of the projection rate. The image enhanced by the dark channel enhancement algorithm based on edge features is

[0084] (8)

[0085] The dark channel enhancement algorithm using edge features enhances the contour and edge detail features of low-light targets. Then MSR is used to enhance the overall brightness and contrast of low-light images. Single-scale Retinex (SSR) is calculated as:

[0086] (9)

[0087] in So is the Gaussian function of the standard deviation, is the output image after single SSR enhancement, is the convolution operation, is the input image after the edge feature-based dark channel enhancement algorithm. The MSR output is a linear combination of the SSR results calculated at multiple scales.

[0088] (10)

[0089] in is the output image after MSR enhancement, is the first in the selected scale set This application selects RGB as three color scales, namely .

[0090] Improvement of lightweight self-correcting target detection network

[0091] As a representative algorithm for anchor-free object detection, RTDETR has achieved further improvements in timeliness and accuracy. RTDETR introduces an efficient hybrid encoder, decouples the interactions and cross-scale features of each scale, fuses and finely processes multi-scale features, and improves the recognition ability of objects of different sizes. In addition, RTDETR abandons the non-maximum suppression (NMS) post-processing step and uses the IOU (intersection over Union) perception query selection matrix instead, directly considering the overlap between detection boxes in the decoding stage, avoiding the timeliness and accuracy loss caused by NMS.

[0092] This application chooses ResNet50 as the backbone feature extraction network (Backbone) of RTDETR to improve the contextual connection between targets, output more accurate feature maps, and improve the performance of the target detection network. Inspired by meta-mobile block and self-calibrated convolutions, the meta-mobile block module is embedded in the P3, P4, and P5 output layers of ResNet50, and the last layer of 1×1 convolution in the meta-mobile block structure is replaced with self-calibrated convolutions. A lightweight ResNet50 (SCC-ResNet50) is designed. The LSC-RTDETR low-light target detection algorithm is proposed. While achieving lightweight, the dependencies between each feature map, the long-range space of the target spatial position, and the channels are adaptively established to obtain richer and more accurate feature maps, and then encoding and decoding are performed according to the feature maps, which improves the performance of target detection.

[0093] like Figure 2As shown in the figure, meta-mobile block is a network module designed for lightweight and efficient feature extraction. It cleverly combines the residual learning concept of residual network (ResNet) with the optimization strategy of mobile network. Based on residual connection, this module only contains convolution operation and efficient multi-head self-attention mechanism (MHSA), abandons other complex structures and ensures the simplicity of calculation. Inside the module, the input features are first processed by 1×1 convolution, and then the position information of key, query and value is calculated by multi-head self-attention mechanism with efficient extended window, and the features generated by these two steps are weighted fused to enhance the expressiveness of the features. Then, deep separable convolution (DW-Conv) is used to achieve downsampling through stride adaptation, which not only effectively reduces the computational complexity, but also takes into account the advantages of dynamic global feature fusion, significantly improving the feature extraction of low-light targets. The meta-mobile block does not need to introduce additional position embedding to provide inductive bias for MHSA in design, which further simplifies the model structure. Compared with conventional convolution, DW-Conv has only 36% of the parameters of conventional convolution when the input and output feature maps are the same. This feature enables meta-mobile block to achieve efficient feature extraction while also realizing lightweight design of network structure.

[0094] The last layer of 1×1 convolution in the meta-mobile block structure is replaced with a self-correcting convolution. The self-correcting convolution decomposes the feature map and expands the receptive field of each convolution layer internally. The self-correcting convolution uses a new self-correcting operation to adaptively build long-range spatial and channel dependencies around each spatial position, thereby enhancing contextual information and enriching output features. The self-correcting convolution is simple and universal, without introducing additional parameters and computational complexity.

[0095] like Figure 3 , the self-rectified convolution splits the input feature map into two scale spaces, namely the original scale feature space and self-correcting scale space In the self-correcting scale space In the example, we first use the convolution kernel as Perform average pooling on the input features to get the input Weights at different scales, added to the input. The features extracted by convolution are corrected by the Sigmoid activation function to generate a corrected weight map. Finally, this weight map is used to correct the feature map in the original scale feature space to achieve the self-correction function. Convolution resizes the feature map and extracts the features extracted by the rectified convolution At the same time, in the original scale feature space, use Convolution on feature map Perform convolution operation to extract original scale features Finally, the features of the self-corrected scale space output And the features output in the original scale feature space Perform a splicing operation to obtain the final output feature containing information of two scale spaces The final output feature It contains both the spatial detail information of the original scale features and the contextual information of the self-corrected scale features, thereby improving the model's ability to extract and understand image semantic information.

[0096] The whole process of self-corrected convolution is calculated as

[0097] (11)

[0098] (12)

[0099] (13)

[0100] (14)

[0101] in, Upsample for nearest neighbor interpolation to restore the image size dimension, For average pooling, the pixels and target spatial positions in each feature map are constructed to construct long-range spatial and channel dependencies, thus obtaining richer and more accurate feature maps. is the convolution operation, is the convolution kernel, For Hadamard, is the activation function Sigmoid.

[0102] like Figure 4 , the meta-mobile block module is embedded into the P3, P4 and P5 output layers of ResNet50, and the last layer of 1×1 convolution in the meta-mobile block structure is replaced by self-correcting convolution. The designed SCC-ResNet50, while reducing redundant calculations, makes full use of the characteristics of self-correcting convolution, deeply mines the dependencies between pixels and target spatial positions in each feature map, obtains richer and more accurate feature maps, and improves classification accuracy.

[0103] Corrected Conv takes the input features Calculated as:

[0104] (15)

[0105] (16)

[0106] (17)

[0107] in It is the Concate feature concatenation operation. is the input feature, and To input features The two feature spaces are split. and They are the features extracted by self-correction convolution and original scale convolution respectively. is the output feature of the SCC-ResNet50 network.

[0108] Experiment and result analysis

[0109] Dataset Introduction

[0110] This application chooses to conduct experiments on the low-light target detection public dataset ExDark to verify the GL-REDETR method. The ExDark dataset is a low-light target detection dataset used for target detection and image enhancement. The data collection environment is low light and dusk. A total of 7363 images under 10 different lighting conditions were collected, and 12 categories were annotated, including bicycles (Bicycle), boats (Boat), bottles (Bottle), buses (Bus), cars (Car), cats (Cat), chairs (Chair), cups (Cup), dogs (Dog), motorcycles (Motorbike), people (People) and tables (Table). The dataset is divided into training set, validation set and test set in a ratio of 8:1:1, with 5884, 741 and 738 images respectively.

[0111] Experimental environment and evaluation indicators

[0112] The experiment uses the deep learning framework Pytorch 1.12.1, Python 3.7.6, CUDA 10.0, the hardware is Intel (R) Xeon (R) CPU E5-2678 v3, and the GPU is NVIDIA GeForce RTX 3060 for single-card training. In the training stage, the pre-trained weights are used, and the Adam optimizer is selected to optimize the model. The batch size is set to 2, the epoch is 200, the initial learning rate is set to 0.01, and the weight decay coefficient is 0.000005. In terms of detection accuracy, AP (accuracy of each category), mAP (average accuracy of each category), mmAP (detection accuracy under different IOUs, IoU=0.5:0.05:0.95), and recall rate R (Recall) are selected. Among the above four evaluation indicators, the larger the value, the higher the detection accuracy and stability of the algorithm model. In addition to detection accuracy, model size and real-time detection performance are also important for target detection. Therefore, the parameter amount Para (Parameter), model size MS (model size) and detection speed FPS (frames per second) are selected to measure. Parameter and Model size usually refer to the structure and parameter size of the model. The smaller the model, the smaller the number of parameters and the amount of calculation. The calculation can be completed in a shorter time, thereby speeding up the operation of the model. FPS is usually used to measure the efficiency of the target detection algorithm, reflecting the detection speed of the algorithm on a given hardware platform. The condition for real-time detection is that FPS is greater than or equal to 30. The higher the FPS value, the better the real-time performance of the algorithm.

[0113] Experimental analysis

[0114] In terms of detection accuracy, AP (accuracy of each category), mAP (average accuracy of each category), mmAP (detection accuracy under different IOU, IoU=0.5:0.05:0.95), and recall rate R (Recall) are selected. Among the above four evaluation indicators, the larger the value, the higher the detection accuracy and stability of the algorithm model. In addition to detection accuracy, model size and real-time detection performance are also important for target detection. Therefore, the parameter amount Para (Parameter), model size MS (model size) and detection speed FPS (frames per second) are selected to measure. Parameter and Model size usually refer to the structure and parameter size of the model. The smaller the model, the smaller the number of parameters and the amount of calculation. The calculation can be completed in a shorter time, thereby speeding up the running speed of the model. FPS is usually used to measure the efficiency of the target detection algorithm and reflects the detection speed of the algorithm on a given hardware platform. The condition for real-time detection is that FPS is greater than or equal to 30. The higher the FPS value, the better the real-time performance of the algorithm.

[0115] Experimental analysis of GENet low-light image enhancement

[0116] In order to verify the enhancement effect of GENet on low-light images and its effect on the target detection network RTDETR, select an image, and its RGB channel intensity and feature map are as follows Figure 5 shown.

[0117] like Figure 5 As shown in the figure, (a) is the original image and the enhanced image through GENet, (b) is the line graph of RGB intensity distribution and the histogram of channel average pixel value. RGB intensity distribution reflects the brightness and color distribution of the image. From the line graph of RGB intensity distribution, it can be seen that the RGB distribution of the original image is very uneven, where the peak value of the B channel intensity exceeds 10,000 within 0-50 pixels, indicating that the brightness and RGB color distribution of the original low-light image are uneven. After enhancement, the image has been greatly improved in brightness and RGB color, narrowing the channel intensity gap of the three RGB channels and increasing the overall brightness of the low-light image. The channel average pixel value of the image reflects the overall effect of the image and the color balance of the image. From the histogram of the channel average pixel value, it can be seen that the channel average pixel value of the original low-light image is low, and the average pixel gap of the three RGB channels is large, indicating that the overall color of the image is low and the color is uneven. After GENet enhancement, the overall color distribution of the image is more uniform, and the color of the image is further balanced, enhancing the brightness and contrast of the low-light image. In summary, through the analysis and calculation of objective and subjective evaluation indicators, the effectiveness of the GENet proposed in this application for low-light image enhancement is verified.

[0118] Figure 5 (c) is the feature map generated after the original image and the enhanced image are extracted by the target detection network convolution feature extraction. It can be seen that the edge contour extraction of the image enhanced by GENet is more complete, and the extracted features are more complete and comprehensive, which improves the accuracy of subsequent target detection.

[0119] Figure 6 The enhancement results of GENet on the low-light dataset show that GENet improves the overall brightness contrast of the image, making the target details and textures that were originally difficult to identify under low-light conditions clearer. At the same time, GENet also highlights the overall contour information of the target, which helps to achieve higher-precision low-light target detection and positioning.

[0120] Ablation experiment

[0121] In order to verify the optimization effect of the improved algorithm on low-light target detection, the ablation experiment was compared and analyzed with RTDETR as the benchmark, and the experimental results are shown in Table 1. Table 1 shows the AP indicators of each category of different improvement strategies, with RTDETR as the benchmark, the GENet method is denoted as M1, and the LSC-RTDETR method is denoted as M2. Each group of ablation experiments is carried out under the same experimental environment and experimental parameters.

[0122] 1. Ablation experiment comparison of AP indicators of various categories

[0123]

[0124] As can be seen from Table 1, compared with the benchmark RTDETR, the mAP of GL-RTDETR has increased by 2.49%, and it also has certain advantages in the detection accuracy of 12 categories. The detection accuracy AP of Bicycle, Boat, Bus, Car, Cat, Motorbike and Table7 targets is the highest. The preprocessing step GENet and the lightweight correction LSC-RTDETR method have a positive effect on low-light detection tasks. Although the accuracy is not the highest in Bottle, Chair, Cup, Dog and People5 categories, it is also comparable. This is because the improved optimization strategies interact with each other, and the interaction between different improvement strategies may be positive or negative, resulting in some non-optimal accuracy. In addition, the SCC-ResNet50 designed in this application increases the number of convolutional layers, resulting in the loss of some features of some small targets during the convolution process, resulting in non-optimal accuracy, but overall, the method proposed in this application has certain advantages in the detection of other categories of targets. Compared with the baseline RTDETR, the mAP of GENet increased by 2.14%, indicating that by enhancing the texture, contour and overall brightness contrast of low-light images, the overall contrast of the target is highlighted, allowing the detection network to extract more features, improving the accuracy of feature extraction, and thus improving detection accuracy. The mAP of LSC-RTDETR increased by 0.64%, indicating that the designed lightweight correction feature extraction network SCC-ResNet50 not only reduces parameters, but also corrects the accuracy of the feature map output by Backbone through the correction network, and establishes channel and spatial connections, ensuring detection accuracy and improving detection effects.

[0125] The ablation experiments of its low-light enhancement network GENet and lightweight correction LSC-RTDETR method on other evaluation indicators are shown in Table 2.

[0126] Table 2 Ablation experiment comparison of different improvement strategies

[0127]

[0128] The ablation experiment in Table 2 shows that the two optimization and improvement strategies of low-light enhancement network GENet and LSC-RTDETR have a positive effect on the overall evaluation indicators. The GL-RTDETR method of this application is the best in mAP, mmAP, R, Para and MS indicators. Since the self-correction network adds convolutional layers, the number of operations increases, resulting in a decrease in the detection speed FPS, but it can still meet the requirements of real-time detection (FPS>30). In addition, low-light image enhancement GENet is a preprocessing of the data set, which has no effect on the detection algorithm parameter Para, model size MS and detection speed FPS.

[0129] In order to intuitively show the attention of each optimization improvement method to the target in the image, the heat map is compared. Figure 7 7(b) and 7(c) show that the low-light enhancement network GENet eliminates some background interference and enhances the contrast between the target and the background; 7(b), 7(d) and 7(e) show that the lightweight correction model outputs a more accurate feature map, which improves the model's attention to the target and makes the algorithm more accurate in locating the target in the image.

[0130] 4.3.2 Comparative Experiments on ExDark Dataset

[0131] In order to verify the effectiveness of the GL-RTDETR method, in the same experimental environment, objective evaluation and visualization results were compared and analyzed with several mainstream general target detection methods (YOLOv5l, YOLOv7x, YOLOv8l and RTDETR) and classic low-light target detection methods ZERO-DCE (ZERO-DCE+YOLOv3) and IA-YOLO. The results are shown in Tables 3 and 4. Figure 8 shown.

[0132] As can be seen from Table 3, after the proposed GENet enhances the low-light image, it improves the overall contrast of the low-light image, enhances the edge contour of the target, and improves the detection performance of each target detection algorithm to varying degrees. Among the six evaluation indicators, the method GL-RTDETR proposed in this application has the best mAP, R, Para and MS indicators. YOLOv5l has the highest mmAP indicator of 44.14%, and RTDETR has the highest FPS of 86.96. This is because RTDETR uses an anchor-free detection method, has no post-processing steps, and has a faster detection speed. Compared with the baseline RTDETR, GL-RTDETR's mAP increased by 2.49%, mmAP increased by 2.34%, recall rate R increased by 2.09%, the number of parameters decreased by 4.95M, and the model size MS decreased by 13.31MB. Although the addition of convolution causes the detection speed FPS to be 14.73 slower, it can still meet the requirements of real-time detection, and has certain superiority and comparability in the comparison algorithm.

[0133] Table 3 Experimental comparison results of different methods

[0134]

[0135] The visual comparison results are as follows Figure 8 As shown, select ExDark data

[0136] Three test images in the set are compared.

[0137] Figure 8 In the first column, all seven algorithms detected the target, but the method of this application has the highest detection accuracy. In the second column, methods 8(a), 8(c), 8(d) and 8(f) all have missed detections, and 8(b) has a false detection. This is because the overall brightness of the low-light image is low, which leads to certain shortcomings of the above methods in the low-light target detection task. However, the methods of this application GL-RTDETR and ZERO-DCE both successfully detected pedestrians and bicycles, and the detection accuracy of the method of this application is the highest. See the local enlarged image. Figure 8The third column. In the fourth column, 8(a) had missed detection and wrong detection, 8(b), 8(d), 8(e) and 8(f) had wrong detection, 8(c) and 8(g) successfully detected the target without wrong detection and missed detection. By comparing the target detection frames of 8(c) and 8(e), it was found that in the detection results of the chair category, the detection frame of YOLOv8l was incomplete and redundant, while the method of this application showed good positioning results and the detection frame was complete, proving that the method GL-RTDETR proposed in this application has a higher precision positioning frame. In summary, after objective evaluation and comparison of visualization results, it was found that the mainstream general target detection methods need to be strengthened in low-light detection tasks. The low-light target detection methods ZERO-DCE and IA-YOLO showed color distortion and overexposure in the low-light enhanced images, resulting in the loss of some contours and detail information during the feature extraction process, affecting the detection accuracy. The proposed method GL-RTDETR can better complete real-time and accurate low-light target detection, overcome the false detection and missed detection caused by low brightness contrast, and improve the detection effect and positioning accuracy of the model.

[0138] in conclusion

[0139] In order to better achieve the low-light target detection task, this application proposes a low-light target detection method GL-RTDETR based on Gaussian pyramid edge enhancement. The low-light image enhancement network GENet is designed using Gaussian pyramid, Retinex and dark channel dehazing algorithm, and edge features are added to the network, so that GENet enhances the overall brightness contrast and target contour features of low-light images, and improves the overall quality of the image. In addition, a lightweight self-correction detection method LSC-RTDETR is designed, and the accuracy of the entire low-light target detection is improved by correcting the feature map output by the feature extraction network. The ablation experiment shows the effectiveness and rationality of each optimization and improvement strategy of the GL-RTDETR method. Compared with the baseline RTDETR, the mAP of GL-RTDETR is increased by 2.49%, the mmAP is increased by 2.34%, the recall rate is increased by 2.09%, the number of parameters is reduced by 4.95M, and the model size is reduced by 13.31MB. Comparative experiments show that the GL-RTDETR method has high detection accuracy and detection effect in low-light scenes. However, during the experiment, it was found that the detection effect of the method of the present application on dense targets needs to be improved. The next research work will focus on the design of the feature fusion network, explore improvement ideas, further improve the detection effect of low-light dense targets and integrate the method into the camera. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claim involved.

[0140] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A self-corrected low-light target detection method based on pyramid edge enhancement, including the design of a low-light image enhancement network and an improvement of a lightweight self-corrected target detection network. Its characteristics are: Low-light image enhancement network design part: GL-RTDETR first inputs the low-light image into the GENet network to enhance the low-light image. GENet first decomposes the input image using a Gaussian pyramid to obtain four images of different sizes. These images are then processed using the edge prior-based dark channel algorithm DNet to enhance the texture and edge contour features of the low-light target and improve the contrast between the foreground target and the background. The MSR algorithm is then used to enhance the overall brightness contrast of the image to obtain a clearer image. Finally, the processed lower-layer images are upsampled and fused to output the enhanced low-light image. Improvements to lightweight self-correcting target detection networks: The enhanced low-light image is input into the LSC-RTDETR target detection network, and a lightweight self-correcting Backbone module is designed. While enhancing the feature extraction capability, it establishes the correlation between adjacent pixels, self-corrects the feature map output by the Backbone, highlights the contour features of the target, further improves the detection and positioning accuracy of low-light targets, reduces the network parameter calculation amount, and speeds up the inference speed.

2. The method for self-correcting low-light target detection based on pyramid edge enhancement according to claim 1, wherein: The low-light image enhancement network design combines the advantages of Gaussian pyramid, dark channel defogging algorithm and Retinex algorithm, and combines the characteristics of low-light images to design a network GENet for low-light image enhancement. GENet first performs a Perform Gaussian pyramid downsampling; (1) in are images of different scales output by the Gaussian pyramid, In order to perform downsampling operation on the input image based on Gaussian pyramid, in this application, Set it to 4, that is, perform four downsamplings and output four images of different scales. Represents Gaussian filtering. After each Gaussian pyramid operation, the width and height of the image are halved, and the resolution is 0.25 times that of the input image. Based on the size of the input image, a total of four layers of Gaussian pyramid images are generated; In order to highlight the texture, contour and edge detail features of the target in low-light images, a dark channel enhancement algorithm based on edge features is used to enhance the contour features of the target and calculate the output image of the four-layer Gaussian pyramid. Dark channel value (2) in is the input image In position Color channel on The strength of Therefore The convolution window centered on , the dark channel value is the intensity of the minimum RGB channel in a certain area of ​​the input image, To calculate the minimum pixel value of the three RGB channels, this minimum value represents the lowest level of "darkness" or "brightness" of the low-light image pixel in the three color channels. The low-light intensity in each area is calculated as: (3) in, is the low light intensity in the selected area, is the maximum value of the pixels in the selected area, indicating the pixel value with the highest brightness in the selected area, and a rough estimate of the transmission map for low-light images: (4) in, is a rough estimated projection map of the low-light image; In order to highlight the edge details and contour features of low-light targets, a dark channel dehazing algorithm based on edge feature prior is used to adjust and correct the rough estimation result of the transmission map. The adjusted transmission map is estimated as follows: (5) in is a preset increment used to enhance the transmittance at the edge. For the input image If it is the target edge, the value is 1, otherwise it is 0. Calculated as: (6) Among them, the edge strength is obtained by the Canny operator. To ensure that the transmittance is in the range of [0,1], the transmission map of formula (5) is adjusted and corrected as follows: (7) in is the minimum value of the projection map. By correcting the projection map, the projection map is mapped between [0, 1], and the correction and dynamic adjustment of the projection rate are realized. The image enhanced by the dark channel enhancement algorithm based on edge features is: (8) The dark channel enhancement algorithm based on edge features is used to enhance the contour and edge detail features of low-light targets. Then, MSR is used to enhance the overall brightness and contrast of low-light images. The single-scale Retinex (SSR) is calculated as: (9) in Therefore is the Gaussian function of the standard deviation, is the output image after single SSR enhancement, is the convolution operation, For the input image after the edge feature-based dark channel enhancement algorithm, the MSR output is a linear combination of the SSR results calculated at multiple scales: (10) in is the output image after MSR enhancement, is the first in the selected scale set scales, this application selects RGB as three color scales, namely .

3. The self-correcting low-light target detection method based on pyramid edge enhancement according to claim 1, wherein: To improve the lightweight self-correcting object detection network, Backbone uses ResNet50 as the backbone feature extraction network for RTDETR, embeds the meta-mobile block module into the P3, P4, and P5 output layers of ResNet50, and replaces the last 1×1 convolution layer in the meta-mobile block structure with self-correcting convolution, designing a lightweight ResNet50. The meta-mobile block is a network module designed for lightweight and efficient feature extraction. It cleverly combines the residual learning concept of the residual network (ResNet) with the optimization strategies of mobile networks. Based on residual connections, this module only contains convolution operations and an efficient multi-head self-attention mechanism, eliminating other complex structures and ensuring computational simplicity. Within the module, the input features are first processed through 1×1 convolution. Then, the position information of the key, query, and value is calculated using the efficient multi-head self-attention mechanism with an expanded window. The features generated in these two steps are weighted and fused to enhance the expressiveness of the features. Next, the depthwise separable convolution (DW-Conv) is used to achieve downsampling through stride adaptation. This not only effectively reduces computational complexity but also takes into account the advantages of dynamic global feature fusion, significantly improving feature extraction of low-light targets. The meta-mobile block design does not require the additional introduction of position embedding to provide an inductive bias for the MHSA, further simplifying the model structure. Compared with conventional convolution, DW-Conv has only 36% of the parameters when the input and output feature maps are the same. This feature makes the meta-mobile block more convenient to use. While achieving efficient feature extraction, the block also realizes the lightweight design of the network structure.

4. The method for self-correcting low-light target detection based on pyramid edge enhancement according to claim 3, wherein: The last 1×1 convolution layer in the meta-mobile block structure is replaced with a self-correcting convolution. This decomposes the feature map and internally expands the receptive field of each convolution layer. Using a novel self-correcting operation, the self-correcting convolution adaptively builds long-range spatial and inter-channel dependencies around each spatial location, thereby enhancing contextual information and enriching output features. Furthermore, the self-correcting convolution is simple and versatile, requiring no additional parameters or computational complexity. Self-rectified convolution splits the input feature map into two scale spaces, namely the original scale feature space and self-correcting scale space ; In the self-correcting scale space In the first step, the convolution kernel is used as Perform average pooling operation on the input features to obtain the input Weights at different scales, added to the input; The features extracted by convolution are corrected by Sigmoid activation function to generate a corrected weight map. Finally, this weight map is used to correct the feature map in the original scale feature space to achieve self-correction function. Convolution resizes the feature map and extracts features extracted by rectified convolution At the same time, in the original scale feature space, use Convolution feature map Perform convolution operation to extract original scale features ; Finally, the features of the self-corrected scale space output And the features output from the original scale feature space Perform splicing operations to obtain the final output features containing two scale space information ; This final output feature It contains both the spatial detail information of the original scale features and the contextual information of the self-corrected scale features, thereby improving the model's ability to extract and understand image semantic information; The whole process of self-correcting convolution is calculated as (11) (12) (13) (14) in, Upsample for nearest neighbor interpolation to restore the image size dimension, For average pooling, long-range spatial and channel dependencies are constructed between pixels and target spatial positions in each feature map to obtain richer and more accurate feature maps; is the convolution operation, is the convolution kernel, For Hadamard, is the activation function Sigmoid; The meta-mobile block module is embedded in the P3, P4, and P5 output layers of ResNet50. The last 1×1 convolution layer in the meta-mobile block structure is replaced with a self-correcting convolution. The designed SCC-ResNet50 not only reduces redundant computations but also fully utilizes the characteristics of self-correcting convolution to deeply explore the dependencies between pixels and object spatial positions in each feature map, thereby obtaining richer and more accurate feature maps and improving classification accuracy. Corrected Conv takes the input features Calculated as: (15) (16) (17) in Concate is the feature concatenation operation. is the input feature, and To input features The two feature spaces are split. and are the features extracted by self-correction convolution and original scale convolution respectively. is the output feature of the SCC-ResNet50 network.

Images