Dark light target detection method based on LP, storage medium and equipment
Through the image enhancement method based on Laplacian pyramid, the problem of low accuracy and robustness of the target detection model under low light conditions is solved, and efficient target detection under low light conditions is achieved, which is suitable for automatic driving and safety monitoring. Wait for the scene.
Patent Information
- Application Number
- CN202411459763.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-18
- Publication Date
- 2025-01-21
AI Technical Summary
The prior art is difficult to effectively improve the accuracy and robustness of the object detection model under low light conditions, and it is easy to introduce noise and artifacts, reducing the performance of the model.
An image enhancement method based on Laplacian pyramid is adopted. The image is decomposed into low-frequency and high-frequency parts through the Laplacian pyramid decomposition module. The global enhancement module and mask generation and fine-tuning modules are used for adaptive enhancement to improve the contrast of the image. and detailed information.
It significantly improves the recognition accuracy and robustness of the target detection model under low-light conditions, reduces noise, and has high computational efficiency, making it suitable for real-time applications.
Smart Images
Figure CN119339061A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision, and in particular to a dark light target detection method, storage medium and device based on LP. Background Art
[0002] With the rapid development of deep learning, object detection technology has made significant progress in many fields, such as the YOLO series, SSD, and Faster R-CNN models. However, most of these models are trained under normal or good lighting conditions and are highly dependent on ambient lighting. When the image is in a low-light environment, the contrast, brightness, and detail information in the image will be significantly reduced, resulting in a sharp drop in the performance of the detection model. This performance degradation is unacceptable in some key application scenarios, such as autonomous driving and safety monitoring tasks, where false detection or missed detection may have serious consequences.
[0003] In order to cope with the challenges of low-light conditions on image processing, some low-light image enhancement techniques have been proposed, such as URetinex-Net and Zero-DCE. These methods are mainly optimized for visual perception, aiming to improve the overall brightness and visual effects of the image. However, they are prone to introduce noise and artifacts during the enhancement process, and are not specifically designed for target detection tasks, so they may reduce the accuracy and robustness of the model in actual detection. Therefore, it is urgent to develop a low-light image enhancement method for target detection tasks. Summary of the invention
[0004] The purpose of the present invention is to solve the problem of low accuracy and robustness of low-light image detection and propose a low-light target detection method based on LP, comprising the following steps:
[0005] S1, obtain low-light images and divide the images into training sets and validation sets;
[0006] S2. constructing a low-light image target detection model, wherein the model includes an image enhancement part and a target detection part, wherein the image enhancement part includes: a Laplacian pyramid decomposition module, a global enhancement module, a mask generation module, a mask fine-tuning module, and a Laplacian pyramid reconstruction module;
[0007] The low-light image is input into the Laplacian pyramid decomposition module to decompose the image into a low-frequency component L and a high-frequency component H = [h0, h1, ...h N-1 ], N is the number of decomposition levels;
[0008] The global enhancement module extracts and enhances the multi-scale features of the low-frequency components to obtain enhanced low-frequency components.
[0009] The mask generation module is based on the low-frequency component L and the enhanced low-frequency component High frequency component with minimum resolution h N-1 Generate a pixel-by-pixel mask m N-1 ;
[0010] The mask is upsampled layer by layer using bilinear interpolation to obtain masks of different scales M = [m N-1 , ..., m1, m0]
[0011] The mask fine-tuning module adjusts the masks of different scales to obtain the adjusted masks of different scales. The adjusted masks of different scales are multiplied with the high-frequency components of each layer to obtain enhanced high-frequency components.
[0012] The Laplacian pyramid reconstruction module performs Laplacian pyramid reconstruction on the enhanced low-frequency component and the enhanced high-frequency component to generate an enhanced image;
[0013] The enhanced image is input into the target detection part to obtain the target detection result;
[0014] S3. Use the training set and the validation set to train and test the model to obtain a final low-light image target detection model, input the low-light image to be detected into the final low-light image target detection model to obtain a target detection result.
[0015] Furthermore, the Laplacian pyramid decomposition decomposes the image into frequency components of multiple scales. The input image is filtered and downsampled layer by layer to generate low-frequency components of different scales. At the same time, the upsampled difference between the low-frequency components of each layer and the low-frequency components of the upper layer generates high-frequency components. The formula is as follows:
[0016]
[0017] Low frequency component L = G N (I), high frequency component h i Represented as h i =G i (I)-Up(G i+1 (I)), where G i (I) represents the low-frequency component of the i-th layer of the Gaussian pyramid, I represents the input image, Down represents downsampling, Gaissian represents Gaussian filtering, and Up represents upsampling.
[0018] Furthermore, the global enhancement module includes an encoder, a decoder, a series of ordinary convolutions and dilated convolutions of different scales;
[0019] The low-frequency component L is firstly extracted with preliminary features through the encoder, and the preliminary features are re-extracted through three identical branches to obtain the first feature, the second feature, and the third feature respectively. The three branches are composed of two convolutional layers and a hole convolutional layer cascaded;
[0020] The first feature, the second feature, and the third feature are concatenated, and the concatenated features are passed through a convolution layer to obtain the fourth feature. The fourth feature is residually connected with the primary feature after passing through a convolution layer to obtain the fifth feature. The fifth feature is activated by the LeakRule function and then passed through the decoder to obtain the sixth feature. The sixth feature is residually connected with the low-frequency component L to obtain the enhanced low-frequency component L. - .
[0021] Furthermore, the convolutional layer of the first cascade of the three branches includes a 1×1 convolution, the convolutional layer of the second cascade includes a combination of a 3×3 convolution and a Relu activation function, and the dilated convolutional layer includes a 3×3 dilated convolution;
[0022] The convolutional layer through which the cascaded features pass includes 1×1 convolution.
[0023] Furthermore, the mask generation module includes four residual blocks and a convolutional block attention module; the first residual block, the second residual block, the third residual block, the convolutional block attention module, and the fourth residual block are cascaded in sequence;
[0024] The low-frequency component L and the enhanced low-frequency component L - , the high-frequency component h_(N-1) with the minimum resolution is feature concatenated and input into the first residual block, and the mask is output by the fourth residual block.
[0025] Furthermore, the mask fine-tuning module includes two convolutional layers and a LeakRule activation function, and the first convolutional layer, the LeakRule activation function, and the second convolutional layer of the mask fine-tuning module are cascaded in sequence.
[0026] Furthermore, the two convolutional layers of the mask fine-tuning module include 1×1 convolutions.
[0027] The present invention further proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned LP-based dark light target detection method is implemented.
[0028] The present invention also proposes an electronic device, comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the above-mentioned LP-based dark-light target detection method.
[0029] The beneficial effects brought by the technical solution provided by the present invention are:
[0030] The present invention divides low-light images into low-frequency and high-frequency parts through the Laplace pyramid decomposition module, and enhances the low-frequency components of the image through the global enhancement module, effectively improving the contrast of the image. At the same time, the adaptive mask of the mask generation module and the mask fine-tuning module is used to specifically enhance the edge and texture information in the image, thereby significantly improving the recognition accuracy and robustness of the target detection model under low-light conditions and reducing noise. This method not only has high computational efficiency and can meet the needs of real-time applications, but also can be seamlessly integrated with existing target detection models and support joint training. This makes the present invention have a wide range of application potentials and is suitable for scenes such as autonomous driving and security monitoring that require high detection accuracy and response speed. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a flow chart of a dark light target detection method based on LP according to an embodiment of the present invention;
[0032] Figure 2 is a structural diagram of a low-light image target detection model according to an embodiment of the present invention;
[0033] Figure 3 is a structural diagram of a global enhancement module according to an embodiment of the present invention;
[0034] Figure 4 is a structural diagram of a mask generation module according to an embodiment of the present invention;
[0035] Figure 5 is a structural diagram of a mask fine-tuning module according to an embodiment of the present invention;
[0036] Figure 6 It is a block diagram of an electronic device in an exemplary embodiment of the present invention. DETAILED DESCRIPTION
[0037] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0038] The flowchart of the dark light target detection method based on LP (Laplacian Pyramid) according to the embodiment of the present invention is as follows: Figure 1 , specifically including the following steps:
[0039] S1. Obtain low-light images and divide the images into training sets and validation sets.
[0040] S2. Build a low-light image target detection model, refer to Figure 2 , Figure 2It is a structural diagram of a low-light image target detection model of an embodiment of the present invention. The model includes an image enhancement part and a target detection part. The image enhancement part includes: a Laplacian pyramid decomposition module, a global enhancement module, a mask generation module, a mask fine-tuning module, and a Laplacian pyramid reconstruction module.
[0041] Laplacian pyramid decomposition decomposes an image into frequency components of multiple scales. The input image is filtered and downsampled layer by layer to generate low-frequency components of different scales. At the same time, the upsampled difference between the low-frequency components of each layer and the low-frequency components of the upper layer generates high-frequency components. Through this decomposition method, the low-frequency components mainly retain the global attribute information of the image, such as brightness and contrast, while the high-frequency components retain the local detail structure information, such as edges and textures. The formula is as follows:
[0042]
[0043] Low frequency component L = G N (I), high frequency component h i Represented as h i =G i (I)-Up(G i+1 (I)), where G i (I) represents the low-frequency component of the i-th layer of the Gaussian pyramid, I represents the input image, I∈R h×w×3 , Down represents downsampling, Gaissian represents Gaussian filtering, Up represents upsampling, and N is the number of decomposition levels.
[0044] The low-light image is input into the Laplacian pyramid decomposition module to decompose the image into a low-frequency component L and a high-frequency component H = [h0,h1,...h N-1 ], in the embodiment of the present invention, reference Figure 2 , N=3.
[0045] The global enhancement module extracts and enhances the multi-scale features of the low-frequency components to obtain enhanced low-frequency components. Enhances the brightness, contrast, and global properties of an image. Figure 3 , Figure 3This is a structural diagram of the global enhancement module of an embodiment of the present invention. The global enhancement module includes an encoder, a decoder, and a series of ordinary convolutions and dilated convolutions of different scales. The low-frequency component L is first extracted with preliminary features by the encoder. The preliminary features are respectively extracted through three identical branches to obtain the first feature, the second feature, and the third feature. In the embodiment of the present invention, the three branches are composed of two convolutional layers and a dilated convolutional layer cascaded. The first cascaded convolutional layer of the three branches includes a 1×1 convolution, the second cascaded convolutional layer includes a combination of a 3×3 convolution and a Relu activation function, and the dilated convolution layer includes a 3×3 dilated convolution. The first feature, the second feature, and the third feature are feature cascaded, and the cascaded features are passed through a 1×1 convolutional layer to obtain a fourth feature. The primary feature is residually connected with the fourth feature after a 1×1 convolutional layer to obtain the fifth feature. The fifth feature is passed through the LeakRule activation function and then passed through the decoder to obtain the sixth feature. The sixth feature is residually connected with the low-frequency component L to obtain an enhanced low-frequency component. By introducing dilated convolution, the network can expand the receptive field of the convolution kernel without increasing the amount of computation and parameters, thereby capturing a wider range of global features. After the encoder initially extracts the features, multi-scale convolution further enhances these features at different scales, and then the decoder performs feature fusion. The network uses multiple residual connection mechanisms to ensure that the original global structural information is retained during the enhancement process. Finally, the enhanced low-frequency features are added to the original low-frequency components to maintain the integrity and consistency of the image's global structural information.
[0046] The mask generation module is based on the low-frequency component L and the enhanced low-frequency component High frequency component with minimum resolution h N-1 Generate a pixel-by-pixel mask m N-1 , low-frequency components contain global attribute information of the image, such as lighting, color, etc. The high-frequency components with the smallest resolution have the largest receptive field, can capture the global structural information of the image, and are highly correlated with the low-frequency components. Figure 4 , Figure 4 The structure diagram of the mask generation module of the embodiment of the present invention includes four residual blocks and a convolutional block attention module. The first residual block, the second residual block, the third residual block, the convolutional block attention module, and the fourth residual block are cascaded in sequence; the low-frequency component L and the enhanced low-frequency component High frequency component with minimum resolution h N-1 After feature concatenation, the first residual block is input, and the fourth residual block outputs the mask m N-1Through a lightweight convolutional residual network combined with a convolutional block attention module, we can effectively learn the specific visual attributes of the image domain and capture the correlation between low-frequency components and high-frequency components, and finally output a pixel-by-pixel mask. The mask can determine which areas of the high-frequency components need to be enhanced (such as the edge area of the target) or suppressed (such as the background noise area) based on the global information provided by the low-frequency components.
[0047] The mask is upsampled layer by layer using bilinear interpolation to obtain masks of different scales M = [m N-1 , ..., m1, m0]. In the embodiment of the present invention, N=3, bilinear interpolation up-sampling is performed layer by layer on mask m2, mask m2 is up-sampled to obtain mask m1, and mask m1 is up-sampled to obtain mask m0.
[0048] The mask fine-tuning module adjusts masks of different scales. The mask fine-tuning module includes two convolutional layers and a LeakRule activation function. The first convolutional layer, the LeakRule activation function, and the second convolutional layer of the mask fine-tuning module are cascaded in sequence. Figure 5 , Figure 5 This is a structural diagram of the mask fine-tuning module of an embodiment of the present invention. Masks of different scales first pass through a 1×1 convolutional layer, then pass through the LeakRule activation function, and finally pass through the second 1×1 convolutional layer of the mask fine-tuning module to obtain adjusted masks of different scales.
[0049] The adjusted masks of different scales are multiplied with the high-frequency components of each layer to obtain enhanced high-frequency components.
[0050] The generated mask will be upsampled by bilinear interpolation layer by layer, and adaptively adjusted through the mask fine-tuning module. The adjusted masks of different scales are multiplied pixel by pixel with the high-frequency components of each layer to control the enhancement degree of each pixel in the high-frequency components. This method avoids a large number of convolution operations on high-resolution components, improves algorithm efficiency, and can more finely adjust high-frequency components to achieve better enhancement effects.
[0051] The Laplacian pyramid reconstruction module will enhance the low-frequency components and enhanced high frequency components Perform Laplacian pyramid reconstruction to generate enhanced images.
[0052] The enhanced image is input into the target detection part, such as the YOLOv5 network, to obtain the target detection result.
[0053] S3. Use the training set and the validation set to train and test the model to obtain a final low-light image target detection model, input the low-light image to be detected into the final low-light image target detection model to obtain a target detection result.
[0054] In an exemplary embodiment, a computer-readable storage medium is included, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned LP-based dark light target detection method is implemented.
[0055] See also Figure 6 In an exemplary embodiment, an electronic device is also included, including at least one processor, at least one memory, and at least one communication bus.
[0056] Among them, a computer program is stored in the memory, and the computer program includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the above-mentioned LP-based dark light target detection method.
[0057] The present invention uses the Exdark dataset as the research object for training and verification of effectiveness. The specific results are as follows:
[0058] The ExDark dataset (Extreme Low-Light Image Dataset) is a public dataset designed specifically for target detection and image classification tasks under low-light conditions. The dataset contains more than 7,000 low-light images, a total of 12 categories of target objects, covering scenes under various extreme low-light conditions, including night, street lights, indoor low light, etc. The images in the dataset show significant loss of brightness, contrast and color information due to insufficient lighting, so it is very challenging. This paper selects 80% of them as training sets and 20% as validation sets to train and validate the model. The mean average precision is used as an indicator to evaluate the performance of the model. The experimental results are shown in Table 1. Among them, mAP stands for mean Average Precision, mAP50 stands for the average precision with an IoU (intersection over Union) threshold of 0.5, and mAP50-95 stands for the average precision with an IoU (intersection over Union) threshold of 0.5 to 0.95.
[0059] Table 1 Experimental results
[0060]
[0061] From the results in Table 1, it can be seen that the method of the present invention is significantly better than the original yolov5. In addition, the inference time of the present method is only 1.7ms, and the number of parameters is only 58K, which meets the requirements of real-time target detection.
[0062] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A dark light target detection method based on LP, characterized in that: The following steps are involved: S1, obtain low-light images and divide the images into training sets and validation sets; S2. constructing a low-light image target detection model, wherein the model includes an image enhancement part and a target detection part, wherein the image enhancement part includes: a Laplacian pyramid decomposition module, a global enhancement module, a mask generation module, a mask fine-tuning module, and a Laplacian pyramid reconstruction module; The low-light image is input into the Laplacian pyramid decomposition module to decompose the image into a low-frequency component L and a high-frequency component H = [h0,h1,...h N-1 ], N is the number of decomposition levels; The global enhancement module extracts and enhances the multi-scale features of the low-frequency components to obtain enhanced low-frequency components. The mask generation module is based on the low-frequency component L and the enhanced low-frequency component High frequency component with minimum resolution h N-1 Generate a pixel-by-pixel mask m N-1 ; Mask m u-1 Perform bilinear interpolation upsampling layer by layer to obtain masks of different scales M = [m N-1 ,...,m1,m0] The mask fine-tuning module adjusts the masks of different scales to obtain the adjusted masks of different scales. The adjusted masks of different scales are multiplied with the high-frequency components of each layer to obtain enhanced high-frequency components. The Laplacian pyramid reconstruction module performs Laplacian pyramid reconstruction on the enhanced low-frequency component and the enhanced high-frequency component to generate an enhanced image; Input the enhanced image into the target detection part to obtain the target detection result; S3. Use the training set and the validation set to train and test the model to obtain a final low-light image target detection model, input the low-light image to be detected into the final low-light image target detection model to obtain a target detection result.
2. The method for detecting dark light targets based on LP according to claim 1, characterized in that: Laplacian pyramid decomposition decomposes an image into frequency components of multiple scales. The input image is filtered and downsampled layer by layer to generate low-frequency components of different scales. At the same time, the upsampled difference between the low-frequency components of each layer and the low-frequency components of the upper layer generates high-frequency components. The formula is as follows: Low frequency component L = G N (I), high frequency component h i Represented as h i =G i (I)-Up(G i+1 (I)), where G i (I) represents the low-frequency component of the i-th layer of the Gaussian pyramid, I represents the input image, Down represents downsampling, Gaissian represents Gaussian filtering, and Up represents upsampling.
3. The method for detecting dark light targets based on LP according to claim 1, characterized in that: The global enhancement module includes an encoder, a decoder, a series of ordinary convolutions and dilated convolutions of different scales; The low-frequency component L is firstly extracted with preliminary features through the encoder, and the preliminary features are re-extracted through three identical branches to obtain the first feature, the second feature, and the third feature respectively. The three branches are composed of two convolutional layers and a hole convolutional layer cascaded; The first feature, the second feature, and the third feature are cascaded, and the cascaded features are passed through a convolution layer to obtain the fourth feature. The fourth feature is residually connected with the feature after the primary feature passes through a convolution layer to obtain the fifth feature. The fifth feature passes through the LeakRule activation function and then passes through the decoder to obtain the sixth feature. The sixth feature is residually connected with the low-frequency component L to obtain the enhanced low-frequency component LL.
4. The method for detecting dark light targets based on LP according to claim 3, characterized in that: The first cascaded convolution layer of the three branches includes 1×1 convolution, the second cascaded convolution layer includes a combination of 3×3 convolution and Relu activation function, and the dilated convolution layer includes 3×3 dilated convolution; The convolutional layer through which the cascaded features pass includes 1×1 convolution.
5. The method for detecting dark light targets based on LP according to claim 1, characterized in that: The mask generation module includes four residual blocks and a convolutional block attention module; the first residual block, the second residual block, the third residual block, the convolutional block attention module, and the fourth residual block are cascaded in sequence; The low-frequency component L and the enhanced low-frequency component High frequency component with minimum resolution h N-1 After feature concatenation, the first residual block is input, and the mask is output by the fourth residual block.
6. The method for detecting dark light targets based on LP according to claim 1, characterized in that: The mask fine-tuning module includes two convolutional layers and a LeakRule activation function. The first convolutional layer, the LeakRule activation function, and the second convolutional layer of the mask fine-tuning module are cascaded in sequence.
7. The method for detecting dark light targets based on LP according to claim 6, characterized in that: The two convolutional layers of the mask fine-tuning module consist of 1×1 convolutions.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises computer-readable instructions, and the processor is configured to call the computer-readable instructions to execute the method according to any one of claims 1 to 7.