Zero reference curve estimation weak light image enhancement method and related device

Abstract

The invention relates to the technical field of image enhancement, in particular to a zero reference curve estimation weak light image enhancement method and a related device, and the method comprises the steps: obtaining a workshop image, inputting the workshop image into a pre-trained weak light image enhancement model when the average brightness value of the workshop image is lower than a preset threshold value, and obtaining a pixel curve parameter graph of RGB three channels; and the workshop image is subjected to pixel adjustment according to the pixel curve parameter diagram of the RGB three channels, so that the workshop pixel image is enhanced, the brightness, contrast and color authenticity of the low-light image can be effectively improved, noise and artifacts can be inhibited, and the method has the characteristics of high efficiency, flexibility and low resource consumption and is suitable for popularization and application. The problem that partial exposure is overexposed or hidden noise is amplified due to the fact that an existing weak light image enhancement technology is not flexible enough to adjust the visual effect of a local area is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and specifically to a zero-reference curve estimation low-light image enhancement method and related device, especially a zero-reference curve estimation low-light image enhancement method and related device for the inspection of printing workshops. Background Art

[0002] With the continuous improvement of the automation and intelligence levels of printing workshops, the application of inspection robots in low-light environments has gradually become an important means to ensure the stable operation of workshop equipment and production quality. The inspection robots rely on the vision system to collect workshop images in real time for monitoring and quality inspection, so higher requirements are put forward for the clarity, brightness and detail performance of the images. However, limited by the complex lighting conditions in printing workshops, especially in low-light environments, due to equipment occlusion, ambient light changes, etc., the images collected by inspection robots often have problems such as insufficient brightness, low contrast, and missing details, which directly affect the monitoring accuracy of the equipment operation status and the real-time analysis ability of production quality.

[0003] Traditional low-light image enhancement technologies mainly include histogram equalization, gamma correction, Retinex algorithm, etc. Although these methods can improve the brightness and contrast of images to a certain extent, they also have the problem of inflexible adjustment of the visual effects of local areas, resulting in overexposure in some parts or amplification of hidden noise. When dealing with images under complex lighting conditions, the effects are often not ideal and the applicable scenarios are limited. Summary of the Invention

[0004] Aiming at the problem in the prior art that the low-light image enhancement technology has inflexible adjustment of the visual effects of local areas, resulting in overexposure in some parts or amplification of hidden noise, the present invention provides a zero-reference curve estimation low-light image enhancement method and related device.

[0005] To achieve the above object, the present invention adopts the following technical solutions: The present invention provides a zero-reference curve estimation low-light image enhancement method, including: Obtain a workshop image; When the average brightness value of the workshop image is lower than a preset threshold, input the workshop image into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map of the RGB three channels; Enhance the workshop image according to the pixel curve parameter map of the RGB three channels to obtain an enhanced image; Wherein, the low-light image enhancement model is obtained by introducing a lightweight denoising module into the first input layer of the Zero-DCE model and replacing the feature extraction module with a module combining depthwise separable convolution and dilated convolution.

[0006] Optionally, the preset threshold is 60.

[0007] Optionally, it further includes a pre-training process of the low-light image enhancement model. The training dataset used in the pre-training process of the low-light image enhancement model is obtained by the following method: Collect images with different exposure parameters in the same scene; Select images with different exposure levels from the existing public low-light image dataset and merge them with the images collected with different exposure parameters in the same scene to obtain the training dataset.

[0008] Optionally, the activation functions of the low-light image enhancement model are the ReLU activation function and the Tanh activation function.

[0009] Optionally, the method for enhancing the workshop image according to the pixel curve parameter map of the RGB three channels to obtain the enhanced image is as follows: Based on the pixel curve parameter map of the RGB three channels, perform pixel-by-pixel adjustment on the workshop image using the luminance enhancement LE curve. The luminance enhancement LE curve is:

[0010] Wherein, is the luminance enhancement LE curve function; represents the pixel coordinates, represents the parameter mapping of the same size as the given image, is the enhanced image of, is the workshop image.

[0011] Optionally, it further includes a pre-training process of the low-light image enhancement model. The loss function used in the pre-training process of the low-light image enhancement model is:

[0012] Wherein, is the smoothness loss function; is the spatial consistency loss function; is the color loss function; is the exposure loss function; is the perceptual loss function; is the weight coefficient of the smoothness loss function; is the weight coefficient of the color loss function; is the weight coefficient of the exposure loss function.

[0013] Optionally, the perceptual loss function is:

[0014] Among them, is the input image; is the target image; is the feature map extracted from a certain layer of the VGG16 network for the input image; is the feature map extracted from a certain layer of the VGG16 network for the target image; is the number of channels of the feature map; is the height of the feature map; is the width of the feature map; The color loss function is:

[0015] Among them, represents the average intensity value of the channel in the enhanced image; represents the average intensity value of the channel in the enhanced image; represents the set of color channel pairs; represents the pair of red and green channels, represents the pair of red and blue channels, represents the pair of green and blue channels; The spatial consistency loss function is:

[0016] Among them, represents the number of local regions; represents the four adjacent regions centered on the region i ; i and j represent the pixel indices in the image, i represents the current pixel, j represents the pixel adjacent to i ; represents the brightness value of the enhanced image Y at the pixel i ; represents the brightness value of the enhanced image Y at the pixel j ; represents the brightness value of the input image I at the pixel i ; represents the brightness value of the input image I at the pixel j ; The exposure loss function is:

[0017] Among them, represents the number of non-overlapping local regions of size 16×16; represents the -th local region in the enhanced image, is the gray level in the RGB color space; is the region index; The smoothness loss function is:

[0018] Among them, is the number of iterations, represents the horizontal gradient operation; vertical gradient operation; represents the current pixel; represents the set of color channels; represents the color channel; represents the -th pixel in the channel's brightness value.

[0019] The present invention also provides a zero-reference curve estimation low-light image enhancement system, including: Image acquisition unit: used to acquire workshop images; Pixel curve parameter map acquisition unit: when the average brightness value of the workshop image is lower than a preset threshold, input the workshop image into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map of the RGB three channels; Enhanced image acquisition unit: enhance the workshop image according to the pixel curve parameter map of the RGB three channels to obtain an enhanced image; Among them, the low-light image enhancement model is obtained by introducing a lightweight denoising module into the first input layer of the Zero-DCE model and replacing the feature extraction module with a module combining depthwise separable convolution and dilated convolution.

[0020] The present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0021] The present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0022] Compared with the prior art, the present invention has the following beneficial effects: A method for estimating a zero-reference curve to enhance low-light images in the present invention. This method obtains a workshop image. When the average brightness value of the workshop image is lower than a preset threshold, the workshop image is input into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map for the three RGB channels, and the workshop image is pixel-adjusted according to the pixel curve parameter map for the three RGB channels to achieve the enhancement of the workshop pixel image. Among them, the low-light image enhancement model is an improved Zero-DCE model. By introducing a pre-denoising module to perform denoising operations on low-light images, it can enhance the image brightness while maintaining the clarity and details of the image. A module combining depthwise separable convolution and dilated convolution is introduced into the convolutional layer of the Zero-DCE model to extract local and global features of the image respectively, effectively improving the feature expression ability. Enhancing the workshop image through the pixel curve parameter map for the three RGB channels can better maintain the details and color balance of the image. Compared with traditional global adjustment methods, this method can more finely control the brightness curve of each channel, thereby avoiding detail loss and color distortion, achieving the comprehensive capture of details and structural information of low-light images, and realizing the flexible adjustment of the visual effects of each region. It can not only avoid problems such as partial overexposure or amplified hidden noise, but also improve the efficiency of feature extraction, reduce the consumption of computing resources, and is more suitable for image analysis in scenes with complex lighting conditions.

[0023] It also includes the pre-training process of the low-light image enhancement model. The training dataset used in the pre-training process of the low-light image enhancement model is obtained by collecting images with different exposure parameters in the same scene, then selecting images with different exposure levels from existing public low-light image datasets and merging them with the images collected with different exposure parameters in the same scene to obtain the training dataset. Among them, the collection of images with different exposure parameters in the same scene can form a multi-exposure sequence, providing real illumination change information to cover image samples with different illumination intensities and detail performances, enabling the model to learn more accurate brightness adjustment strategies, thereby avoiding distortion or over-adjustment when enhancing the image, improving the generalization ability of the model, enabling it to better adapt to different low-light environments in practical applications, and improving the robustness of the model.

[0024] The activation functions of the low-light image enhancement model are the ReLU activation function and the Tanh activation function. The ReLU activation function can effectively suppress the problem of gradient disappearance, avoid the overfitting of the model to noise or irrelevant details, and improve the training efficiency of the network. The Tanh activation function can enhance the nonlinear fitting ability of the network, enable the model to learn more complex mapping relationships, and improve the contrast and detail performance of low-light images.

[0025] In the loss function adopted by the low-light image enhancement model, a perceptual loss function, a color loss function, a spatial consistency loss function, an exposure loss function, and a smoothness loss function are introduced to optimize the training of the network. Among them, the perceptual loss function optimizes the model by comparing the differences between the enhanced image and the target image in the high-level feature space, which can better retain the semantic information of the image, avoid the loss of important content in the enhanced image, and effectively avoid artifacts or distortion problems during the enhancement process. The color loss function is used to constrain the consistency of the color distribution between the enhanced image and the target image, ensuring that the color distribution of the enhanced image is consistent with the real scene, avoiding color deviation or oversaturation phenomena, and effectively restoring and balancing the colors of the image. The spatial consistency loss function is used to constrain the spatial structure consistency between the enhanced image and the input image in the local area, effectively reducing artifacts or noise that may be introduced during the enhancement process. The exposure loss function is used to constrain the exposure level of the enhanced image to make it close to the target exposure level, effectively controlling the brightness of the enhanced image, avoiding overexposure or underexposure phenomena, being able to adapt to the image enhancement requirements under different lighting conditions, and enhancing the robustness of the model. The smoothness loss function is used to constrain the local smoothness of the enhanced image, avoiding the amplification of noise or artifacts caused by over-enhancement, effectively suppressing the noise that may be introduced during the enhancement process, making the transition of the enhanced image smoother, and improving the overall visual effect. The above loss functions are used in combination, which can optimize the training process from multiple perspectives and comprehensively improve the quality of the enhanced image. It can not only maintain the semantic information, color authenticity, and local structure of the image, but also effectively control the exposure level, suppress noise, and improve the smoothness of the image. Finally, the enhanced image is more natural, clear, and in line with human visual perception.

[0026] The present invention also provides a zero-reference curve estimation low-light image enhancement system. Through the highly centralized image acquisition unit, pixel curve parameter map acquisition unit, and enhanced image acquisition unit, the system can effectively enhance the workshop image, effectively improve the brightness, contrast, and color authenticity of the low-light image, while suppressing noise and artifacts. It has the characteristics of high efficiency, flexibility, and low resource consumption, is suitable for applications in real-time monitoring scenarios such as workshops, can significantly improve the image quality in low-light environments, and provides reliable support for subsequent image analysis and decision-making.

[0027] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented. The terminal device optimizes the model structure through a lightweight denoising module and a module combining depthwise separable convolution and dilated convolution, and can achieve efficient image enhancement processing with limited computing resources, meeting the real-time requirements. The processor can quickly complete the image enhancement task by executing the optimized computer program, reducing the processing delay, and is suitable for real-time application scenarios such as workshop monitoring.

[0028] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented. By storing the low-light image enhancement method of the present invention in the form of a computer program in a computer-readable storage medium, this implementation method not only improves the efficiency and effect of low-light image enhancement, but also provides users with a convenient and reliable usage experience, enabling the program to be quickly deployed to a variety of devices and scenarios, and is suitable for wide application in fields such as workshop monitoring and industrial inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic flow chart of a zero-reference curve estimation low-light image enhancement method of the present invention.

[0030] Figure 2 It is a schematic detailed flow chart of the entire process of a zero-reference curve estimation low-light image enhancement method of the present invention.

[0031] Figure 3 It is a framework diagram of a low-light image enhancement model of the present invention.

[0032] Figure 4 It is a comparison chart before and after image enhancement using the method of the present invention; among them, a, c, and e are original low-light images in different scenarios respectively; b, d, and f are the enhanced images corresponding to a, c, and e in sequence.

[0033] Figure 5 It is a structural diagram of a zero-reference curve estimation low-light image enhancement system of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0035] It should be noted that the terms "first", "second", etc. in the description, claims and above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0036] The following further elaborates on the present invention with specific embodiments, which is an explanation rather than a limitation of the present invention.

[0037] See Figure 1 and Figure 2 , the present invention discloses a method for enhancing low-light images by estimating a zero-reference curve, including: S1: Obtain workshop images, specifically: Obtain workshop images, and discriminate the image quality, select images with an average brightness value lower than a preset threshold as input, and perform image enhancement; preferably, the preset threshold is 60; S2: When the average brightness value of the workshop image is lower than the preset threshold, input the workshop image into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map of the RGB three channels; The pre-training of the low-light image enhancement model includes: Collection of training data sets: Collect images with different exposure parameters in the same scene; Collect images with different exposure parameters in the same scene, that is, collect images of the printing workshop under different exposure parameter conditions, and obtain a multi-exposure sequence image by adjusting shooting parameters such as exposure time and aperture size, and construct a low-light image data set of the printing workshop as the training data set; Use a high-precision industrial camera to sample the typical working environment of the printing workshop multiple times, and collect images of the same scene under different exposure parameter conditions to form a multi-exposure sequence to cover image samples with different light intensities and detail performances.

[0038] Select images with different exposure levels from the existing public low-light image dataset and merge them with the images with different exposure parameters in the same scene collected, to obtain a training dataset, that is, to enrich the diversity and generalization ability of the training dataset. Select representative low-light images from the existing public low-light image dataset and fuse them with the actual low-light images collected in the printing workshop to form a low-light image dataset for printing workshop inspection. Preferably, the public low-light image dataset is a single image contrast enhancement dataset (Single Image Contrast Enhancement, SICE). Divide the training dataset, that is, the low-light image dataset of the printing workshop, into a training set, a validation set and a test set according to the ratio of 70%, 15% and 15% for the training, parameter tuning and performance evaluation of the low-light image enhancement model. Build a low-light image enhancement model and train it using the training dataset. Build a low-light image enhancement model, named ZeroPT-net model. Among them, the low-light image enhancement model is obtained by introducing a lightweight denoising module into the first input layer of the Zero-DCE model and replacing the feature extraction module with a module that combines depthwise separable convolution and dilated convolution (hereinafter referred to as depthwise separable dilated convolution). Perform downsampling on the input training dataset and normalize the pixel values to the interval [0,1] to ensure the numerical stability and computational efficiency of the subsequent network. The activation functions of the low-light image enhancement model are ReLU activation function and Tanh activation function. See Figure 3 , design the architecture of the low-light image enhancement model, which consists of eight convolutional layers. The overall network model structure includes: a lightweight denoising module, a feature extraction module and a feature fusion module. Lightweight denoising module (layer 1): Introduce a lightweight denoising module at the initial stage of the low-light image enhancement model, which can effectively remove noise and improve the quality of the input image.

[0039] Feature extraction module (layers 2-7): Combine the depthwise separable convolution and dilated convolution modules to extract the local and global features of the image respectively, gradually extract the key information through multiple layers of convolution, and retain the details at the same time to ensure that the enhanced image reaches the optimal effect in terms of brightness, contrast and detail performance.

[0040] Feature fusion module (layer 8): Through feature stitching and fusion technology, organically combine the local and global features to generate the final pixel-level adjustment parameters for guiding the subsequent image enhancement operations.

[0041] Furthermore, the specific parameters of each layer of the low-light image enhancement model are as follows: Layer 1: 3×3 convolution, with 3 input channels, 3 output channels, a stride of 1, padding of 1, and the ReLU activation function.

[0042] Layer 2: Depthwise separable dilated convolution: with 3 input channels, 32 output channels, a convolution kernel size of 3×3, a dilation rate of 2, padding of 2, a stride of 1, and the ReLU activation function; Layers 3 to 5: Depthwise separable dilated convolution: with 32 input channels, 32 output channels, a convolution kernel size of 3×3, a dilation rate of 2, padding of 2, a stride of 1, and the ReLU activation function; Layer 6: Depthwise separable dilated convolution: with 64 input channels (feature concatenation of Layer 4 and Layer 5), 32 output channels, a convolution kernel size of 3×3, a dilation rate of 2, padding of 2, a stride of 1, and the activation function: ReLU; Layer 7: Depthwise separable dilated convolution: with 64 input channels (feature concatenation of Layer 3 and Layer 6), 32 output channels, a convolution kernel size of 3×3, a dilation rate of 2, padding of 2, a stride of 1, and the ReLU activation function.

[0043] Layer 8: Depthwise separable dilated convolution: with 64 input channels (feature concatenation of Layer 2 and Layer 7), 3 output channels, a convolution kernel size of 3×3, a dilation rate of 2, padding of 2, a stride of 1, and the Tanh activation function.

[0044] Among them, the ReLU activation function is:

[0045] The Tanh activation function is:

[0046] Among them, is the input workshop image; The loss function used in the pre-training process of the low-light image enhancement model is:

[0047] Among them, is the smoothness loss function; is the spatial consistency loss function; is the color loss function; is the exposure loss function; is the perceptual loss function; is the weight coefficient of the smoothness loss function; is the weight coefficient of the color loss function; is the weight coefficient of the exposure loss function; the above weight coefficient ensures the trade-off effect of different losses in the total optimization objective. During the training process, is 1600, is 5, is 10; Among them, the perceptual loss function is used to enhance the network's ability to capture high-level semantic features and ensure the visual perception consistency between the enhanced image and the real image. The features of the enhanced image in the high-level feature layer are extracted through the pre-trained VGG16 network, and the formula is:

[0048] Among them, is the input workshop image; is the target image; is the feature map extracted from a certain layer of the VGG16 network for the input image; is the feature map extracted from a certain layer of the VGG16 network for the target image; is the number of channels of the feature map; is the height of the feature map; is the width of the feature map; The color loss function is used to constrain the color stability after image enhancement, reduce the color difference between different channels, and avoid color deviation. The formula is:

[0049] Among them, represents the average intensity value of the channel in the enhanced image; represents the average intensity value of the channel in the enhanced image; represents the set of color channel pairs; ( p , q ) represents a pair of channels; represents the red-green channel pair, represents the red-blue channel pair, represents the green-blue channel pair; The spatial consistency loss function is used to maintain the spatial structure consistency between the enhanced image and the original image, and is achieved by comparing the local gradient differences between the enhanced image and the original image. The formula is:

[0050] Among them, represents the number of local regions; represents the regioni Four adjacent regions centered on; i and j Indicates the pixel index in the image, i Represents the current pixel, j Represents the one adjacent to i Adjacent pixels; Indicates the enhanced image Y At the pixel i The brightness value at; Indicates the enhanced image Y At the pixel j The brightness value at; Indicates the input image I At the pixel i The brightness value at; Indicates the input image I At the pixel j The brightness value at; The exposure loss function Is used to constrain the overall brightness distribution of the image, ensure the uniform brightness of the enhanced image, avoid the appearance of too dark or too bright regions, and measure the distance between the average intensity value of the local region of the image and the good exposure level E. The formula is:

[0051] Where, Indicates the number of non-overlapping local regions of size 16×16; Indicates the average intensity value of the th local region in the enhanced image, Is the gray level in the RGB color space; Is the region index, indicating the k th 16×16 non-overlapping local region; The smoothness loss function Adds illumination smoothness loss to each curve parameter map To constrain the monotonic relationship between adjacent pixels, reduce noise and artifacts. The formula is:

[0052] Where, Is the number of iterations, Indicates the horizontal gradient operation; Vertical gradient operation; Represents the current pixel; Indicates the set of color channels; Indicates the color channel; Indicates the th pixel in the Channel brightness value; During the training process, the adaptive optimizer Adam is used to train the low-light image enhancement model. The initial learning rate is set to 0.0001, and the weight decay coefficient is set to 0.0001 to reduce the risk of overfitting. During the training process, the gradient is clipped each time the parameters are updated, and the upper limit of the gradient clipping is set to 0.1 to avoid the problem of gradient explosion. The batch size Batch Size is 8 and the total number of training epochs Epochs is 100 during the training process. The loss value is output every 10 iterations for real-time monitoring of the training effect. At the same time, the model snapshot is saved every training cycle for subsequent debugging and performance comparison.

[0053] S3: Enhance the workshop image according to the pixel curve parameter map of the RGB three channels to obtain an enhanced image, specifically: That is, the pixel curve parameter map of the RGB three channels is decomposed into 3 curve parameter maps, and 8 iterations of adjustment are performed pixel by pixel in combination with the workshop image. Specifically, the brightness enhancement LE curve is used to optimize the image brightness, contrast and details, and finally output a high-quality enhanced image that meets the quality requirements of the printing workshop image analysis and processing, specifically: Based on the pixel curve parameter map of the RGB three channels, use the brightness enhancement LE curve to adjust the workshop image pixel by pixel. The brightness enhancement LE curve is:

[0054] Among them, is the brightness enhancement LE curve function; represents the pixel coordinates, represents the parameter mapping of the same size as the given image, is the enhanced image of, is the workshop image.

[0055] See Figure 4 , for the comparison chart before and after image enhancement using the method of the present invention. It can be seen that while the image enhancement can be achieved by this method, the original low-light image before enhancement can be seen. Affected by insufficient light, the overall brightness is low. The brightness of the dark area of the enhanced image is effectively improved, and at the same time, the overexposure phenomenon is avoided. The enhanced image retains more structural information and details while ensuring clarity.

[0056] It can be seen that this method can effectively enhance low-light images in a complex lighting environment and is more suitable for actual scenarios with complex lighting such as printing workshop inspections.

[0057] See Figure 5 , the present invention provides a zero-reference curve estimation low-light image enhancement system, including: Image acquisition unit: used to acquire workshop images; Pixel curve parameter map acquisition unit: When the average brightness value of the workshop image is lower than the preset threshold, the workshop image is input into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map of the RGB three channels; Enhanced image acquisition unit: Enhance the workshop image according to the pixel curve parameter map of the RGB three channels to obtain an enhanced image; Among them, the low-light image enhancement model is obtained by introducing a lightweight denoising module into the first input layer of the Zero-DCE model and replacing the feature extraction module with a module combining depthwise separable convolution and dilated convolution.

[0058] Through the highly centralized image acquisition unit, pixel curve parameter map acquisition unit and enhanced image acquisition unit, this system realizes the effective enhancement of the workshop image, can effectively improve the brightness, contrast and color authenticity of the low-light image, while suppressing noise and artifacts, has the characteristics of high efficiency, flexibility and low resource consumption, is suitable for application in real-time monitoring scenarios such as workshops, can significantly improve the image quality in low-light environments, and provides reliable support for subsequent image analysis and decision-making.

[0059] The present invention provides a terminal device including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0060] The computer program can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention.

[0061] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0062] The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0063] The memory can be used to store the computer program and / or module. By running or executing the computer program and / or module stored in the memory, and invoking the data stored in the memory, the processor implements various functions of the terminal device.

[0064] If the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0065] In summary, the present invention provides a method and related device for enhancing low-light images by estimating a zero-reference curve. By obtaining a workshop image, when the average brightness value of the workshop image is lower than a preset threshold, the workshop image is input into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map for the three RGB channels, and the workshop image is pixel-adjusted according to the pixel curve parameter map for the three RGB channels to achieve the enhancement of the workshop pixel image. Among them, the low-light image enhancement model is an improved Zero-DCE model, and a preposed denoising module is introduced to perform denoising operations on the low-light image. A module combining depthwise separable convolution and dilated convolution is introduced into the convolutional layer of the Zero-DCE model to extract local and global features of the image respectively, effectively improving the feature expression ability. The low-light image enhancement model is jointly optimized by a perceptual loss function, a color loss function, a spatial consistency loss function, an exposure loss function, and a smoothness loss function. Using this model for image enhancement can more finely control the brightness curve of each channel, achieve the comprehensive capture of details and structural information of low-light images, not only avoid the problems of partial overexposure or amplified hidden noise, but also improve the efficiency of feature extraction, reduce the consumption of computing resources, and is more suitable for image analysis in scenes with complex lighting conditions.

[0066] The above are only the preferred embodiments of the present invention and are not intended to limit the technical solutions of the present invention. Those skilled in the art should understand that, without departing from the spirit and principles of the present invention, the technical solutions can be subject to several simple modifications and substitutions, and these modifications and substitutions also fall within the protection scope covered by the claims.

Claims

1. A method for low-light image enhancement using zero-reference curve estimation, characterized in that: include: Get images of the workshop; When the average brightness value of the workshop image is lower than the preset threshold, the workshop image is input into the pre-trained low-light image enhancement model to obtain the pixel curve parameter map of the RGB three channels; The workshop image is enhanced according to the pixel curve parameter map of the RGB three channels to obtain an enhanced image; The low-light image enhancement model is obtained by introducing a lightweight denoising module in the first input layer of the Zero-DCE model and replacing the feature extraction module with a module combining a depthwise separable convolution and a dilated convolution.

2. The zero reference curve estimation low light image enhancement method according to claim 1, characterized in that: The preset threshold is 60.

3. The zero reference curve estimation low light image enhancement method according to claim 1, characterized in that: The method also includes a pre-training process of a low-light image enhancement model, wherein the training data set used in the pre-training process of the low-light image enhancement model is obtained in the following manner: Collect images with different exposure parameters under the same scene; Images with different exposure levels are selected from the existing public low-light image dataset and merged with images with different exposure parameters collected in the same scene to obtain a training dataset.

4. The zero reference curve estimation low light image enhancement method according to claim 1, characterized in that: The activation functions of the low-light image enhancement model are the ReLU activation function and the Tanh activation function.

5. The zero reference curve estimation low light image enhancement method according to claim 1, characterized in that: The method for enhancing the workshop image according to the pixel curve parameter map of the RGB three channels to obtain the enhanced image is: Based on the pixel curve parameter diagram of the RGB three channels, the workshop image is adjusted pixel by pixel using the brightness enhancement LE curve, and the brightness enhancement LE curve is: in, It is the brightness enhancement LE curve function; represents pixel coordinates, represents a parameter map of the same size as a given image, yes The enhanced image, For workshop images.

6. The zero reference curve estimation low light image enhancement method according to claim 1, characterized in that: It also includes a pre-training process of a low-light image enhancement model, wherein the loss function used in the pre-training process of the low-light image enhancement model is for: in, is the smoothness loss function; is the spatial consistency loss function; is the color loss function; is the exposure loss function; is the perceptual loss function; is the weight coefficient of the smoothness loss function; is the weight coefficient of the color loss function; is the weight coefficient of the exposure loss function.

7. The method for low-light image enhancement using zero reference curve estimation according to claim 6, characterized in that: The perceptual loss function for: in, is the input image; is the target image; It is the feature map extracted from the input image at a certain layer of the VGG16 network; The feature map extracted from a certain layer of the VGG16 network for the target image; is the number of channels of the feature map; is the height of the feature map; is the width of the feature map; The color loss function for: in, Indicates the enhanced image The average intensity value of the channel; Indicates the enhanced image The average intensity value of the channel; Represents a collection of color channel pairs; Represents the red channel and green channel pair, Represents the red channel and blue channel pair, Represents the green channel and blue channel pair; The spatial consistency loss function for: in, Indicates the number of local areas; Indicates by region i Four adjacent areas as the center; i and j represents the pixel index in the image, i Represents the current pixel, j Representatives and i Adjacent pixels; Represents enhanced image Y In pixels i The brightness value at ; Represents enhanced image Y In pixels j The brightness value at ; Represents the input image I In pixels i The brightness value at ; Represents the input image I In pixels j The brightness value at ; The exposure loss function for: in, Represents the number of non-overlapping local regions of size 16×16; Indicates the enhanced image The average intensity value of the local area, is the grayscale in the RGB color space; is the region index; The smoothness loss function for: in, is the number of iterations, represents the horizontal gradient operation; Vertical gradient operation; Indicates the current pixel; Represents a collection of color channels; Indicates color channel; Indicates Pixels in The brightness value of the channel.

8. A zero reference curve estimation weak light image enhancement system, characterized in that: include: Image acquisition unit: used to acquire workshop images; A pixel curve parameter map acquisition unit is used to input the workshop image into a pre-trained low-light image enhancement model to obtain a pixel curve parameter map of RGB three channels when the average brightness value of the workshop image is lower than a preset threshold; Enhanced image acquisition unit: enhances the workshop image according to the pixel curve parameter map of the RGB three channels to obtain an enhanced image; The low-light image enhancement model is obtained by introducing a lightweight denoising module in the first input layer of the Zero-DCE model and replacing the feature extraction module with a module combining a depthwise separable convolution and a dilated convolution.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images