Weak light image enhancement method and system based on GANs and storage medium

Abstract

The embodiment of the invention discloses a GANs-based weak light image enhancement method and system and a storage medium, and belongs to the technical field of image enhancement, and the method comprises the steps: carrying out the enhancement of an original weak light image according to an automatic color gradation enhancement algorithm, and obtaining a contrast-enhanced weak light image; adjusting the brightness and texture of the weak light image according to a self-adaptive dimming algorithm of the SWGF to obtain an enhanced image I; based on an edge-reserved decomposition model, decomposing the enhanced image I into a detail layer and a base layer, sharpening the detail layer, and performing adaptive segmentation histogram equalization processing on the base layer; establishing a generative adversarial network, training the processed detail layer and the base layer, taking an improved SSIM loss function as an optimization target, and when the optimization target meets a threshold range, completing the training of the generative adversarial network; and determining an optimal enhanced image according to the trained generative adversarial network. According to the method, the weak light image can be comprehensively enhanced, and effective data are provided for subsequent image analysis.

Description

Technical Field

[0001] The present invention relates to the field of image enhancement technology, and in particular to a GANs-based low-light image enhancement method, system and storage medium. Background Art

[0002] Image enhancement, as one of the core technologies in the field of image processing, aims to improve image quality to meet the needs of specific applications and the visual experience of observers. Images acquired under ideal conditions can often be directly used in various applications. However, in actual scenarios, due to a variety of uncontrollable factors such as noise, fog, weak light, etc., image quality is often severely degraded, manifested as image blur, color distortion, reduced contrast, and loss of details.

[0003] These problems not only reduce the visual quality of the images, but also limit the effectiveness of the images in subsequent processing, analysis and engineering applications, especially for images taken in low-light environments. Due to insufficient light, the images usually exhibit significant defects such as low contrast, low visibility and unclear details. After undergoing image conversion, storage and transmission, the quality of these images often deteriorates further, making it difficult to meet the needs of scientific research or engineering practice.

[0004] To overcome these challenges, traditional methods include spatial domain-based image enhancement techniques, such as histogram equalization and filtering algorithms, and frequency domain-based image enhancement methods, such as Fourier transform and wavelet transform. However, these methods often have limited effects when processing complex degraded images. Summary of the invention

[0005] Based on this, it is necessary to propose a low-light image enhancement method based on GANs to address the above problems.

[0006] A low-light image enhancement method based on GANs, the method comprising the following steps: The original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; According to the SWGF adaptive dimming algorithm, the low-light image is adjusted The brightness and texture of the image are obtained to obtain an enhanced image I; Based on an edge-preserving decomposition model, the enhanced image I is decomposed into a detail layer and a base layer, the detail layer is sharpened, and the base layer is subjected to an adaptive segmented histogram equalization process; Establishing a generative adversarial network, training the processed detail layer and base layer, and taking the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed. The optimal enhanced image is determined based on the trained generative adversarial network.

[0007] In the above scheme, the original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. , specifically including: Acquire three single-channel histograms of the original low-light image: a red channel histogram, a green channel histogram, and a blue channel histogram; The pixel values ​​corresponding to the lowest 5% and highest 5% of the pixel values ​​of the three single-channel histograms are counted respectively, and recorded as the minimum pixel threshold and the maximum pixel threshold ; The pixels below the minimum threshold in the three single-channel histograms are The pixel value is assigned to 0, which is higher than the maximum pixel threshold The pixel value is assigned to 225; The minimum pixel threshold in the three single channel histograms and the maximum pixel threshold Linear grayscale stretching is performed on pixel values ​​between .

[0008] In the above scheme, the adaptive dimming algorithm according to SWGF is used to adjust the low-light image The brightness and texture of the image are obtained by obtaining an enhanced image I, which specifically includes: The low-light image Decomposed into approximate layers and residual layers:

[0009]

[0010] in, is the approximate layer, is the residual layer, For low-light images, is the regularization coefficient, SWGF rL Filter radius; For the approximate layer To perform dynamic compression:

[0011] in, is the natural logarithm operator, is a constant, is the approximate layer, is the approximate layer after dynamic compression; Determine the variables for global illumination enhancement:

[0012] in, is the light compression factor, is the first enhancement factor, is the approximate layer after dynamic compression, Variables enhanced for global illumination; The enhanced approximation layer is determined according to the following formula :

[0013] in, is the variable for global light enhancement, To enhance the approximation layer; Get the second enhancement factor :

[0014] in, represents the standard deviation of the image, is the approximate layer, To enhance the approximation layer, is the second enhancement factor; Determine the enhanced residual layer :

[0015] in, is the second enhancement factor, is the residual layer, To strengthen the residual layer; According to the enhanced approximation layer and the enhanced residual layer Determine the enhanced image I:

[0016] Where I is the enhanced image, To enhance the approximation layer, To enhance the residual layer.

[0017] In the above scheme, the edge-preserving decomposition model decomposes the enhanced image I into a detail layer and a base layer, sharpens the detail layer, and performs adaptive segmented histogram equalization processing on the base layer, specifically including: For each enhanced image I, determine its base layer :

[0018] Where I is the enhanced image, is the base layer corresponding to the enhanced image, μ is the mean filter, It is convolution processing; Determine the detail layer by subtracting the enhanced image from the base layer:

[0019] in, is the detail layer of the corresponding image, I is the enhanced image, is the base layer corresponding to the enhanced image.

[0020] In the above solution, the sharpening of the detail layer specifically includes:

[0021] in, To perform histogram equalization on the image, To perform Gaussian smoothing, is the detail layer image after sharpening, and 0.01 is the scaling factor is the input detail layer.

[0022] In the above solution, the sharpening of the detail layer specifically includes:

[0023] in, To perform histogram equalization on the image, To perform Gaussian smoothing, is the detail layer image after sharpening, and 0.01 is the scaling factor is the input detail layer.

[0024] In the above solution, the step of performing adaptive segmented histogram equalization processing on the base layer specifically includes: Statistics of the base layer Histogram h of Divide the histogram h equally into a low grayscale interval sub-histogram, a medium grayscale interval sub-histogram and a high grayscale interval sub-histogram; Determine the cumulative distribution functions of the low grayscale interval sub-histogram, the medium grayscale interval sub-histogram, and the high grayscale interval sub-histogram respectively and perform histogram equalization; Remap each grayscale subrange back to the base layer The corresponding gray value position in the image is obtained to obtain the enhanced base layer image. .

[0025] In the above scheme, the generation adversarial network is established, the processed detail layer and base layer are trained, and the improved SSIM loss function is used as the optimization target. When the optimization target meets the threshold range, the generation adversarial network training is completed, which specifically includes: The generative adversarial network includes a generator and two discriminators; Determine the enhanced base layer image according to the generator And the sharpened detail layer image The corresponding preliminary synthetic image ; The preliminary composite image is determined by the two discriminators fusion effect.

[0026] In the above scheme, the preliminary composite image is judged by two discriminators The fusion effects include: The preliminary synthesized image is determined by a loss function The fusion effect:

[0027]

[0028] in, , For two source images and , is the fused image, W is the number of image blocks after segmentation, represents the standard deviation, is the structural similarity value of the segmented image, is the loss function, Source Image and fused image The similarity of Source Image and fused image similarity;

[0029] in, is the mean, M and N are the image sizes in the horizontal and vertical directions respectively, is the pixel value of image X at position (i, j);

[0030] Among them, X and Y are the enhanced base layer images respectively. And the sharpened detail layer image , is the covariance of X and Y, and is the stability coefficient, is the similarity between X and Y, is the mean of X, is the mean of Y, for The mean of for The mean of for The standard deviation of for The standard deviation of .

[0031] The present application also proposes a low-light image enhancement system based on GANs, the system comprising: an automatic color scale enhancement unit, an adaptive dimming unit, an image processing unit and a generative adversarial network unit; The automatic color scale enhancement unit is used to enhance the original low-light image according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; The adaptive dimming unit is used to adjust the low-light image according to the adaptive dimming algorithm of SWGF. The brightness and texture of the image are obtained to obtain an enhanced image I; The image processing unit is used to decompose the enhanced image I into a detail layer and a base layer based on an edge-preserving decomposition model, and to sharpen the detail layer and perform adaptive segmented histogram equalization processing on the base layer; The generative adversarial network unit is used to establish a generative adversarial network, train the processed detail layer and base layer, and use the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed; and the optimal enhanced image is determined based on the generative adversarial network after the training is completed.

[0032] The present application also proposes a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps: The original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; According to the SWGF adaptive dimming algorithm, the low-light image is adjusted The brightness and texture of the image are obtained to obtain an enhanced image I; Based on an edge-preserving decomposition model, the enhanced image I is decomposed into a detail layer and a base layer, the detail layer is sharpened, and the base layer is subjected to an adaptive segmented histogram equalization process; Establishing a generative adversarial network, training the processed detail layer and base layer, and taking the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed. The optimal enhanced image is determined based on the trained generative adversarial network.

[0033] The embodiment of the present invention has the following beneficial effects: firstly, the original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; According to the SWGF adaptive dimming algorithm, adjust the low-light image brightness and texture to obtain an enhanced image I; based on an edge-preserving decomposition model, decompose the enhanced image I into a detail layer and a base layer, sharpen the detail layer, and perform adaptive segmented histogram equalization on the base layer; establish a generative adversarial network, train the processed detail layer and base layer, and use the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the training of the generative adversarial network is completed; determine the optimal enhanced image based on the trained generative adversarial network. This method combines multiple technologies such as automatic color scale enhancement, adaptive dimming, image decomposition and reconstruction, and generative adversarial network training to achieve comprehensive enhancement of low-light images. The optimal enhanced image finally outputted is significantly improved in contrast, brightness, texture details and overall visual effect, effectively solving the problem of poor image quality under low-light conditions, and providing high-quality input data for subsequent image analysis, recognition and other applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] in: Figure 1 Schematic diagram of the process of a low-light image enhancement method based on GANs in one embodiment. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] In the following description, a large number of specific details are given in order to provide a more thorough understanding of the present invention; however, it is obvious to those skilled in the art that the present invention can be implemented without one or more of these details; in other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described, and it should be understood that the present invention can be implemented in different forms and should not be interpreted as limited to the embodiments presented here; on the contrary, providing these embodiments will make the disclosure thorough and complete and fully convey the scope of the present invention to those skilled in the art. The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present invention. When used herein, the singular forms "a", "an" and "said / the" are also intended to include the plural forms, unless the context clearly indicates otherwise, and it should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0038] With the rapid development of computer vision and deep learning technology, image enhancement strategies based on deep learning have gradually emerged. By training deep learning models, these methods can automatically learn complex features in images and effectively restore image clarity and details. In addition, image enhancement methods and image fusion methods based on Retinex theory also provide new ideas for low-light image enhancement.

[0039] Although a variety of image enhancement methods have been proposed, effective solutions to problems such as image blur, low contrast, and poor visibility in low-light environments are still a research hotspot. In order to further improve the quality of low-light images, this paper proposes a low-light image enhancement algorithm with automatic color gradation enhancement and adaptive dimming based on a generative adversarial network (GAN), which aims to automatically adjust the brightness and contrast of the image and restore the detailed information of the image through the powerful ability of deep learning, thereby improving the overall quality and visual effect of the image.

[0040] The technical solution for enhancing low-light infrared images of the present invention is mainly divided into two stages. The first stage is mainly to enhance the source infrared image. The enhancement stage mainly includes two steps. First, an automatic color scale enhancement algorithm is used to enhance the low-light image to obtain a clearer image data set; secondly, by fusing the local and global contrast of the image, an adaptive dimming algorithm based on SWGF is proposed to improve the brightness and texture details of the low-light visible light image; the second stage is to introduce a generative adversarial network to perform adversarial enhancement on the enhanced infrared image. The second stage also includes two steps. First, a decomposition model based on edge preservation is used to decompose the enhanced infrared image into a detail layer and a base layer, and then the detail layer and the base layer are respectively subjected to detail sharpening processing and adaptive segmented histogram equalization processing; then the processed detail layer and base layer are input into the generative adversarial network, and the optimal enhanced image is obtained through adversarial training. The generative adversarial network includes a generator and two discriminators. The generator generates a fused image. The two discriminators are used to improve the perceptual quality of the fused image. The output is then refined through an adversarial learning process between the generator and the discriminator. The two discriminators are a texture detail discriminator and a basic information discriminator. The texture detail discriminator utilizes the characteristics of the fourth layer of VGG-16 to retain more significant detail information, and the basic information discriminator utilizes the characteristics of the first layer of VGG-6 to retain more detailed information.

[0041] In order to thoroughly understand the present invention, a detailed structure will be presented in the following description to illustrate the technical solution proposed by the present invention; optional embodiments of the present invention are described in detail as follows, but in addition to these detailed descriptions, the present invention may also have other implementation methods.

[0042] like Figure 1 As shown, in one embodiment, a low-light image enhancement method based on GANs is provided. The low-light image enhancement method based on GANs includes steps S101 to S105, which are described in detail as follows: S101, enhancing the original low-light image according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast ; The automatic color scale method improves the image by changing the grayscale value distribution and dynamic range of pixels. Its principle is to use a mapping-based mathematical function to convert the grayscale value of a single pixel into other grayscale values. By processing the original low-light image through the automatic color scale enhancement algorithm, the image contrast can be effectively improved. This step is the key to image preprocessing and lays a good foundation for subsequent processing, making the dark details in the image clearer and the bright parts not overexposed.

[0043] In some embodiments, the original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. , specifically including: Obtain three single-channel histograms of the original low-light image: red channel histogram, green channel histogram, and blue channel histogram; The pixel values ​​corresponding to the lowest 5% and highest 5% of the pixel values ​​of the three single-channel histograms are counted respectively, and recorded as the minimum pixel threshold and the maximum pixel threshold ; The pixels below the minimum pixel threshold in the three single-channel histograms are The pixel value is assigned to 0, which is higher than the maximum pixel threshold The pixel value is assigned to 225; For the minimum pixel threshold in the three single channel histograms and the maximum pixel threshold Linear grayscale stretching is performed on pixel values ​​between .

[0044] Preferably, the gray value of a single pixel can be converted to other gray values ​​by linear transformation. Linear gray value transformation is also called linear stretching, which is a linear function of the gray value of the original image. The formula is as follows:

[0045] The input and output images are and , the linear transformation coefficients are C and R.

[0046] The values ​​of the coefficients in the following formula can be adjusted to enhance the image to different degrees. The following is the general formula for linear grayscale stretching:

[0047] In the formula, and are the maximum and minimum grayscale values ​​of the input image, respectively. and are the maximum and minimum grayscale values ​​of the output image, respectively. In order to enhance brightness and contrast, the dynamic range of the image is increased from Convert to .

[0048] Furthermore, the present invention uses linear mapping to convert the remaining pixel values ​​(at the minimum pixel threshold) and the maximum pixel threshold ) to , as follows: Statistical single channel histogram, find out The thresholds of the first and last positions are denoted by and When counting a single-channel histogram, the first and last 5% thresholds refer to the pixel value range corresponding to the 5% with the lowest frequency and the 5% with the highest frequency in the histogram. That is, the first and last 5% thresholds are to sort the pixel values ​​in the histogram from low to high according to frequency, and then find the range corresponding to the pixel values ​​with cumulative frequencies reaching 5% and 95%. will be lower than The pixel value is assigned to 0, and the pixel value above The pixel value is assigned to 255, that is, the maximum grayscale value of the output image is set to 250, and the minimum grayscale value of the output image is set to 0; The remaining pixel values ​​of the image are corrected according to the linear grayscale stretching formula, namely:

[0049] Perform linear grayscale stretching operations on the above three channels respectively; After completing the correction of each channel as mentioned above, the dynamic range of the grayscale values ​​that account for a relatively large proportion will be widened, and a contrast-enhanced color image will be obtained. The histogram after automatic color level enhancement is more dispersed, the pixels are evenly distributed, and the overall brightness is enhanced, which is convenient for subsequent visual observation and analysis.

[0050] The above step S101 preliminarily widens the dynamic range of the grayscale value of the low-light image to make the image clearer, but some fine textures and edge details of the low-light image are not processed well, and it is still easy to ignore many details of the low-light image, resulting in low overall contrast of the image and poor target and environment recognition capabilities. Therefore, in step 102, an adaptive light adjustment method based on SWGF (Saliency Weighted Global Filter) is proposed. This method combines local and global contrast enhancement, and can enhance image details and textures while ensuring moderate exposure. Through the enhancement method, the visual quality of the low-light image is significantly improved, especially in the weak detail area, which further improves the quality of the final enhancement result.

[0051] S102, adjust the low-light image according to the SWGF adaptive dimming algorithm The brightness and texture of the image are obtained to obtain an enhanced image I; By further adjusting the brightness and texture of the image, aiming to balance the brightness distribution of the image while retaining or enhancing the texture details of the image, this step enhances the visual appeal of the image, making it more natural and easy to observe.

[0052] Preferably, the local contrast is used to The enhancement formula is as follows:

[0053] in, For images In pixels The average value around.

[0054] right Apply a low-pass bilateral filter to compute , the calculation formula is as follows:

[0055] in, and Usually represents the pixel position in the image, which can be a coordinate point in two-dimensional space, such as , , is the distance between two pixel locations, is a Gaussian function.

[0056] Calculate Gaussian function , the formula is as follows:

[0057] in, , .

[0058] After enhancing the image by using local contrast, the following will decompose the enhancement strategy for the enhanced image. Perform decomposition enhancement.

[0059] In some embodiments, the low-light image is adjusted according to the adaptive dimming algorithm of SWGF. The brightness and texture of the image are obtained by obtaining an enhanced image I, which specifically includes: Low-light images Decomposed into approximate layers and residual layers:

[0060]

[0061] in, is the approximate layer, is the residual layer, For low-light images, is the regularization coefficient, SWGF rL Filter radius; Preferably, set The filter radius for , regularization coefficient for .

[0062] For the approximate layer To perform dynamic compression:

[0063] in, is the natural logarithm operator, is a constant, is the approximate layer; Determine the variables for global illumination enhancement:

[0064] in, is the light compression factor, is the first enhancement factor, is the approximate layer after dynamic compression; The enhanced approximation layer is determined according to the following formula :

[0065] in, Variables enhanced for global illumination; Get the second enhancement factor :

[0066] in, Represents the standard deviation of the image; Determine the enhanced residual layer :

[0067] in, is the second enhancement factor, is the residual layer; According to the enhanced approximation layer and enhanced residual layer Determine the enhanced image I:

[0068] Where I is the enhanced image, To enhance the approximation layer, To enhance the residual layer.

[0069] In the above steps, in order to facilitate image compression and enhancement, dynamic compression operation is performed on the approximation layer in the logarithmic domain, and in order to prevent the logarithmic value from being 1, =1, at the same time, when When the light compression factor is less than 1, the contrast of the approximate layer will decrease. Specifically, calculate the light compression factor , the calculation formula is as follows:

[0070] In the formula, , and They are The maximum, minimum and average values ​​of the data are experimentally verified to be ideal when the cutoff values ​​are 2 and 5.

[0071] Furthermore, the enhancement factor is calculated , the calculation formula is as follows:

[0072] when hour, The image contrast will be enhanced while ensuring that the maximum output is greater than the maximum intensity.

[0073] The above steps can be used to design the dynamic compression factor according to the different average light intensity of the image in different environments. , applying an appropriate compression factor Adaptive illumination adjustment is achieved to effectively reduce the complexity of manually testing compression factors of different images. In addition, in order to avoid introducing too much brightness information during image processing, which may cause overexposure of the image, an appropriate light enhancement factor is also designed. To enhance the contrast, ensure that the maximum output value does not exceed the maximum intensity value of the original image, which helps to maintain the dynamic range of the image and prevent over-saturated areas in the image to maintain the authenticity and visualization effect of the image.

[0074] S103, based on the edge-preserving decomposition model, decomposing the enhanced image I into a detail layer and a base layer, sharpening the detail layer, and performing adaptive segmented histogram equalization processing on the base layer; The enhanced image is decomposed into a detail layer and a base layer and processed separately. The detail layer contains high-frequency information of the image (such as edges and textures), while the base layer contains low-frequency information (such as overall brightness and color). This decomposition method helps to more finely control different parts of the image and improve the processing effect. The sharpening of the detail layer enhances the clarity of the image, while the adaptive segmented histogram equalization of the base layer further improves the brightness distribution of the image and reduces the overexposed and underexposed areas.

[0075] The enhanced image is subjected to multi-scale edge-preserving decomposition through guided filters to obtain the corresponding base layer and a series of detail layers. Due to the characteristics of guided filtering, the decomposed base layer retains most of the structural contour features of the source image, and the detail layer retains a small part of the contour brightness and texture details of the source image. This decomposition method effectively improves the edge retention of the image.

[0076] The purpose of multi-scale edge-preserving decomposition is to obtain a base layer containing most of the structural features of the source image and a series of detail layers containing a small amount of texture information. The infrared image is decomposed using guided filters and subtractors. , get the filtered image, which is the base layer of the source image , the source image and base layer Do the difference and get the difference image as the detail layer .

[0077] In some embodiments, based on the edge-preserving decomposition model, the enhanced image I is decomposed into a detail layer and a base layer, and the detail layer is sharpened, and the base layer is subjected to adaptive piecewise histogram equalization processing, specifically including: For each enhanced image I, determine its base layer :

[0078] Where I is the enhanced image, is the base layer corresponding to the enhanced image, μ is the mean filter, It is convolution processing; Determine the detail layer by subtracting the enhanced image from the base layer:

[0079] in, is the detail layer of the corresponding image, I is the enhanced image, is the base layer corresponding to the enhanced image.

[0080] Preferably, μ is a mean filter with a window size of , and its value is set to 35.

[0081] In some embodiments, the detail layer is sharpened, specifically comprising:

[0082] in, To perform histogram equalization on the image, To perform Gaussian smoothing, is the detail layer image after sharpening, and 0.01 is the scaling factor is the input detail layer.

[0083] In some embodiments, the base layer is subjected to adaptive piecewise histogram equalization processing, specifically comprising: Statistical base layer Histogram h of The histogram h is equally divided into a low grayscale interval sub-histogram, a medium grayscale interval sub-histogram and a high grayscale interval sub-histogram; Determine the cumulative distribution functions of the low grayscale interval sub-histogram, the medium grayscale interval sub-histogram and the high grayscale interval sub-histogram respectively and perform histogram equalization; Remap each grayscale subrange back to the base layer The corresponding gray value position in the image is obtained to obtain the enhanced base layer image. .

[0084] Segmented histogram equalization divides the pixels of the image into three parts of low, medium and high gray levels according to the size of the pixel values, and then performs histogram equalization in the corresponding three approximately equally divided subspaces, which greatly improves the brightness, contrast and clarity of the image, while avoiding the over-enhancement and under-enhancement of the global histogram equalization, and overcomes the defects of artifacts and insufficient brightness of the platform histogram.

[0085] S104, establishing a generative adversarial network, training the processed detail layer and base layer, and taking the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed. GANs can generate images that are closer to reality and of higher quality through adversarial learning between the generator and the discriminator. Introducing the improved SSIM (structural similarity index) loss function as the optimization target helps to better maintain the structural information of the image during the training process, avoid distortion, and improve the visual quality of the image. When the optimization target meets the preset threshold, it means that GANs has been trained and can output the optimal enhanced image.

[0086] GANs, or generative adversarial networks, are a deep learning model that consists of two main parts: a transformer-based generator and two discriminators, the texture detail discriminator (DIS-D) and the base information discriminator (DIS-B). Typically, the final synthesized infrared image is obtained by the generator, and then the output infrared image is refined in the adversarial learning process between the generator and the discriminator.

[0087] The generator is used to generate an image after fusion of the detail layer and the base layer. After merging the two layers of images in the channel dimension, the initial feature extraction is performed through the convolutional neural network, and the mixed CNN features are input into the transformer fusion module to learn the global fusion relationship. Considering the consumption of computing resources and the representation of features, three downsampling operators are added before the transformer fusion module. In this process, the learned fusion relationship is upsampled to different scales and multiplied by the corresponding features to obtain preliminary results. The fusion features of the two scales are upsampled to the original image size and then superimposed to obtain the final fusion result.

[0088] The discriminator is used to improve the perceived quality of the output infrared image. Two discriminators are set here: the output image and the texture detail layer image, and the output image and the basic information layer image. These two discriminators provide the output image with detail information of the texture detail layer and an important part of the basic information layer.

[0089] In the training phase, the detail layer image and the base layer image are input into the generator to obtain a preliminary composite image. Then, the initial composite image passes through two discriminators, and the effect of the fused image is fed back through the loss function. The above two steps are performed alternately to realize the adversarial training between the generator and the discriminator. Finally, a generator with an ideal generation effect is obtained, achieving the purpose of infrared image enhancement.

[0090] In some embodiments, a generative adversarial network is established, the processed detail layer and base layer are trained, and the improved SSIM loss function is used as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed, which specifically includes: The generative adversarial network consists of a generator and two discriminators; Determine the enhanced base layer image according to the generator And the sharpened detail layer image The corresponding preliminary synthetic image ; Determine the initial synthetic image through two discriminators fusion effect.

[0091] In some embodiments, the preliminary composite image is judged by two discriminators. The fusion effects include: Determine the initial synthetic image through the loss function The fusion effect:

[0092]

[0093] in, , For two source images and , is the fused image, W is the number of image blocks after segmentation, represents the standard deviation, is the structural similarity value of the segmented image, is the loss function, Source Image and fused image The similarity of Source Image and fused image similarity;

[0094] in, is the mean, M and N are the image sizes in the horizontal and vertical directions respectively, is the pixel value of image X at position (i, j);

[0095] Among them, X and Y are the enhanced base layer images respectively. And the sharpened detail layer image , is the covariance of X and Y, and is the stability coefficient, is the similarity between X and Y, is the mean of X, is the mean of Y, for The mean of for The mean of for The standard deviation of for The standard deviation of .

[0096] The number of image blocks after the W segmentation is as follows: the size of each image block is set to 11×11. Through repeated adversarial training of the generator and the discriminator of the framework, the processed detail layer and base layer images are optimally fused. When the loss function reaches a dynamic balance, it indicates that the training is completed, and finally the enhanced image with the best effect is obtained, thereby achieving the purpose of enhancing the low-light image.

[0097] S105. Determine the optimal enhanced image according to the trained generative adversarial network.

[0098] In summary, the infrared image enhancement algorithm proposed in the present invention firstly adopts automatic color scale enhancement means in the source image enhancement stage, and obtains an infrared image with significantly enhanced contrast by widening the dynamic range of the grayscale value of the low-light image, thereby enhancing the brightness and clarity of the low-light image and improving the detail characteristics of the image to a certain extent. Secondly, an adaptive dimming method is used to fuse the local and global contrast of the low-light image, and a new adaptive dimming algorithm is proposed, which not only enhances the overall contrast and weak texture details of the low-light image, but also ensures the proper exposure of the overall image. In the generative adversarial training enhancement stage, the adversarial learning of the generator and the two discriminators is used to continuously refine the output enhanced image, and finally an ideal generator is obtained. The generator can be used to obtain the best synthesis scheme of the detail layer image and the base layer image, and the best enhanced image is obtained. Through the two-stage enhancement processing of the low-light image, the low-light image is effectively enhanced.

[0099] This application also proposes a low-light image enhancement system based on GANs, the system comprising: an automatic color scale enhancement unit, an adaptive dimming unit, an image processing unit and a generative adversarial network unit; The automatic color scale enhancement unit is used to enhance the original low-light image according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; Adaptive dimming unit, used to adjust low-light images according to SWGF's adaptive dimming algorithm The brightness and texture of the image are obtained to obtain an enhanced image I; An image processing unit, used for decomposing the enhanced image I into a detail layer and a base layer based on an edge-preserving decomposition model, sharpening the detail layer, and performing adaptive segmented histogram equalization processing on the base layer; The generative adversarial network unit is used to establish a generative adversarial network, train the processed detail layer and base layer, and use the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed; the optimal enhanced image is determined based on the generative adversarial network after the training.

[0100] The present application also proposes a readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps: The original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; Adjust low-light images based on SWGF's adaptive dimming algorithm The brightness and texture of the image are obtained to obtain an enhanced image I; Based on the edge-preserving decomposition model, the enhanced image I is decomposed into a detail layer and a base layer, and the detail layer is sharpened, and the base layer is subjected to adaptive segmented histogram equalization processing; A generative adversarial network is established to train the processed detail layer and base layer, and the improved SSIM loss function is used as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed. The optimal enhanced image is determined based on the trained generative adversarial network.

[0101] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present application. It should be pointed out that, for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. The above disclosure is only a preferred embodiment of the present invention, and of course it cannot be used to limit the scope of the rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present invention.

Claims

1. A low-light image enhancement method based on GANs, characterized in that: The method comprises: The original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; According to the SWGF adaptive dimming algorithm, the low-light image is adjusted The brightness and texture of the image are obtained to obtain an enhanced image I; Based on an edge-preserving decomposition model, the enhanced image I is decomposed into a detail layer and a base layer, the detail layer is sharpened, and the base layer is subjected to an adaptive segmented histogram equalization process; Establishing a generative adversarial network, training the processed detail layer and base layer, and taking the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed. The optimal enhanced image is determined based on the trained generative adversarial network.

2. The GANs-based low-light image enhancement method according to claim 1, characterized in that: The original low-light image is enhanced according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. , specifically including: Acquire three single-channel histograms of the original low-light image: a red channel histogram, a green channel histogram, and a blue channel histogram; The pixel values ​​corresponding to the lowest 5% and highest 5% of the pixel values ​​of the three single-channel histograms are counted respectively, and recorded as the minimum pixel threshold and the maximum pixel threshold ; The pixels below the minimum threshold in the three single-channel histograms are The pixel value is assigned to 0, which is higher than the maximum pixel threshold The pixel value is assigned to 225; The minimum pixel threshold in the three single channel histograms and the maximum pixel threshold Linear grayscale stretching is performed on pixel values ​​between .

3. The GANs-based low-light image enhancement method according to claim 2, characterized in that: The adaptive dimming algorithm according to SWGF adjusts the low-light image The brightness and texture of the image are obtained by obtaining an enhanced image I, which specifically includes: The low-light image Decomposed into approximate layers and residual layers: in, is the approximate layer, is the residual layer, For low-light images, is the regularization coefficient, SWGF rL Filter radius; For the approximate layer To perform dynamic compression: in, is the natural logarithm operator, is a constant, is the approximate layer, is the approximate layer after dynamic compression; Determine the variables for global illumination enhancement: in, is the light compression factor, is the first enhancement factor, is the approximate layer after dynamic compression, Variables enhanced for global illumination; The enhanced approximation layer is determined according to the following formula : in, is the variable for global light enhancement, To enhance the approximation layer; Get the second enhancement factor : in, represents the standard deviation of the image, is the approximate layer, To enhance the approximation layer, is the second enhancement factor; Determine the enhanced residual layer : in, is the second enhancement factor, is the residual layer, To strengthen the residual layer; According to the enhanced approximation layer and the enhanced residual layer Determine the enhanced image I: Where I is the enhanced image, To enhance the approximation layer, To enhance the residual layer.

4. The GANs-based low-light image enhancement method according to claim 3, characterized in that: The edge-preserving-based decomposition model decomposes the enhanced image I into a detail layer and a base layer, sharpens the detail layer, and performs adaptive segmented histogram equalization processing on the base layer, specifically including: For each enhanced image I, determine its base layer : Where I is the enhanced image, is the base layer corresponding to the enhanced image, μ is the mean filter, It is convolution processing; Determine the detail layer by subtracting the enhanced image from the base layer: in, is the detail layer of the corresponding image, I is the enhanced image, is the base layer corresponding to the enhanced image.

5. The GANs-based low-light image enhancement method according to claim 4, characterized in that: The sharpening of the detail layer specifically includes: in, To perform histogram equalization on the image, To perform Gaussian smoothing, is the detail layer image after sharpening, and 0.01 is the scaling factor is the input detail layer.

6. The GANs-based low-light image enhancement method according to claim 5, characterized in that: The performing adaptive segmented histogram equalization processing on the base layer specifically includes: Statistics of the base layer Histogram h of Divide the histogram h equally into a low grayscale interval sub-histogram, a medium grayscale interval sub-histogram and a high grayscale interval sub-histogram; Determine the cumulative distribution functions of the low grayscale interval sub-histogram, the medium grayscale interval sub-histogram, and the high grayscale interval sub-histogram respectively and perform histogram equalization; Remap each grayscale subrange back to the base layer The corresponding gray value position in the image is obtained to obtain the enhanced base layer image. .

7. The GANs-based low-light image enhancement method according to claim 6, characterized in that: The generation adversarial network is established, the processed detail layer and the base layer are trained, and the improved SSIM loss function is used as the optimization target. When the optimization target meets the threshold range, the generation adversarial network training is completed, which specifically includes: The generative adversarial network includes a generator and two discriminators; Determine the enhanced base layer image according to the generator And the sharpened detail layer image The corresponding preliminary synthetic image ; The preliminary composite image is determined by the two discriminators fusion effect.

8. The GANs-based low-light image enhancement method according to claim 7, characterized in that: The preliminary composite image is judged by two discriminators The fusion effects include: The preliminary synthesized image is determined by a loss function The fusion effect: in, , For two source images and , is the fused image, W is the number of image blocks after segmentation, represents the standard deviation, is the structural similarity value of the segmented image, is the loss function, Source Image and fused image The similarity of Source Image and fused image similarity; in, is the mean, M and N are the image sizes in the horizontal and vertical directions respectively, is the pixel value of image X at position (i, j); Among them, X and Y are the enhanced base layer images respectively. And the sharpened detail layer image , is the covariance of X and Y, and is the stability coefficient, is the similarity between X and Y, is the mean of X, is the mean of Y, for The mean of for The mean of for The standard deviation of for The standard deviation of .

9. A low-light image enhancement system based on GANs, characterized in that: The system comprises: an automatic color scale enhancement unit, an adaptive dimming unit, an image processing unit and a generative adversarial network unit; The automatic color scale enhancement unit is used to enhance the original low-light image according to the automatic color scale enhancement algorithm to obtain a low-light image with enhanced contrast. ; The adaptive dimming unit is used to adjust the low-light image according to the adaptive dimming algorithm of SWGF. The brightness and texture of the image are obtained to obtain an enhanced image I; The image processing unit is used to decompose the enhanced image I into a detail layer and a base layer based on an edge-preserving decomposition model, and to sharpen the detail layer and perform adaptive segmented histogram equalization processing on the base layer; The generative adversarial network unit is used to establish a generative adversarial network, train the processed detail layer and base layer, and use the improved SSIM loss function as the optimization target. When the optimization target meets the threshold range, the generative adversarial network training is completed; and the optimal enhanced image is determined based on the generative adversarial network after the training is completed.

10. A readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

Images