Color polarization image enhancement system and method in dark environment

Abstract

The invention belongs to the technical field of polarization image processing and display, and discloses a color polarization image enhancement system and method in a dark environment. The method comprises the following steps: firstly, inputting a color polarization intensity image in a dark environment into a preprocessing module, and carrying out preliminary brightening, denoising and color correction to obtain a preprocessed color polarization intensity image; then, the preprocessed color polarization intensity image is input into an enhancement module to be processed, and an enhanced Stokes parameter image is obtained; and finally, inputting the enhanced Stokes parameter image into a resolving module to obtain an enhanced color polarization result image. According to the invention, while the polarization information of the color polarization image is kept, the color polarization image in the dark environment is enhanced, so that the problems of insufficient brightness, low contrast ratio and fuzzy details caused by environmental reasons when the color polarization camera shoots the image in the dark environment are solved; and the identification degree and the information processing capability of the observer on the target object in the polarization image are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of polarization image processing and display, and particularly relates to a color polarization image enhancement system and method in a dim environment. Background Art

[0002] Polarization imaging technology can provide richer scene information than traditional imaging by obtaining light intensity information at different polarization angles. However, in low-light or dim environments, due to insufficient light, the quality of color polarization images is usually poor, showing problems such as low brightness, much noise, and color distortion. Existing methods are difficult to effectively enhance the brightness and contrast of color polarization images while maintaining image details, removing noise, and maintaining the integrity of polarization information. Therefore, there is an urgent need for an enhancement method that can improve the quality of color polarization images in a dim environment.

[0003] Currently, image enhancement in dim environments is generally divided into three categories. First, non-local block matching methods can effectively restore image detail information, but they cannot accurately correct color bias because they do not consider color distortion problems. Second, Retinex (retinal hypothesis)-based methods can effectively suppress noise and correct color deviation in a dark environment, but they rely on specific prior knowledge and have limited applicability. Third, deep learning-based methods can simultaneously achieve denoising and color bias correction. However, due to insufficient consideration of the differences between polarization imaging and traditional intensity imaging, it is often impossible to obtain ideal intensity, degree of polarization, and polarization angle images simultaneously after enhancement. Summary of the Invention

[0004] To overcome the problems in the related art, the disclosed embodiments of the present invention provide a color polarization image enhancement system and method in a dim environment to effectively improve the brightness of color polarization images, remove noise, correct colors, and enhance image details, making the finally obtained color polarization result image clearer, more realistic, and retaining polarization information. Tests and verifications are carried out on the public dataset PLIE, fully demonstrating the effectiveness of this method.

[0005] The present invention provides a color polarization image enhancement system in a dim environment, including:

[0006] (1) A preprocessing module: a brightening module (101), a denoising module (102), and a color correction module (103); (2) An enhancement module: an intensity image enhancement network module (201), an image recombination module (202), and a Stokes parameter image enhancement network module (203); (3) A solution module: a solution module (301).

[0007] The preprocessing module is used to preprocess the color polarization intensity image in a dim environment to improve the image quality.

[0008] The enhancement module is used to enhance the Stokes parameter image;

[0009] The solution module is used to solve the enhanced Stokes parameter image to obtain the DoLP and AoP images. The present invention also provides a method for enhancing a color polarization image in a dim environment, including the following steps:

[0010] S101: Take the color polarization intensity image I 1 in a dim environment as input and input it into the brightening module. Use the adaptive gamma correction method to adjust the brightness of the image to obtain a preliminarily brightened color polarization intensity image I 1gamma ;

[0011]

[0012] S102: Input the preliminarily brightened color polarization intensity image I 1gamma in S101 into the denoising module. Use the non-local means denoising method to denoise the image to obtain a preliminarily brightened and denoised color polarization intensity image I 1γ_denosied .

[0013]

[0014] S103: Input the preliminarily brightened and denoised color polarization intensity image I 1γ_denosied in S102 into the color correction module. Adjust the color temperature through the white balance method to eliminate color cast and obtain a preliminarily brightened, denoised and color-corrected color polarization intensity image, that is, the preprocessed color polarization intensity image I 2 .

[0015]

[0016] S201: Input the preprocessed color polarization intensity image I 2 in parallel into the intensity image enhancement network module (201) to obtain four enhanced color polarization intensity images I 2e at different angles;

[0017]

[0018] S202: Input the four enhanced color polarization intensity images I 2e at different angles in S201 into the image recombination module (202). Recombine the four images according to the calculation formula of the Stokes parameter to obtain the corresponding Stokes parameter images and

[0019] S203: Input the Stokes parameter images in S202 and into the Stokes parameter image enhancement network module (203) in parallel to obtain the enhanced Stokes parameter image I 3 , including and

[0020] S301: Solve the enhanced Stokes parameter image I 3 to obtain the enhanced color polarization result image I 4 , that is, the DoLP and AoP images.

[0021] Furthermore, in step S201, the intensity image enhancement network module (201) processes the preprocessed color polarization intensity image I 2 in parallel, uses a residual dense network to extract feature information, better captures image details, and effectively removes noise, so that the enhanced color polarization intensity image I 2e has richer details and significantly improved overall quality;

[0022] The intensity image enhancement network module (201) effectively extracts the detailed features of the image through the combination of a convolutional layer, a residual dense block, and an activation function component, retains the detailed information of the image, and avoids the detail blurring that appears in the dark and weak environment.

[0023] The image recombination module (202) recombines the four enhanced color polarization intensity images I at different angles obtained in step S201 according to the Stokes parameter calculation formula to obtain the Stokes parameter image 2e and and

[0024] The image recombination module (202) can accurately obtain the corresponding Stokes parameter image of the four color polarization intensity images at different angles through the combination of a convolutional layer, a feature fusion layer, and a Stokes parameter calculation layer.

[0025] In step S203, the Stokes parameter image enhancement network module (203) processes the Stokes parameter image and in parallel, and effectively enhances the detailed information of the image through the combination of a double-branch feature extraction network and a residual dense network, so as to obtain the enhanced Stokes parameter image I 3 , including and

[0026] The Stokes parameter image enhancement network module (203) includes a convolutional layer, Laplacian edge detection, a residual dense block, and a non-linear activation function. The non-linear activation function is used for the non-linear fitting of the network. Considering that the value of the Stokes parameter may be negative, LReLU is used as the activation function. The upper branch of the dual-branch feature extraction network is composed of convolutional layers and is responsible for extracting shallow features, while the lower branch combines Laplacian edge detection and convolutional layers to form a deep feature extraction network. The fusion of shallow features and deep features improves the comprehensiveness and accuracy of feature extraction.

[0027] It also includes: constructing a loss function between the predicted value and the true value using global content similarity, edge structure similarity, and polarization information similarity; setting a total loss function to quantify the difference between the enhanced color polarization image and the color polarization image under normal illumination, so that the enhanced color polarization image has better global content and color distribution.

[0028] By setting a global content similarity loss function, it is used to evaluate the global content similarity between the enhanced color polarization image and the color polarization image under normal illumination, making the enhanced image more realistic.

[0029] By setting an edge structure similarity loss function, it is used to retain edge structures and texture features.

[0030] By setting a polarization information similarity loss function, it is used to ensure the effectiveness of network enhancement.

[0031] By setting a total loss function for multiple objectives, different types of losses are balanced.

[0032] Aiming at the problem of enhancing color polarization images in dim environments, the present invention fully combines polarization information and deep learning methods to achieve high-quality image enhancement and has the following significant advantages: The present invention combines deep learning technology and uses a residual dense network and a dual-branch feature extraction network to effectively enhance color polarization images in dim environments. This method can fully learn the complex features and patterns of images, achieve noise removal, color cast correction, and detail enhancement, thereby improving image quality and laying a foundation for subsequent image analysis and recognition tasks. While enhancing the brightness of polarization images in dim environments, the present invention adopts a refined processing strategy to effectively retain the detail information of the images, making the enhanced images clearer.

[0033] In addition, by optimizing the enhancement strategy, the problem of blurred details caused by traditional enhancement methods is avoided, making the target more prominent and improving the recognition ability of the observer. In the process of enhancing brightness and contrast by traditional image enhancement methods, some polarization information may be lost. However, through the optimized enhancement strategy of the present invention, while enhancing brightness and visual quality, the integrity of polarization information is maintained, ensuring the accuracy of degree of linear polarization (DoLP) and angle of polarization (AoP) information, thereby improving the reliability of subsequent applications.

[0034] In summary, by comprehensively utilizing polarization information, deep learning, and refined enhancement strategies, the present invention realizes high-quality enhancement of color polarization images in dim environments. Its ultimate goal is to improve the observer's recognition ability of the target in polarization images, making the images clearer, with less noise, and more complete polarization information, thereby enhancing the accuracy and reliability of target recognition, and having broad application value in fields such as night monitoring, medical imaging, and remote sensing detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings herein are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure;

[0036] Figure 1 is a schematic diagram of the principle of the color polarization image enhancement system in a dim environment provided by an embodiment of the present invention;

[0037] Figure 2 is a schematic diagram of the structural principle of the intensity image enhancement network module (201) provided by an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of the structural principle of the Stokes parameter image enhancement network module (203) provided by an embodiment of the present invention.

[0039] In the figure: 1, preprocessing module; 101, brightening module; 102, denoising module; 103, color correction module; 2, enhancement module; 201, intensity image enhancement network module; 202, image recombination module; 203, Stokes parameter image enhancement network module; 3, calculation module; 301, calculation module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given with reference to the accompanying drawings. Many specific details are set forth in the following description to fully understand the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0041] As Figure 1 shown, a method for enhancing color polarization images in a dim environment. The color polarization image enhancement network in this dim environment is composed of a deep learning network, which includes a preprocessing module 1, an enhancement module 2, and a solution module 3. The method includes the following steps:

[0042] In this embodiment, the color polarization intensity images in the dim environment include a total of four different angles, and the present invention selects 0°, 45°, 90°, and 135°.

[0043] S101: Take the color polarization intensity image I 1 in the dim environment as the input and input it into the brightening module. Use the adaptive gamma correction method to adjust the brightness of the image to obtain the preliminarily brightened color polarization intensity image I 1gamma ;

[0044]

[0045] S102: Input the preliminarily brightened color polarization intensity image I 1gamma in S101 into the denoising module. Use the non-local means denoising method to denoise the image to obtain the preliminarily brightened and denoised color polarization intensity image I 1γ_denosied ;

[0046]

[0047] S103: Input the preliminarily brightened and denoised color polarization intensity image I 1γ_denosied in S102 into the color correction module. Adjust the color temperature through the white balance method to eliminate color cast and obtain the preliminarily brightened, denoised, and color-corrected color polarization intensity image, that is, the preprocessed color polarization intensity image I 2 ;

[0048]

[0049] The present invention combines brightening (101), denoising (102), and color correction (103) to form a preprocessing module, which can gradually optimize the image quality, ensure the integrity of polarization information, improve the effect of the subsequent enhancement network, and adapt to different dim environments. This phased optimization strategy not only improves the visual quality of the image but also enhances the accuracy and stability of target recognition, and is a key part of the entire polarization image enhancement process.

[0050] S201: Input the preprocessed color polarization intensity image I 2 in parallel into the intensity image enhancement network module (201) to obtain the enhanced color polarization intensity images I 2e at four different angles;

[0051]

[0052] S202: Input the four enhanced color polarization intensity images I at different angles in S201 2e into the image recombination module (202), recombine the four images of the intensity images according to the calculation formula of the Stokes parameters, and obtain the corresponding Stokes parameter images and

[0053] S203: Input the Stokes parameter images and in parallel into the Stokes parameter image enhancement network module (203) to obtain the enhanced Stokes parameter image I 3 , including and

[0054] S301: Solve the enhanced Stokes parameter image I 3 to obtain the enhanced color polarization result image I 4 , that is, the DoLP and AoP images

[0055] S201: Input the preprocessed color polarization intensity image I 2 in parallel into the intensity image enhancement network module (201) to obtain four enhanced color polarization intensity images I at different angles 2e ;

[0056]

[0057] S202: Input the four enhanced color polarization intensity images I at different angles in S201 2e into the image recombination module (202), recombine the four images of the intensity images according to the calculation formula of the Stokes parameters, and obtain the corresponding Stokes parameter images and

[0058] S203: Input the Stokes parameter images and in parallel into the Stokes parameter image enhancement network module (203) to obtain the enhanced Stokes parameter image I 3 , including and

[0059] As Figure 2 shown, it is a schematic structural diagram of the intensity image enhancement network module 201. The specific steps of this process are as follows:

[0060] Input the preprocessed color polarization intensity images at 0°, 45°, 90°, and 135° in step S103 into the intensity image enhancement network module 201 to generate and obtain the enhanced color polarization intensity images I at four different angles 2e , In this embodiment, the intensity image enhancement network module 201 of the present invention consists of a depthwise separable module, an inverse recovery module, and a residual dense block to form a residual dense network, which retains more detailed information of the image and learns the subtle differences of the color polarization image in a dim environment. Among them, the depthwise separable convolution module specifically includes a depthwise separable convolution layer, a batch normalization layer, and an activation function, and the inverse recovery module specifically includes a transposed convolution layer. In this instance of the intensity image enhancement network module 201, a total of 3 four-layer residual dense blocks are configured, combined with 3 depthwise separable convolution modules and 3 inverse recovery modules, which can improve the calculation efficiency while retaining the detailed information, thereby achieving a better enhancement effect;

[0061] As Figure 3 shown, it is a schematic structural diagram of the Stokes parameter image enhancement network module 203. The specific steps of this process are as follows:

[0062] Input the 3 Stokes parameter images in step S202 and into the Stokes parameter image enhancement network module 203 to obtain the enhanced Stokes parameter image I 3 , including and In this embodiment, the Stokes parameter image enhancement network module 203 of the present invention includes a depthwise separable convolution module, an inverse recovery module, a residual dense block, a Laplacian edge detection, and a non-linear activation function. Among them, the depthwise separable convolution module specifically includes a depthwise separable convolution layer, a batch normalization layer, and an activation function, and the inverse recovery module specifically includes a transposed convolution layer. 3 depthwise separable convolution modules form the upper branch network of the double-branch feature extraction network, and the Laplacian edge detection and 3 depthwise separable convolution modules form the lower branch network of the double-branch feature extraction network. In the embodiment of the present invention, 4 six-layer residual dense blocks, 3 inverse recovery modules, and a ReLu non-linear activation function are configured. Through the effective combination of the double-branch feature extraction network and the residual dense block network, the detailed information of the image can be significantly enhanced and the image quality can be improved.

[0063] Setting of the loss function of the present invention:

[0064] Global content similarity loss L con :

[0065]

[0066] Among them, is the Stokes parameter image in a dim environment; is the enhanced Stokes parameter image, where n ∈ {0, 1, 2}; is the feature extraction function; N is the number of dimensions of the feature representation.

[0067] The edge structure similarity loss L edge :

[0068]

[0069] Among them, represents the gradient operation.

[0070] The polarization information similarity loss L pol :

[0071]

[0072] Among them, θ represents the calculation of the polarization angle.

[0073] Total loss function:

[0074] L total = λ 0 L con + λ 1 L edge + λ 2 L pol (3)

[0075] Among them, λ 0 , λ 1 , λ 2 are respectively hyperparameters in the network, λ 0 = 1, λ 1 = 0.4, λ 2 = 0.4.

[0076] The purpose of the content loss is to quantify the difference between the enhanced color polarization image and the color polarization image under normal brightness, so as to help the network better retain the content and detail information of the image. Through this loss, the network can effectively retain key image features, such as edges, textures, and structures, in a dim environment, ensuring that the generated image is consistent with the original image in important visual elements;

[0077] The purpose of the edge structure similarity loss is to ensure that the enhanced color polarization image can retain an edge structure similar to that of the color polarization image under normal brightness, thereby improving the fidelity of the detail and contour information in the image.

[0078] The purpose of the polarization information similarity loss is to quantify the difference in polarization information between the enhanced color polarization image and the color polarization image under normal brightness, ensuring that the generated image can effectively retain and transmit the polarization characteristics of the original image. Through this loss, the network can maintain the consistency of the key polarization information, i.e., the angle of polarization (AoP), as much as possible during image generation or enhancement, thereby improving the performance of the image in low-light environments and enhancing the image's recognizability and detail retention.

[0079] In steps S202 and S301 of this embodiment, in this embodiment, the enhancement result is subjected to Stokes calculation to obtain the Stokes parameter image after calculation. Degree of polarization image and angle of polarization image

[0080] The specific calculation process is shown in formulas (5) and (6):

[0081]

[0082]

[0083] To further illustrate the related effects of the embodiments of the present invention, the following experiments are carried out: The present invention selects the publicly available dataset PLIE as the training and test set to verify the performance of the present invention. The enhancement effect of the present invention is compared with the enhancement algorithms based on deep learning in recent years, and a quantitative evaluation is carried out. Using the PSNR, SSIM, and PCQI evaluation metrics, 50 groups of color polarization images in low-light environments are used for testing, and the average value of the test results is taken. The quantitative evaluation results are shown in Table 1; The experimental results show that the method for enhancing color polarization images in low-light environments of the present invention obtains the optimal values for the PSNR and SSIM values of the enhanced color polarization S0 and DoLP images, and most of the indicators are better than other algorithms.

[0084] Table 1 Comparison results of different algorithms

[0085]

[0086] The above is only a relatively preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention should be covered within the protection scope of the present invention.

Claims

1. A color polarization image enhancement system in a dim environment, characterized in that: include: (1) Preprocessing module: This module is composed of a brightening module (101), a denoising module (102) and a color correction module (103), and is used to obtain a preprocessed color polarization intensity image. (2) Enhancement module: This module is composed of an intensity image enhancement network module (201), an image reorganization module (202) and a Stokes parameter image enhancement network module (203), and is used to obtain an enhanced Stokes parameter image. (3) Solving module: The solving module (301) is used to solve the enhanced Stokes parameter image to obtain enhanced color polarization result images, namely DoLP and AoP images.

2. A color polarization image enhancement method in a dim environment, characterized in that: The steps include: S101: Taking the color polarization intensity image I1 in a dim environment as input, inputting it into the brightening module, and adjusting the brightness of the image using an adaptive gamma correction method to obtain a preliminarily brightened color polarization intensity image I 1gamma ; S102: The color polarization intensity image I preliminarily brightened in S101 is 1gamma Input to the denoising module, use the non-local mean denoising method to denoise the image, and obtain the color polarization intensity image I after preliminary brightening and denoising 1γ_denosied . S103: The color polarization intensity image I after preliminary brightening and denoising in S102 is 1γ_denosied The image is input to the color correction module, and the color temperature is adjusted by the white balance method to eliminate the color cast, so as to obtain the color polarization intensity image after preliminary brightening, denoising and color correction, that is, the preprocessed color polarization intensity image I2. The color polarization intensity image in a dim environment includes four different angles, and the present invention selects 0°, 45°, 90° and 135°. S201: Inputting the preprocessed color polarization intensity image I2 in parallel to the intensity image enhancement network module (201) to obtain enhanced color polarization intensity images I2 at four different angles. 2e ; S202: The four color polarization intensity images I at different angles after enhancement in S201 are 2e The image is input to the image recombining module (202), and the four images are recombined according to the calculation formula of the Stokes parameter to obtain the corresponding Stokes parameter image. and S203: The Stokes parameter image in S202 and The image is input in parallel to the Stokes parameter image enhancement network module (203) to obtain the enhanced Stokes parameter image I3, including and S301: Solve the enhanced Stokes parameter image I3 to obtain the enhanced color polarization result image I4, that is, the DoLP and AoP images. The formula is as follows:

3. The color polarization image enhancement method in a dim environment according to claim 2, characterized in that: In step S201, the intensity image enhancement network module (201) processes the preprocessed color polarization intensity image I2 in parallel, extracts feature information using a residual dense network, better captures image details, and effectively removes noise; in step S203, the Stokes parameter image enhancement network module (203) processes the Stokes parameter image I2 in parallel, extracts feature information using a residual dense network, better captures image details, and effectively removes noise; and Parallel processing, through the combination of dual-branch feature extraction network and residual dense network, effectively enhances the detail information of the image. The intensity image enhancement network module (201) effectively extracts the detail features of the image through the combination of the convolution layer, the residual dense block and the activation function component, retains the detail information of the image, and avoids the blurring of details in a dim environment; The image reconstructing module (202) can accurately obtain Stokes parameter images corresponding to four color polarization intensity images at different angles by combining a convolution layer, a feature fusion layer and a Stokes parameter calculation layer; The Stokes parameter image enhancement network module (203) includes a convolution layer, a Laplace edge detection, a residual dense block and a nonlinear activation function. The nonlinear activation function is used for nonlinear fitting of the network. Considering that the value of the Stokes parameter may be negative, LReLU is used as the activation function.

4. The color polarization image enhancement method in a dim environment according to claim 2 is characterized in that , also includes: constructing a loss function between a predicted value and a true value by using global content similarity, edge structure similarity and polarization information similarity; setting a total loss function for quantifying the difference between the enhanced color polarization image and the color polarization image under normal illumination, so that the enhanced color polarization image has better global content and color distribution; By setting a global content similarity loss function, the global content similarity between the enhanced color polarization image and the color polarization image under normal illumination is evaluated, so that the enhanced image is more realistic. By setting the edge structure similarity loss function, it is used to preserve the edge structure and texture features; By setting the polarization information similarity loss function, the effectiveness of network enhancement is ensured; By setting the total loss function to be used for multiple objectives, different types of losses can be balanced.

Images