Three-channel low-illumination image enhancement method based on polarization guiding

Abstract

The application discloses a three-channel low-illumination image enhancement method based on polarization guidance, specifically comprising the following steps: synchronously collecting a gray image, a color original image and a polarization feature image corresponding to the same image and preprocessing; calculating a gray channel adaptive parameter and a color brightness channel adaptive parameter; performing double-channel adaptive stochastic resonance processing on the preprocessed gray image and the color original image according to the gray channel adaptive parameter and the color brightness channel adaptive parameter respectively, to obtain an enhanced gray image and a color image; and fusing the enhanced gray image and the color image and the polarization feature image to obtain a fused image. The application solves the problems in the prior art that the smooth light reflection area is prone to producing halo artifacts and edge saturation after enhancement, the details of the diffuse reflection dark area are insufficiently recovered, and the brightness enhancement, color fidelity and edge quality are difficult to be simultaneously improved due to the fact that the polarization physical information is not effectively utilized.

Description

Technical Field

[0001] This invention belongs to the technical field of low-light imaging and multimodal image processing methods, and relates to a three-channel low-light image enhancement method based on polarization guidance. Background Technology

[0002] In low-light scenarios such as nighttime security monitoring, target recognition, and material analysis, it is necessary to enhance the acquired low-light images. Traditional low-light imaging mainly uses single-channel imaging, but single-channel imaging has inherent limitations, as follows: (1) Grayscale images retain brightness details but lack color, and are prone to blurring due to noise in low light; (2) Color images contain color information but are affected by filters, resulting in insufficient light throughput. The signal-to-noise ratio (SNR) drops sharply under low illumination, leading to color distortion and noise spots. (3) Polarization images can reflect the physical properties of an object's surface (such as the polarization difference between metal / non-metal and rough / smooth surfaces), but when imaged alone, the signal is weak and lacks intuitive visual information. Furthermore, existing technologies have not effectively utilized their physical properties to enhance the image.

[0003] While some multi-channel fusion techniques exist in the current technology, they suffer from two major bottlenecks: First, they ignore the physical meaning of polarization information, treating polarized images merely as ordinary visual features for fusion, failing to consider the degree of polarization. Distinguishing between smooth reflective surfaces and rough diffuse reflective surfaces leads to high... Areas (such as smooth surfaces like metal and glass) are prone to halo artifacts, reflection diffusion, and edge saturation after enhancement. First, the recovery of faint details in dark areas (such as walls, fabrics, and other diffuse reflective surfaces) is insufficient, resulting in limited improvement in overall contrast and signal-to-noise ratio in dark areas. Second, the random resonance parameters are fixed or rely solely on grayscale statistics, failing to adapt to the signal differences between grayscale channels, color brightness channels, and polarization channels. This can easily lead to insufficient brightness enhancement, color drift, and artifacts at the fusion edges, ultimately making it difficult to simultaneously achieve brightness recovery, detail enhancement, color fidelity, and reflection suppression in low-light images. Summary of the Invention

[0004] The purpose of this invention is to provide a three-channel low-light image enhancement method based on polarization guidance, which solves the problems in the prior art where the lack of effective use of polarization physical information leads to halo artifacts and edge saturation after enhancement of smooth reflective areas, insufficient recovery of details in diffuse dark areas, and difficulty in synergistically improving brightness enhancement, color fidelity and edge quality.

[0005] The technical solution adopted in this invention is a three-channel low-light image enhancement method based on polarization guidance, which is implemented according to the following steps: Step 1: Simultaneously acquire the grayscale image, the original color image, and the polarization feature map corresponding to the same image through three channels; Step 2: Preprocess the grayscale image, original color image, and polarization feature map acquired in Step 1; Step 3: Calculate the adaptive parameters for the grayscale channel and the adaptive parameters for the color brightness channel; Step 4: Perform dual-channel adaptive stochastic resonance processing on the preprocessed grayscale image and the original color image according to the grayscale channel adaptive parameters and the color brightness channel adaptive parameters, respectively, to obtain the enhanced grayscale image and color image. Step 5: Fuse the enhanced grayscale image, color image, and polarization feature map to obtain a fused image.

[0006] Preferably, when acquiring polarization feature maps, a micro-polarization array camera is used. The micro-polarization array camera is equipped with pixel-level micro-polarizers, integrating four-directional polarization filter units (0°, 45°, 90°, and 135°) on the sensor pixel array. Four polarization components are acquired simultaneously with a minimum repeating unit of 2×2 pixels, achieving a resolution of 1224×1024 and a pixel size of 3.45μm. The output includes polarization intensity maps of 0°, 45°, 90°, and 135°. , , , Calculate the Stokes parameters based on the four-directional polarization intensity diagram:

[0007] Calculate the degree of polarization based on Stokes parameters : ; in, This represents the degree of polarization at pixel (x, y). For Stokes parameters; Calculate the polarization angle :

[0008] in, The polarization angle at pixel (x, y); Generate polarization feature map Polarization feature map any point in the middle The pixel value at that location is: .

[0009] Preferably, the preprocessing of the acquired grayscale image in step 2 specifically includes: For the acquired grayscale image BM3D denoising is performed as follows: grayscale image Divided into Pixel blocks, in Search window by Euclidean distance (threshold) The process involves filtering similar blocks, stacking them into a 3D matrix, applying wavelet transform layer by layer, and performing coefficient shrinkage through hard thresholding in the transform domain. Finally, the wavelet-transformed 3D matrix is ​​inversely transformed and weighted fused to output a denoised image. ; Denoising image Normalization is performed: Will Pixel values ​​mapped to The interval is used to obtain the normalized grayscale image. :

[0010] in, For denoised image Centered at pixel Pixel value at that location, grayscale image after normalization Located at pixel Pixel value at; Step 2 involves processing the acquired original color images. The preprocessing process specifically involves: Step 2.1, process the original color image To perform de-mosaic processing, specifically: Restored to RGB image using bilinear interpolation ; Step 2.2, will Perform color space conversion, specifically: Will Convert to YUV color space and separate the luminance component. Blue chromaticity component and red chromaticity component :

[0011] Where R, G, and B are the R channel values, G channel values, and B channel values ​​of the RGB image, respectively. Step 2.3, for the luminance component Nonlocal mean filtering is used for the blue chromaticity component. and red chromaticity component use Gaussian filtering yields , ; Step 2 involves preprocessing the polarization feature map as follows: Degree of polarization of polarization feature map and polarization angle Median filtering is performed for noise reduction, specifically, the polarization feature map is... and Each applied Median filtering; polarization degree after median filtering and polarization angle Outlier correction is performed as follows: right The pixels are forced to be set to ; right The pixels are forced to be set to ; Stokes parameters Pixels are marked as invalid polarization regions; Will Normalization to .

[0012] Preferably, step 3 specifically includes: Step 3.1: Extract the normalized grayscale images respectively. Compared with the luminance component after non-local mean filtering Local features; Step 3.2: Obtain the adaptive parameters for the grayscale channel and the adaptive parameters for the color channel based on local features.

[0013] Preferably, step 3.1 specifically involves: defining pixels. Centered sliding window Calculate the grayscale image within the sliding window respectively. Compared with the luminance component after non-local mean filtering Local brightness variance Texture complexity Local signal-to-noise ratio and grayscale images Significance of polarization structure ; Among them, the luminance variance The formula is as follows:

[0014] in, or , For the corresponding sliding window Any pixel within, grayscale image In the sliding window any pixel within pixel value, Luminance component In the sliding window any pixel within Pixel brightness value at that location, This represents the average brightness within the corresponding window;

[0015] Where N is the sliding window The total number of pixels within; Texture complexity The formula is as follows:

[0016] in,

[0017] in, , Sliding window for the corresponding image Inner pixel The directional gradient in the x and y directions, when calculating a grayscale image. Texture complexity hour, , Grayscale images sliding window Inner pixel The directional gradient in the x and y directions; when calculating the brightness component. Texture complexity hour, , These are the luminance components. sliding window Inner pixel The directional gradient in the x and y directions; Local signal-to-noise ratio The formula is as follows:

[0018] in, It is the minimum value; The unit is dB; Significance of polarization structure The formula is as follows:

[0019] in, for Structure mask image At pixel Pixel value at the location, pixel Degree of polarization at that location; Structure mask image Obtain it in the following way: right Binarized edge images are generated using Canny edge detection. And then Perform morphological dilation to generate a structure mask image. .

[0020] Preferably, step 3.2 specifically includes: Based on grayscale image Local brightness variance Texture complexity Local signal-to-noise ratio and polarization structure salience Calculate the adaptive parameters of the grayscale channels and :

[0021]

[0022] in, , , ; Based on the luminance components after nonlocal mean filtering Texture complexity Local signal-to-noise ratio Calculate adaptive parameters for color luminance channels and :

[0023]

[0024] in, , , .

[0025] Preferably, step 4, which involves adaptive stochastic resonance processing of the preprocessed grayscale image based on adaptive grayscale channel parameters, specifically includes: (1) Construct an adaptive bistable stochastic resonance model, specifically as follows:

[0026] in, grayscale image after normalization , , It is Gaussian white noise that follows a normal distribution. , For time step; in, grayscale image Noise intensity:

[0027] in, As the reference noise intensity, , , , ; grayscale image corresponding and as well as Substituting into the above equation, we get ; (2) The adaptive bistable stochastic resonance model is solved iteratively using the improved Euler method formula. The iterative formula is as follows:

[0028]

[0029]

[0030] in, For time step; Termination condition: 3 consecutive times Or reach the maximum number of iterations, 20; (3) Map the output results that meet the termination conditions back to The enhanced grayscale image is obtained. .

[0031] Preferably, step 4, which involves adaptive stochastic resonance processing of the preprocessed original color image based on adaptive parameters of the color luminance channel, specifically involves: (1) Construct an adaptive bistable stochastic resonance model, specifically as follows:

[0032] in, The luminance component after nonlocal mean filtering. The result of the normalization process is: , It is Gaussian white noise that follows a normal distribution. , For time step; in, The luminance component after non-local mean filtering Noise intensity:

[0033] in, As the reference noise intensity, , , , ; The luminance component after nonlocal mean filtering corresponding and as well as Substituting into the above equation, we get ; (2) The adaptive bistable stochastic resonance model is solved iteratively using the improved Euler method formula. The iterative formula is as follows:

[0034]

[0035]

[0036] in, For time step; Termination condition: 3 consecutive times Or reach the maximum number of iterations, 15; (3) Map the output results that meet the termination conditions back to The enhanced luminance component is obtained. ; (4) Use the enhanced luminance component And the Gaussian filter obtained in step 2 , Perform a YUV inverse matrix conversion back to RGB space to obtain the enhanced color image. .

[0037] Preferably, step 5 specifically includes: For regions outside the invalid polarization region: right , , Three-channel weighted fusion is performed, and the fusion formula is as follows:

[0038] in, for Image at any pixel pixel values, for Image at any pixel pixel values, Polarization feature map At any pixel Pixel values; For grayscale channel weights, For color channel weights, Polarization channel weights; , , by With the core as the main focus, specifically: For regions outside the invalid polarization region:

[0039]

[0040]

[0041] For invalid polarization regions, combine invalid polarization region mask. The fusion method is determined as follows: when When the polarization information is valid, three-channel fusion is used, which means that the fusion method is the same as that used for other regions outside the invalid polarization region. when When the polarization information is invalid, dual-channel fusion is used, and the fusion formula is:

[0042] in:

[0043]

[0044] Map the fusion result to Rounding down yields the fused image. .

[0045] Preferably, it also includes the fused image Post-processing is performed, specifically as follows: Computational fusion of images Global average brightness L:

[0046] in, To merge the image height, i.e., the number of image rows; To merge the image width, i.e. the number of image columns, Represents fused images At pixel Pixel value at; Based on global average brightness For fused images Dynamic adjustment Values ​​are used to obtain the corrected image. Specifically:

[0047]

[0048] in, To correct the image At any pixel Pixel value at; Calculate the corrected image Sobel edge intensity map : ; in, Edge intensity map At pixel Edge strength value at the location, , Correcting images Structure mask image Gradient in the x and y directions; The final enhanced image for:

[0049] in, To enhance the image At pixel Pixel value at; For structure mask image At pixel The pixel value at the location.

[0050] The beneficial effects of this invention are: This invention is achieved through Guided weight allocation and random resonance parameter modulation enable high Smooth reflective regions reduce random resonance noise intensity and improve system stability, thereby suppressing halo artifacts, reflection diffusion, and edge saturation in low-light enhancement; making low-light enhancement more effective. The diffuse reflection region is enhanced with random resonance excitation to restore faint textures and contour details in dark areas, thereby improving the signal-to-noise ratio and overall contrast in dark areas. At the same time, in the polarization-invalid region, it automatically degenerates into grayscale and color dual-channel fusion to avoid fusion anomalies caused by unreliable polarization signals, thus achieving a synergistic improvement in brightness recovery, detail enhancement, color fidelity, and reflection suppression in low-light images.

[0051] In this invention, the grayscale channel parameters are simultaneously constrained by SNR, texture, and polarization, while the color channel parameters are modulated by polarization nonlinear gain, resulting in a synergistic improvement in both PSNR and SSIM of the image under low illumination. The post-processing sharpening of this invention only applies to the structural mask area, which effectively suppresses noise amplification in smooth areas while enhancing edge clarity, resulting in overall image quality superior to uniform sharpening schemes across the entire image. Attached Figure Description

[0052] Figure 1 This is a flowchart of the three-channel low-light image enhancement method based on polarization guidance of the present invention; Figure 2 This is the optical path diagram for image acquisition in the three-channel low-light image enhancement method based on polarization guidance of this invention. Detailed Implementation

[0053] The following detailed description is provided in conjunction with specific implementation methods.

[0054] This invention relates to a three-channel low-light image enhancement method based on polarization guidance, the process of which is as follows: Figure 1 As shown, the specific steps are as follows: Step 1: Simultaneously acquire the grayscale image, the original color image, and the polarization feature map corresponding to the same image through three channels; Step 2: Preprocess the grayscale image, original color image, and polarization feature map acquired in Step 1; Step 3: Calculate the adaptive parameters for the grayscale channel and the adaptive parameters for the color brightness channel; Step 4: Perform dual-channel adaptive stochastic resonance processing on the preprocessed grayscale image and the original color image according to the grayscale channel adaptive parameters and the color brightness channel adaptive parameters, respectively, to obtain the enhanced grayscale image and color image. Step 5: Fuse the enhanced grayscale image, color image, and polarization feature map to obtain a fused image.

[0055] Example 2 Based on Example 1, when acquiring images, the acquisition path is as follows: Figure 2 As shown, a three-channel imaging system is constructed as follows: Grayscale channel: Back-illuminated CMOS sensor (resolution 1280×720, pixel size 5μm, quantum efficiency ≥75%@550nm, dynamic range ≥100dB), filterless design to receive full-spectrum light from 400-900nm, exposure time 0.1-10ms (adaptively adjusted according to scene lighting), outputting grayscale images. .

[0056] Color channel: Bayer array CMOS sensor (resolution 1280×720, pixel size 5μm), equipped with an infrared cutoff filter (transmission band 400-700nm, cutoff depth OD4), outputting raw color images. .

[0057] Polarization Channel: When acquiring polarization feature maps, a micro-polarization array camera is used. The micro-polarization array camera is equipped with pixel-level micro-polarizers, integrating four-directional polarization filter units (0°, 45°, 90°, and 135°) on the sensor pixel array. Four polarization components are acquired simultaneously with a minimum repeating unit of 2×2 pixels, achieving a resolution of 1224×1024 and a pixel size of 3.45μm. The output includes polarization intensity maps in four directions: 0°, 45°, 90°, and 135°. , , , Calculate the Stokes parameters based on the four-directional polarization intensity diagram:

[0058] Calculate the degree of polarization based on Stokes parameters (Degree of Linear Polarization): ; Among them, polarization degree A higher value indicates a smoother surface (such as metal or glass), while a lower value indicates a rougher surface (such as fabric or concrete). This represents the degree of polarization at pixel (x, y). For Stokes parameters; Calculate the polarization angle :

[0059] in, The polarization angle at pixel (x, y); Using the four-quadrant arctangent function To handle correctly Quadrant ambiguity, ordinary exist The result deviation was 90°. Generate polarization feature map Polarization feature map any point in the middle The pixel value at that location is: .

[0060] The synchronization and optical path design in this embodiment is as follows: Optical path multiplexing: A two-stage beam splitter is used (the first stage has a beam splitting ratio of 4:6, with 40% of the light intensity allocated to the grayscale channel; the second stage allocates the remaining 60% to the color channel and polarization channel in a 1:1 ratio, i.e., 30% each). The prism material is N-BK7 optical glass, coated with a broadband anti-reflection film (transmittance ≥98% / surface) to ensure sufficient light in each channel. Synchronization control: The FPGA (Field Programmable Gate Array) generates a 50Hz synchronization trigger signal, which is connected to the three external trigger interfaces of the sensors via coaxial cables. The exposure time error of the three channels is ≤10μs. The three channels are geometrically calibrated by a calibration board (11×8 checkerboard, 20mm grid). The spatial alignment deviation is ≤1 pixel. The four-directional pixels of the polarization channel are already synchronized by the hardware microarray, and no additional inter-frame synchronization is required.

[0061] Example 3 Based on Example 2, the preprocessing of the acquired grayscale image in step 2 specifically includes: For the acquired grayscale image BM3D denoising is performed as follows: grayscale image Divided into Pixel blocks, in Search window by Euclidean distance (threshold) The process involves filtering similar blocks, stacking them into a 3D matrix, and then applying wavelet transform (sym4 wavelet, 3-level decomposition) to the 3D matrix layer by layer. Coefficients are then shrunk in the transform domain using hard thresholding.

[0062] in, The noise standard deviation is estimated from the mean of the standard deviations of the vignetting regions (four 16×16 corner blocks) in the image. Indicates the first The positions of similar blocks in the three-dimensional transform domain Transformation coefficients at; Indicates to Transform coefficients after hard thresholding; For indicator functions, when The value is 1 if the condition is met, otherwise it is 0. Hard threshold; The noise standard deviation can be determined from the four corners of the image. Estimation of the standard deviation and mean of the dark corner block. This is a hard threshold.

[0063] The 3D matrix after wavelet transform is inversely transformed and then weighted and fused to output a denoised image. ; Denoising image Normalization is performed: Will Pixel values ​​mapped to The interval is used to obtain the normalized grayscale image. :

[0064] in, For denoised image Centered at pixel Pixel value at that location, grayscale image after normalization Located at pixel Pixel value at; Step 2 involves processing the acquired original color images. The preprocessing process specifically involves: Step 2.1, process the original color image To perform de-mosaic processing, specifically: Restored to RGB image using bilinear interpolation ; Step 2.2, will Perform color space conversion, specifically: Will Convert to YUV color space and separate the luminance component. Blue chromaticity component and red chromaticity component :

[0065] Where R, G, and B are the R channel values, G channel values, and B channel values ​​of the RGB image, respectively. Step 2.3, for the luminance component Nonlocal mean filtering (search window) block size Similarity bandwidth , for (Channel noise standard deviation) to preserve luminance details; for the blue chromaticity component and red chromaticity component use Gaussian filtering (standard deviation 0.8) yields , Smooth chromatic noise; Step 2 involves preprocessing the polarization feature map as follows: Degree of polarization of polarization feature map and polarization angle Median filtering is performed to remove the salt-and-pepper pulse noise specific to polarization detection. Specifically, this involves processing the polarization feature map... and Each applied Median filtering; polarization degree after median filtering and polarization angle Outlier correction is performed as follows: right The pixels (caused by strong noise) are forced to be set to ; right The pixel points (calculation error) are forced to be set to ; Stokes parameters Pixels are marked as invalid polarization regions; Will Normalization to .

[0066] Example 4 Based on Example 3, step 3 specifically includes: Step 3.1: Extract the normalized grayscale images respectively. Compared with the luminance component after non-local mean filtering Local features; Step 3.2: Obtain the adaptive parameters for the grayscale channel and the adaptive parameters for the color channel based on local features.

[0067] Step 3.1 specifically involves defining pixels. Centered sliding window Calculate the grayscale image within the sliding window respectively. Compared with the luminance component after non-local mean filtering Local brightness variance Texture complexity Local signal-to-noise ratio and grayscale images Significance of polarization structure ; Among them, the luminance variance The formula is as follows:

[0068] in, or , For the corresponding sliding window Any pixel within, grayscale image In the sliding window any pixel within pixel value, Luminance component In the sliding window any pixel within Pixel brightness value at that location, This represents the average brightness within the corresponding window;

[0069] Where N is the sliding window The total number of pixels within; Texture complexity The formula is as follows:

[0070] in,

[0071] in, , Sliding window for the corresponding image Inner pixel The directional gradient in the x and y directions, when calculating a grayscale image. Texture complexity hour, , Grayscale images sliding window Inner pixel The directional gradient in the x and y directions; when calculating the brightness component. Texture complexity hour, , These are the luminance components. sliding window Inner pixel The directional gradient in the x and y directions; Local signal-to-noise ratio The formula is as follows:

[0072] in, It is the minimum value; The unit is dB; Significance of polarization structure The formula is as follows:

[0073] The saliency of polarization structure is combined with the degree of polarization and the structure mask. Regions with high saliency values ​​are smooth surface edges (such as metal frames). in, for Structure mask image At pixel Pixel value at the location, For pixels Degree of polarization at that location; Structure mask image Obtain it in the following way: right Binarized edge images are generated using Canny edge detection. (Dual threshold) ), and then to Perform morphological dilation to generate a structure mask image. ; This indicates the grayscale normalized image. The maximum gradient magnitude of the entire image obtained after gradient calculation; Gray-scale normalized image At pixel gradient magnitude for:

[0074] in, and They are respectively exist direction and Gradient of direction; but:

[0075] The lower threshold in Canny's dual thresholding and high threshold They are respectively:

[0076] Gradient magnitude greater than The pixels are marked as strong edges, between and Pixels connected to strong edges are marked as weak edges, and the remaining pixels are the background. After hysteresis thresholding, a binary edge map is obtained. And then Morphological dilation is performed to obtain the structural mask. .

[0077] Step 3.2 specifically involves: Based on grayscale image Local brightness variance Texture complexity Local signal-to-noise ratio and polarization structure salience Calculate the adaptive parameters of the grayscale channels and :

[0078]

[0079] in, , , ; Physical meaning: Increase in the low SNR region (weak signal) To trigger stochastic resonance gain; the high DOP region (smooth metal surface) simultaneously reduces (Avoid over-enhancement) and increase (Improve system stability); Based on the luminance components after nonlocal mean filtering Texture complexity Local signal-to-noise ratio Calculate adaptive parameters for color luminance channels and : ( )

[0080] in, , , .

[0081] Physical significance: High DOP areas (smooth surfaces) reduce This reduces the amplitude of nonlinear enhancement and protects color stability.

[0082] Example 5 Based on Example 4, step 4, which involves adaptive stochastic resonance processing of the preprocessed grayscale image according to the adaptive parameters of the grayscale channels, specifically involves: (1) Construct an adaptive bistable stochastic resonance model, specifically as follows:

[0083] in, grayscale image after normalization , , It is Gaussian white noise that follows a normal distribution. , For time step; in, grayscale image Noise intensity:

[0084] in, As the reference noise intensity, , , , ; grayscale image corresponding and as well as Substituting into the above equation, we get ; (2) The adaptive bistable stochastic resonance model is solved iteratively using the improved Euler method formula. The iterative formula is as follows:

[0085]

[0086]

[0087] in, For time step; Termination condition: 3 consecutive times Or reach the maximum number of iterations, 20; (3) Map the output results that meet the termination conditions back to The enhanced grayscale image is obtained. .

[0088] Step 4 involves adaptive stochastic resonance processing of the preprocessed original color image based on adaptive parameters for the color luminance channel. (1) Construct an adaptive bistable stochastic resonance model, specifically as follows:

[0089] in, The luminance component after nonlocal mean filtering. The result of the normalization process is: , It is Gaussian white noise that follows a normal distribution. , For time step; in, The luminance component after non-local mean filtering Noise intensity:

[0090] in, As the reference noise intensity, , , , ; The luminance component after nonlocal mean filtering corresponding and as well as Substituting into the above equation, we get ; (2) The adaptive bistable stochastic resonance model is solved iteratively using the improved Euler method formula. The iterative formula is as follows:

[0091]

[0092]

[0093] in, For time step; Termination condition: 3 consecutive times Or reach the maximum number of iterations, 15; (3) Map the output results that meet the termination conditions back to The enhanced luminance component is obtained. ; (4) Use the enhanced luminance component And the Gaussian filter obtained in step 2 , Perform a YUV inverse matrix conversion back to RGB space to obtain the enhanced color image. .

[0094] Polarization information represents a physical quantity (not a visual signal), and without random resonance processing, it is directly used as the preprocessed signal. Figures and Participate in integration.

[0095] Example 6 Based on Example 5, step 5 specifically includes: For regions outside the invalid polarization region: right , , Three-channel weighted fusion is performed, and the fusion formula is as follows:

[0096] in, for Image at any pixel pixel values, for Image at any pixel pixel values, Polarization feature map At any pixel Pixel values; For grayscale channel weights, For color channel weights, Polarization channel weights; , , by With the core as the main focus, specifically: For regions outside the invalid polarization region:

[0097]

[0098]

[0099] Weight normalization verification: (Hengchengli, and) (Values ​​are irrelevant) Value range: hour ; hour ; For invalid polarization regions, combine invalid polarization region mask. The fusion method is determined as follows: when When the polarization information is valid, three-channel fusion is used, which means that the fusion method is the same as that used for other regions outside the invalid polarization region. when When the polarization information is invalid, dual-channel fusion is used, and the fusion formula is:

[0100] in:

[0101]

[0102] Map the fusion result to Rounding down yields the fused image. .

[0103] Physical meaning: High → Smooth surface → Strong ability to distinguish polarization information materials → high; Low → Surface roughness → Visual information is more critical → and high.

[0104] Example 7 Based on Example 6, the present invention also includes the fused image Post-processing is performed, specifically as follows: Computational fusion of images Global average brightness L:

[0105] in, To merge the image height, i.e., the number of image rows; To merge the image width, i.e. the number of image columns, Represents fused images At pixel Pixel value at; Based on global average brightness For fused images Dynamic adjustment Values ​​are used to obtain the corrected image. Specifically:

[0106]

[0107] When the index is less than 1, it has a brightening effect on low-brightness pixels; in, To correct the image At any pixel Pixel value at; Calculate the corrected image Sobel edge intensity map : ; in, Edge intensity map At pixel Edge strength value at the location, , Correcting images Structure mask image Gradient in the x and y directions; The final enhanced image for:

[0108] in, To enhance the image At pixel Pixel value at; For structure mask image At pixel The pixel value at the specified location is truncated to the sharpened result. Output the final enhanced image .

[0109] Example 8 Based on Example 7, the hardware setup for implementing this invention is as follows: Grayscale / color sensor: Back-illuminated CMOS (resolution 1280×1024, frame rate ≥60fps, pixel size ≤5μm), grayscale sensor without filter, color sensor with Bayer filter; lens focal length 16mm, aperture F / 1.4, effective incident aperture 11.4mm. Polarization camera: Equipped with a CMOS sensor with a pixel-level micro-polarization array (resolution ≥2448×2048, pixel size ≤3.45μm, integrating 0° / 45° / 90° / 135° four-directional polarization unit), and paired with a 16mm C-mount lens; Optical path integration: Two-stage N-BK7 optical glass prisms (first stage with a splitting ratio of 40:60, second stage with a splitting ratio of 50:50), each surface is coated with a broadband anti-reflection film, the overall size is 30mm×30mm×50mm, and the three sensors are arranged in parallel by an aluminum alloy mounting bracket. Synchronization control: The FPGA programmable logic device generates a 50Hz TTL synchronization signal, which is connected to the external trigger interface of three sensors via a coaxial cable. The trigger delay difference is ≤5μs. Geometric calibration is completed through an 11×8 checkerboard grid (grid size 20mm). The three-channel registration error is ≤1 pixel.

[0110] Processing platform: Embedded processor (clock speed ≥ 1.5GHz) + GPU coprocessing, BM3D denoising and nonlocal mean filtering are accelerated in parallel by GPU; Polarization parameter calculation ( , , Accelerated by FPGA lookup table, single frame computation time ≤2ms (1280×720 resolution); Typical frame processing time (1280×720): preprocessing (including BM3D) is about 12ms, random resonance processing is about 15ms, fusion and postprocessing is about 5ms, totaling about 32ms (meeting the 30fps real-time requirement).

Claims

1. A three-channel low-light image enhancement method based on polarization guidance, characterized in that, The specific steps are as follows: Step 1: Simultaneously acquire the grayscale image, original color image, and polarization feature map corresponding to the same image using three channels; Step 2: Preprocess the acquired grayscale image, original color image, and polarization feature map; Step 3: Calculate the adaptive parameters for the grayscale channel and the adaptive parameters for the color brightness channel; Step 4: Perform dual-channel adaptive stochastic resonance processing on the preprocessed grayscale image and the original color image according to the adaptive parameters of the grayscale channel and the color brightness channel to obtain the enhanced grayscale image and color image. Step 5: Fuse the enhanced grayscale image, color image, and polarization feature map to obtain a fused image.

2. The three-channel low-light image enhancement method based on polarization guidance according to claim 1, characterized in that, When acquiring polarization feature maps, a micro-polarization array camera is used. This camera is equipped with pixel-level micro-polarizers and integrates four-directional polarization filter units (0°, 45°, 90°, and 135°) on the sensor pixel array. Four polarization components are acquired simultaneously with a minimum repeating unit of 2×2 pixels, achieving a resolution of 1224×1024 and a pixel size of 3.45μm. The output includes polarization intensity maps for the four directions: 0°, 45°, 90°, and 135°. , , , Calculate the Stokes parameters based on the four-directional polarization intensity diagram: Calculate the degree of polarization based on Stokes parameters : ； in, This represents the degree of polarization at pixel (x, y). It is a Stokes parameter; Calculate the polarization angle : in, The polarization angle at pixel (x, y); Generate polarization feature map Polarization feature map any point in the middle The pixel value at that location is: 。 3. The three-channel low-light image enhancement method based on polarization guidance according to claim 2, characterized in that, The preprocessing of the acquired grayscale image in step 2 specifically involves: For the acquired grayscale image BM3D denoising is performed as follows: grayscale image Divided into Pixel blocks, in The search window filters similar blocks by Euclidean distance, stacks the filtered similar blocks into a 3D matrix, and then applies wavelet transform to the 3D matrix layer by layer. In the transform domain of the wavelet transform, coefficient shrinkage is performed using hard thresholding. The wavelet-transformed 3D matrix is ​​then inversely transformed and weighted fused to output a denoised image. ; Denoising image Normalization is performed: Will Pixel values ​​mapped to The interval is used to obtain the normalized grayscale image. : in, For denoised image Centered at pixel Pixel value at that location, grayscale image after normalization Located at pixel Pixel value at; In step 2, the acquired original color image The preprocessing process specifically involves: Step 2.1, process the original color image To perform de-mosaic processing, specifically: Restored to RGB image using bilinear interpolation ; Step 2.2, will Perform color space conversion, specifically: Will Convert to YUV color space and separate the luminance component. Blue chromaticity component and red chromaticity component : Where R, G, and B are the R channel values, G channel values, and B channel values ​​of the RGB image, respectively. Step 2.3, for the luminance component Nonlocal mean filtering is used for the blue chromaticity component. and red chromaticity component use Gaussian filtering yields , ; The preprocessing of the polarization feature map in step 2 specifically involves: Degree of polarization of polarization feature map and polarization angle Median filtering is performed for noise reduction, specifically, the polarization feature map is... and Each applied Median filtering; polarization degree after median filtering and polarization angle Outlier correction is performed as follows: right The pixels are forced to be set to ; right The pixels are forced to be set to ; Stokes parameters Pixels are marked as invalid polarization regions; Will Normalization to .

4. The three-channel low-light image enhancement method based on polarization guidance according to claim 3, characterized in that, Step 3 specifically involves: Step 3.1: Extract the normalized grayscale images respectively. Compared with the luminance component after non-local mean filtering Local features; Step 3.2: Obtain the adaptive parameters for the grayscale channel and the adaptive parameters for the color channel based on local features.

5. The three-channel low-light image enhancement method based on polarization guidance according to claim 4, characterized in that, Step 3.1 specifically involves defining pixels. Centered sliding window Calculate the grayscale image within the sliding window respectively. Compared with the luminance component after non-local mean filtering Local brightness variance Texture complexity Local signal-to-noise ratio and grayscale images Significance of polarization structure ; Among them, the luminance variance The formula is as follows: in, or , For the corresponding sliding window Any pixel within, grayscale image In the sliding window any pixel within pixel value, Luminance component In the sliding window any pixel within Pixel brightness value at that location, This represents the average brightness within the corresponding window; Where N is the sliding window The total number of pixels within; The texture complexity The formula is as follows: in, in, , Sliding window for the corresponding image Inner pixel The directional gradient in the x and y directions, when calculating a grayscale image. Texture complexity hour, , Grayscale images sliding window Inner pixel The directional gradient in the x and y directions; when calculating the brightness component. Texture complexity hour, , These are the luminance components. sliding window Inner pixel The directional gradient in the x and y directions; The local signal-to-noise ratio The formula is as follows: in, It is the minimum value; The unit is dB; The polarization structure saliency The formula is as follows: in, for Structure mask image At pixel Pixel value at the location, For pixels Degree of polarization at that location; The structural mask image Obtain it in the following way: right Binarized edge images are generated using Canny edge detection. And then Perform morphological dilation to generate a structure mask image. .

6. The three-channel low-light image enhancement method based on polarization guidance according to claim 5, characterized in that, Step 3.2 specifically involves: Based on grayscale image Local brightness variance Texture complexity Local signal-to-noise ratio and polarization structure salience Calculate the adaptive parameters of the grayscale channels and : in, , , ; Based on the luminance components after nonlocal mean filtering Texture complexity Local signal-to-noise ratio Calculate adaptive parameters for color luminance channels and : in, , , .

7. The three-channel low-light image enhancement method based on polarization guidance according to claim 6, characterized in that, In step 4, the adaptive stochastic resonance processing of the preprocessed grayscale image based on the adaptive parameters of the grayscale channels is specifically as follows: (1) Construct an adaptive bistable stochastic resonance model, specifically as follows: in, grayscale image after normalization , , It is Gaussian white noise that follows a normal distribution. , For time step; in, grayscale image Noise intensity: in, As the reference noise intensity, , , , ; grayscale image corresponding and as well as Substituting into the above equation, we get ; (2) The adaptive bistable stochastic resonance model is solved iteratively using the improved Euler method formula. The iterative formula is as follows: in, For time step; Termination condition: 3 consecutive times Or reach the maximum number of iterations, 20; (3) Map the output results that meet the termination conditions back to The enhanced grayscale image is obtained. .

8. The three-channel low-light image enhancement method based on polarization guidance according to claim 7, characterized in that, In step 4, the adaptive stochastic resonance processing of the preprocessed original color image based on the adaptive parameters of the color luminance channel is specifically performed as follows: (1) Construct an adaptive bistable stochastic resonance model, specifically as follows: in, The luminance component after nonlocal mean filtering. The result of the normalization process is: , It is Gaussian white noise that follows a normal distribution. , For time step; in, The luminance component after non-local mean filtering Noise intensity: in, As the reference noise intensity, , , , ; The luminance component after nonlocal mean filtering corresponding and as well as Substituting into the above equation, we get ; (2) The adaptive bistable stochastic resonance model is solved iteratively using the improved Euler method formula. The iterative formula is as follows: in, For time step; Termination condition: 3 consecutive times Or reach the maximum number of iterations, 15; (3) Map the output results that meet the termination conditions back to The enhanced luminance component is obtained. ; (4) Use the enhanced luminance component And the Gaussian filter obtained in step 2 , Perform a YUV inverse matrix conversion back to RGB space to obtain the enhanced color image. .

9. The three-channel low-light image enhancement method based on polarization guidance according to claim 8, characterized in that, Step 5 specifically involves: For regions outside the invalid polarization region: right , , Three-channel weighted fusion is performed, and the fusion formula is as follows: in, for Image at any pixel pixel values, for Image at any pixel pixel values, Polarization feature map At any pixel Pixel values; For grayscale channel weights, For color channel weights, Polarization channel weights; The , , by With the core as the main focus, specifically: For regions outside the invalid polarization region: For invalid polarization regions, combine invalid polarization region mask. The fusion method is determined as follows: when When the polarization information is valid, three-channel fusion is used, which means that the fusion method is the same as that used for other regions outside the invalid polarization region. when When the polarization information is invalid, dual-channel fusion is used, and the fusion formula is: in: Map the fusion result to Rounding down yields the fused image. .

10. The three-channel low-light image enhancement method based on polarization guidance according to claim 9, characterized in that, It also includes the fused image Post-processing is performed, specifically as follows: Computational fusion of images Global average brightness L: in, To merge the image height, i.e., the number of image rows; To merge the image width, i.e. the number of image columns, Represents fused images At pixel Pixel value at; Based on global average brightness For fused images Dynamic adjustment Values ​​are used to obtain the corrected image. Specifically: in, To correct the image At any pixel Pixel value at; Calculate the corrected image Sobel edge intensity map : ； in, Edge intensity map At pixel Edge strength value at the location, , Correcting images Structure mask image Gradient in the x and y directions; The final enhanced image for: in, To enhance the image At pixel Pixel value at; For structure mask image At pixel The pixel value at the location.

Images