A haze scene restoration method based on polarization enhancement and residual total variation denoising

Abstract

The application discloses a kind of fog scene recovery methods based on polarization enhancement and residual total variation denoising, comprising the following steps: polarization filter defogging is carried out to polarized fog map by median filtering, and preliminary defogging image is obtained;Degree of polarization detail information is obtained using Stokes vector, abnormal value of degree of polarization image is removed by mathematical statistical analysis, and degree of polarization image is enhanced in detail;Degree of polarization image is handled using residual back mapping-TV denoising method, and low-noise degree of polarization image is obtained;Degree of polarization detail information is fused into polarized defogging image using multi-scale fusion method, target detail information is enhanced, and final recovery image is obtained.The application uses the above-mentioned kind of fog scene recovery method based on polarization enhancement and residual total variation denoising, and the detail information of target is clearly recovered under complex fog scene;It has obvious advantages in image detail, definition and stability, etc., and shows stronger adaptability and reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image data processing, and particularly relates to a fog scene restoration method based on polarization enhancement and residual total variation denoising. BACKGROUND

[0002] In the bad weather environment such as haze, there are a large number of suspended particles in the air, which interfere with the transmission of light scattering and absorption, and bring great influence to the imaging detection system, resulting in phenomena such as the decline of image contrast and the reduction of visibility, which seriously affects the detection scene such as target detection, urban traffic monitoring and military reconnaissance in computer vision tasks.

[0003] Early part of the algorithm is based on the orthogonal polarization difference model, using polarization difference information by manually selecting the sky area method to estimate the atmospheric polarization degree to solve the atmospheric light, and finally get the defogging image. Later, on the basis of the model, three or four different angle polarization images are used to estimate the atmospheric light parameters combined with Stokes vector, and then the haze-free image is inverted. The polarization angle information is used to automatically estimate the global parameter atmospheric polarization degree, which can avoid the selection of the sky area, but there is color distortion phenomenon in part of the scene. The transmission map optimization scheme of depth-chroma compensation regularization, and the image defogging optimization scheme of chroma-depth compensation regularization, and denoising processing in the whole processing process. The atmospheric light gradient prior information is used as a constraint condition to separate the target layer and the atmospheric layer information, and the atmospheric layer information is used to realize polarization reconstruction. In the intensity channel of HSI color space, the polarization optical defogging method is used for defogging processing, and the image color distortion is corrected combined with the color constancy principle. The fog image reconstruction method based on infinite atmospheric light polarization orthogonal decomposition constructs a multi-regularization constraint atmospheric scattering light blind separation model to realize accurate estimation of the atmospheric scattering light, but the color distortion phenomenon is more serious in the area with high haze concentration.

[0004] The current image defogging method is prone to over-processing of near scene image and insufficient processing of far scene image when processing near and far scene target regions at the same time, which greatly restricts the defogging effect of the image. Therefore, in view of this problem, the present application provides a fog scene restoration method based on polarization enhancement and residual total variation denoising, which uses the polarization filtering method to preprocess the large-scale information of the image as the base layer, uses the residual reflection map-TV denoising method to denoise the polarization degree image, obtains the low-noise polarization degree image on the basis of retaining the image detail information, uses the good information retention of the polarization degree information as the detail layer, and uses the multi-scale fusion idea to combine the base layer and the detail layer to process the image, suppresses the over-processing of the near scene image, and improves the processing effect of the far scene image. SUMMARY

[0005] The purpose of this invention is to provide a method for restoring foggy scenes based on polarization enhancement and residual total variation denoising. This method processes polarization information based on the intensity distribution of polarization information to enhance the intensity of texture details in the polarization image. It utilizes the residual inverse mapping-TV denoising method to denoise the polarization image, obtaining a low-noise polarization image while preserving image details. By fully leveraging the excellent information preservation properties of polarization information and fusing it into the target information, the method effectively enhances target details and clearly restores target information in complex foggy scenes.

[0006] To achieve the above objectives, this invention provides a method for restoring foggy scenes based on polarization enhancement and residual total variation denoising, comprising the following steps:

[0007] Step S1: Perform median filtering to remove haze from the polarization haze image to achieve polarization filtering and obtain a preliminary dehazed image;

[0008] Step S2: Obtain detailed polarization information using Stokes vectors, remove outliers from the polarization image through mathematical statistical analysis, and enhance the details of the polarization image.

[0009] Step S3: Use the residual inverse mapping-TV denoising method to denoise the polarization image to obtain a low-noise polarization image.

[0010] Step S4: Using a multi-scale fusion method, the polarization degree detail information is fused into the polarization dehazing image to enhance the target detail information and obtain the final restored image.

[0011] Preferably, in step S1, polarization imaging technology is used to image the target scene, acquiring foggy images under three polarization states of 0°, 60°, and 120° in the same scene, denoted as I0, I... 60 I 120 Calculate the Stokes vector as follows:

[0012]

[0013] In the formula, I represents the total intensity of the incident light; Q and U represent the polarization information of the incident light, respectively.

[0014] The linear polarization degree P of the image is shown below:

[0015]

[0016] Preferably, median filtering is used to filter the three polarization images, based on atmospheric light characteristics, from I0, I 60 and I 120 The atmospheric light intensities A0 and A2 at the corresponding polarization angles are obtained. 60and A 120 Based on the characteristics of high brightness and low saturation of atmospheric light in the sky region, the area with the highest brightness (top 5%) and lowest saturation (bottom 5%) is considered the sky region. The average value of the brightest 0.1% pixels in the selected sky region is taken as the atmospheric light intensity value A at infinity of the image. ∞ As shown below:

[0017]

[0018] In the formula, I 0 Represents the original image; I P I represents the image after polarization modulation. P =I·P; N represents the number of pixels in the selected sky region;

[0019] Using Stokes vectors, the obtained A0 and A 60 and A 120 Synthesized into atmospheric light A, based on an atmospheric scattering model combining A and A ∞ The preliminary dehazed image is obtained through calculation.

[0020] Preferably, in step S2, the Stokes vector is used to obtain polarization detail information, and through mathematical statistical analysis, a threshold is set to remove outliers in the polarization image to obtain image boundary values. Then, the image is subjected to pixel value intensity normalization processing to achieve polarization image detail enhancement.

[0021] Preferably, in step S3, the polarization image is denoised using the residual inverse mapping-TV denoising method to obtain a low-noise polarization image. The specific process is as follows:

[0022] Step S31: The total variation TV algorithm represents the model as follows:

[0023]

[0024] In the formula, u represents the ideal noise-free image; E(u) represents the energy function; k is the linear blur operator; f represents the initial noisy image; and γ is the regularization parameter.

[0025] Step S32: Design a method for processing residual images with dual thresholds, extracting detail information from the residual images again, and using inverse mapping to supplement the detail information into the denoised image, thus preserving the detail information from the denoising process; wherein, the threshold calculation method for processing residual images with dual thresholds is as follows:

[0026]

[0027] In the formula, dif represents the residual image; T h T lα1 and α2 represent the high and low thresholds for partitioning, respectively; α1 and α2 represent threshold parameters, and the residual region is partitioned by adjusting the threshold parameters.

[0028] Step S33: By performing nonlinear transformation processing on the residual region, some detailed information is recovered, as shown below:

[0029]

[0030] In the formula, dif option (x,y) represents the residual map containing detailed information after processing; ε is an adjustment parameter of 0-1 to control the degree of preservation of noise and weak details.

[0031] Preferably, in step S4, a multi-scale fusion method is used to fuse polarization detail information into the polarization dehazed image, enhancing the target detail information and obtaining the final restored image. The specific process is as follows:

[0032] Step S41: Introduce the LatLRR model, an implicit low-rank representation, to perform low-rank decomposition on the image data and reveal the global structure and local salient features;

[0033] Step S42: Perform dual-weight fusion on the saliency map and gradient map features of the low-rank layer of the image;

[0034] Step S43: Perform image fusion on the salient layers of the image based on the improved Laplacian energy and function;

[0035] Step S44: Overlay the low-rank layer and the salient layer to obtain the brightness map of the fused image, and perform local window pixel value consistency verification of the fused image to obtain the final fused enhanced image.

[0036] Preferably, in step S41, the two images are converted to the LAB color space, and an implicit low-rank representation LatLRR model is introduced to perform low-rank decomposition on the image data, revealing the global structure and local salient features, and separating noise from the source image information; wherein, the implicit low-rank representation model is as follows:

[0037]

[0038] In the formula, Z is the low-rank coefficient; L is the significance coefficient; E is the sparse noise component; λ is the balance coefficient; |||| * denoted by the nuclear norm; X represents the image data matrix; X·Z represents the low-rank layer of the image, i.e., the base layer, which contains the target and background details of the entire image; L·X represents the salient layer of the image, i.e., the detail layer, which reflects the local salient regions and edge information.

[0039] Preferably, in step S42, the saliency map and gradient map features of the low-rank layer of the image are fused with dual weights, and the weighting formula is as follows:

[0040]

[0041] In the formula, ω1(x,y) and ω2(x,y) are the image fusion weights; S1(x,y) and S2(x,y) are the saliency maps of the two images; G1(x,y) and G2(x,y) are the gradient information maps of the two images; β is an adaptive coefficient used to adjust the proportion of the saliency map and the gradient map in the fusion weights, as shown below:

[0042]

[0043] In the formula, ΔS(x,y) represents the difference between two saliency images at pixel (x,y); ΔG(x,y) represents the difference between two gradient information images at pixel (x,y); Ω is the neighborhood centered on pixel (x,y), and the average difference of the neighborhood is used as the difference of that pixel.

[0044] Based on this, the results of grassroots integration are as follows:

[0045] I col_base (x,y)=ω1I dehaze_base (x,y)+ω2I Dop_base (x,y) (16);

[0046] In the formula, I col_base (x,y) represents the results of grassroots integration; I dehaze_base (x,y) represents the base layer of the dehazed image; I Dop_base (x,y) represents the base layer of the enhanced polarization image.

[0047] Preferably, in step S42, image fusion is performed on the salient layers of the image based on the improved Laplacian energy and function, as follows:

[0048] NSML, in addition to calculating the variable-step Laplacian operator value ML for each pixel in the horizontal and vertical directions, also calculates the variable-step Laplacian operator values ​​in all four diagonal directions; the variable-step Laplacian operator value ML is as follows:

[0049]

[0050] In the formula, ML(x,y) represents the variable step size Laplacian operator value of a pixel; I(x,y) represents the image calculated using the Laplacian operator; step represents the variable step size parameter calculated by ML.

[0051] The improved Laplace energy and function are as follows:

[0052]

[0053] In the formula, NSML(x,y) represents the sum of the Laplace energy of the region; N represents the neighborhood size;

[0054] Therefore, NSML is used to fuse the detail layers, and the fused result of the detail layers is shown below:

[0055]

[0056] In the formula, I col_detail (x,y) represents the fusion result of the detail layer; I dehaze_detail (x,y) represents the detail layer of the dehazed image; I Dop_detail (x,y) represents the detail layer of the polarization degree image; NSML dehaze (x,y) represents the Laplacian energy of the dehazed image; NSML Dop (x,y) represents the Laplace energy of the polarization image.

[0057] Preferably, in step S42, the base layer and the detail layer are overlaid to obtain a brightness map of the fused image. The fused image undergoes a local window pixel value consistency check. Then, the image is converted from the LAB color space to the RGB color space to obtain the final fused and enhanced image, as shown below:

[0058] I col (x,y)=I col_base (x,y)+I col_detail (x,y) (20).

[0059] Therefore, the present invention employs the aforementioned method for restoring foggy scenes based on polarization enhancement and residual total variation denoising, with the following beneficial effects:

[0060] (1) The polarization degree information is processed based on the polarization information intensity distribution, which effectively enhances the intensity of the polarization degree image texture detail information;

[0061] (2) Make full use of the residual details in the denoised residual image, use the threshold region division method to process the residual region, obtain some information and back-map it to the noisy image, which effectively enhances the ability to retain details in the denoised image.

[0062] (3) By making full use of the good information retention properties of polarization information and fusing polarization information into target information, the target detail information can be effectively enhanced and the target information can be clearly restored in complex fog and haze scenes;

[0063] (4) The image is converted into the LAB color space, and some high-frequency noise is suppressed by low-rank decomposition. The preliminary dehazed image is fused with the polarization image by multi-scale fusion, which effectively enhances the target information in the dehazed result image.

[0064] (5) The present invention has excellent target information recovery capability, which enables it to recover the detailed information of the target more clearly in complex fog and haze scene environment; it has obvious advantages in image detail, clarity and stability, and shows stronger adaptability and reliability.

[0065] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0066] Figure 1 For atmospheric scattering models;

[0067] Figure 2 This is a technical flowchart of a fog scene restoration method based on polarization enhancement and residual total variation denoising according to the present invention.

[0068] Figure 3 This is a flowchart illustrating a specific implementation example of the fog scene restoration method based on polarization enhancement and residual total variation denoising according to the present invention.

[0069] Figure 4 This is a 0° polarized haze image;

[0070] Figure 5 This is a 60° polarized haze image;

[0071] Figure 6 This is a 120° polarized haze image;

[0072] Figure 7 This is the initial image of the polarization filtering dehazing process;

[0073] Figure 8 This is a polarization degree image;

[0074] Figure 9 This is a diagram showing the results of enhanced polarization degree.

[0075] Figure 10 This is a grayscale image of the polarization enhancement result;

[0076] Figure 11 Image showing the result of polarization degree denoising;

[0077] Figure 12 The image is normalized from the residual map between the denoised image and the noisy image.

[0078] Figure 13 The final image after dehazing by fusing polarization information. Detailed Implementation

[0079] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0080] Example

[0081] like Figure 1 As shown, the atmospheric scattering model is used to describe the fog imaging process, and this model is further developed and widely used in various computer vision systems, as shown below:

[0082] I 0 =D+A=J·t+A ∞ ·(1-t)(1);

[0083] In the formula, I 0 Represents the original foggy image captured by the camera; J represents the scene's reflected light, which is the image to be recovered for defogging based on an atmospheric scattering model; A ∞ represents the atmospheric light intensity at infinity; t represents the transmission rate.

[0084] In the formula, the first term J·t is considered to be the portion of the scene-reflected light that successfully reaches the camera without being scattered, called the directly transmitted light, denoted as D; while the second term A ∞ ·(1-t) represents the portion of global atmospheric light that reaches the camera after being scattered by the medium, denoted as atmospheric light A.

[0085] The dehazing objective based on this model is to estimate A from the acquired foggy images, based on formula (1). ∞ , from I0, I 60 and I 120 A0 and A are obtained by filtering based on atmospheric optical properties. 60 and A 120 Atmospheric light A is obtained from the Stokes vector, and J is then recovered, as shown below:

[0086]

[0087] For example, when a beam of linearly polarized light with a Stokes vector of S = (I, Q, U) passes through a polarizer with a polarization angle of α, the polarization image intensity I, expressed in terms of the Stokes vector, can be obtained using the relationship between the Stokes vector and the Mueller matrix. θ As shown below:

[0088]

[0089] When an incident beam of light passes through polarizers with orientations of 0°, 60°, and 120°, the corresponding polarization image light intensity values ​​are I0, I1, I2, and I3, respectively. 60 and I 120Substituting this into equation (3) above, we can calculate the Stokes vector as shown below:

[0090]

[0091] In the formula, I represents the total intensity of the incident light; Q and U represent the polarization information of the incident light, respectively.

[0092] Then, the linear polarization degree P and polarization angle Aop are obtained, as shown below:

[0093]

[0094] Based on the above-mentioned principle derivation, this invention proposes a method for restoring foggy scenes based on polarization enhancement and residual total variation denoising, such as... Figure 2 and Figure 3 As shown, it includes the following steps:

[0095] Step S1: Perform median filtering to remove haze from the polarization haze image to achieve polarization filtering and obtain a preliminary dehazed image.

[0096] Step S11: Use polarization imaging technology to image the target scene and obtain three polarization images of the same scene.

[0097] Images of foggy weather were acquired using a polarization camera at three polarization states: 0°, 60°, and 120°, as shown below. Figure 4 , Figure 5 and Figure 6 As shown, denoted as I0 and I... 60 and I 120 The Stokes vector is calculated as follows:

[0098]

[0099] The linear polarization degree P of the image is shown below:

[0100]

[0101] Step S12: Use median filtering to filter the three polarization images to obtain preliminary dehazed images.

[0102] Based on atmospheric optical characteristic filtering, from I0, I 60 and I 120 We obtain A0 and A 60 and A 120 By combining the characteristics of high brightness and low saturation of atmospheric light in the sky region, a suitable sky region is selected. The region with the highest brightness (top 5%) and lowest saturation (bottom 5%) is considered as the sky region. The average value of the brightest 0.1% pixels in the selected sky region is taken as the atmospheric light intensity value A at infinity of the image. ∞ As shown below:

[0103]

[0104] In the formula, I 0 Represents the original image; I P I represents the image after polarization modulation. P =I·P; N represents the number of pixels in the selected sky region.

[0105] Using Stokes vectors, the obtained A0 and A 60 and A 120 Synthesized into atmospheric light A, based on an atmospheric scattering model combining A and A ∞ The preliminary dehazed image is calculated, as follows: Figure 7 As shown.

[0106] Step S2: Use Stocks vectors to obtain detailed polarization information, remove outliers from the polarization image through mathematical statistical analysis, and enhance the details of the polarization image.

[0107] Statistical analysis of the intensity distribution of pixel values ​​in a large number of polarization images revealed that the intensity distribution of most pixel values ​​in the images is concentrated in a certain area, forming a distinct distribution peak. A few pixel values ​​with excessively high or low intensity will form a nearly straight line with almost no fluctuation on either side of the distribution peak.

[0108] Therefore, by selecting an appropriate threshold, these abnormally strong pixel values ​​are excluded, thus obtaining image boundary values. Pixel value intensity normalization is then applied to the image to obtain a polarization-enhanced image, such as... Figure 8 , Figure 9 and Figure 10 As shown.

[0109] Step S3: Use residual inverse mapping combined with TV denoising algorithm to denoise the polarization image, remove most of the interference noise, and obtain a low-noise polarization image.

[0110] Step S31, the TV (Total Variation) algorithm represents the model as follows:

[0111]

[0112] In the formula, u represents the ideal noise-free image; E(u) represents the energy function; k is the linear blur operator; f represents the initial noisy image; and γ is the regularization parameter, which balances the weights of the regularization term and the data fidelity term by adjusting the value of γ. The larger the γ is, the more significant the denoising effect, but the smoother the image will be.

[0113] Step S32: When denoising the polarization image, calculate the difference between the noisy image and the denoised image as the residual image, and use high and low thresholds to divide the residual region.

[0114] The TV algorithm's denoising process is a continuous iterative optimization process. However, during the denoising process, some texture details of the image are inevitably mistakenly identified as noise and removed. This results in the residual image containing not only noise information but also some detail information, and this detail information increases with the number of iterations.

[0115] Therefore, a method for processing residual images with dual thresholds was designed to extract detailed information from the residual images and then supplement the detailed information into the denoising image through inverse mapping, so that it has better information preservation in the denoising process.

[0116] The threshold calculation method for the residual map in the dual-threshold processing is as follows:

[0117]

[0118] In the formula, dif represents the residual image; T h T l α1 and α2 represent the high and low thresholds for the division, respectively; α1 and α2 represent the threshold parameters, and the appropriate threshold range is selected by adjusting the threshold parameters to divide the residual region.

[0119] Step S33: By performing further nonlinear transformation processing on the residual region, some detailed information can be recovered, as shown below:

[0120]

[0121] In the formula, dif option (x,y) represents the residual map containing detailed information after processing; ε is an adjustment parameter of 0-1 to control the degree of preservation of noise and weak details.

[0122] Based on the statistical characteristics of noise and edge information, regions with large residuals likely correspond to true detail information; therefore, these regions are preserved entirely. For regions with small residuals, which are typically a mixture of noise and some weak detail information, the degree of preservation of noise and weak detail is controlled using an adjustable parameter ε ranging from 0 to 1. For the remaining regions with medium residuals, a soft thresholding approach is used to reduce the information intensity without completely eliminating it, avoiding artifacts. Figure 11 and Figure 12 As shown.

[0123] Step S4: Using a multi-scale fusion method, the polarization degree detail information is fused into the polarization dehazing image to enhance the target detail information and obtain the final restored image.

[0124] Step S41: Convert the two images to the LAB color space, introduce the implicit low-rank representation LatLRR model, and perform low-rank decomposition on the image data to reveal the global structure and local salient features.

[0125] Multi-scale fusion aims to generate more comprehensive and task-adaptive fusion results by integrating features or data from different scales or resolutions. Due to the scattering and reflection of atmospheric particles in foggy scenes, images captured by imaging systems contain a large amount of noise. Even after dehazing and good preservation of polarization information, the restored image and polarization information map still contain significant amounts of noise.

[0126] To address the issue of noise interference, a Latent Low-Rank Representation (LatLRR) model is proposed, leveraging the noise sensitivity of the LRR model. This model performs row and column low-rank representations of corrupted data and adds nuclear norm constraints. By introducing hidden data, the robustness of the model is enhanced, and the low-rank decomposition of the data reveals the global structure and local salient features. Based on the properties of low-rank representation, noise can also be effectively separated from the source image information.

[0127] The Latent Low-Rank Representation (LatLRR) model is shown below:

[0128]

[0129] In the formula, λ is the balance coefficient; * denoted by KN norm; X represents the image data matrix; Z is the low-rank coefficient; X·Z represents the low-rank layer of the image, which contains most of the target and background detail information of the entire image. The low-rank layer is also called the base layer, which contains the main intensity and brightness information of the image; L is the saliency coefficient; L·X represents the salient layer of the image, which mainly reflects the local salient regions and edge information. The salient layer is also called the detail layer, which reflects the detailed texture information of the image; E is the sparse noise component.

[0130] Step S42: Perform weighted fusion of features from the saliency map and gradient map of the low-rank layer image.

[0131] Leveraging the characteristics of saliency maps, excellent fusion results are achieved in salient target regions of fused images. The gradient map is introduced as a parameter to assist in adjusting the processing effect of some image details. Considering both gradient information and saliency maps, a dual-weight fusion method based on saliency map and gradient is designed, with the following weighting formula:

[0132]

[0133] In the formula, ω1(x,y) and ω2(x,y) are the image fusion weights; S1(x,y) and S2(x,y) are the saliency maps of the two images; G1(x,y) and G2(x,y) are the gradient information maps of the two images; β is an adaptive coefficient used to adjust the proportion of the saliency map and the gradient map in the fusion weights, as shown below:

[0134]

[0135] In the formula, ΔS(x,y) represents the difference between two saliency images at pixel (x,y); ΔG(x,y) represents the difference between two gradient information images at pixel (x,y); Ω is the neighborhood centered on pixel (x,y). In order to reduce the calculation error, the average difference of the neighborhood is used as the difference of the pixel.

[0136] When the saliency difference between two images is large, the value of β will tend to be closer to 1. In this case, the image fusion weights will be mainly based on the saliency of the images. Conversely, if the gradient difference is large, it reflects that the detail image will contain more detail information. In this case, the value of β will tend to be closer to 0, and the fusion weights will be mainly based on gradient information to fully enhance the preservation of detail information in the fused image.

[0137] Based on this, the results of grassroots integration are as follows:

[0138] I col_base (x,y)=ω1I dehaze_base (x,y)+ω2I Dop_base (x,y) (16);

[0139] In the formula, I col_base (x,y) represents the results of grassroots integration; I dehaze_base (x,y) represents the base layer of the dehazed image; I Dop_base (x,y) represents the base layer of the enhanced polarization image.

[0140] Step S43: Based on the improved Laplacian energy and function, perform preliminary fusion of detail images on the salient layer of the image.

[0141] In the salient layer of an image, high-frequency information reflects the changes in pixel intensity values. Dramatic high-frequency information often corresponds to significant details such as texture and edges. However, in image fusion, directly using the maximum value fusion method often overlooks the correlation between pixels.

[0142] Therefore, in order to obtain a fused image with good visual effects and rich details, the characteristic of Laplacian energy to reflect the texture and edge information of the image is used to perform preliminary fusion of the detailed images. Then, the preliminary fused image and the initial image are checked for consistency based on local windows to avoid pixel value jumps between pixels in the fused image that would affect the final fusion effect.

[0143] NSML is an improvement on the traditional SML algorithm. In addition to calculating the variable step size Laplacian operator value ML for each pixel in the horizontal and vertical directions, it also calculates the variable step size Laplacian operator values ​​in all four directions along the diagonal.

[0144] The variable-step-size Laplace operator value ML is shown below:

[0145]

[0146] In the formula, ML(x,y) represents the variable-step Laplacian operator value of a pixel; I(x,y) represents the image calculated using the Laplacian operator; and step represents the variable-step parameter calculated by ML. When the noise interference is small, the value of step should be set relatively small; conversely, when the noise interference is large, the value of step should be set relatively large to reduce the interference of noise on the calculation results.

[0147] The improved Laplace energy and function are as follows:

[0148]

[0149] In the formula, NSML(x,y) represents the Laplace energy of the region; N represents the neighborhood size.

[0150] Since NSML can better reflect the feature information of an image, the pixel value of the image corresponding to the larger value in the NSML of the two images is taken as the pixel value of the detail layer fused image, so as to obtain stronger detail information to be incorporated into the fused image.

[0151] Therefore, NSML is used to fuse the detail layers, and the fused result of the detail layers is shown below:

[0152]

[0153] In the formula, I col_detail (x,y) represents the fusion result of the detail layer; I dehaze_detail (x,y) represents the detail layer of the dehazed image; I Dop_detail (x,y) represents the detail layer of the polarization degree image; NSML dehaze (x,y) represents the Laplacian energy of the dehazed image; NSML Dop(x,y) represents the Laplace energy of the polarization image.

[0154] Step S44: Perform a local window-based consistency check between the preliminary fused image and the initial image, that is, convert the image from the LAB color space to the RGB color space to obtain the final enhanced image, thus obtaining the final fused enhanced image, as shown. Figure 13 As shown.

[0155] The base layer and detail layer are overlaid to obtain the brightness map of the fused image. The fused image undergoes local window pixel value consistency verification to avoid abrupt changes in neighboring pixel values. Then, the image is converted from the LAB color space to the RGB color space to obtain the final fused and enhanced image, as shown below:

[0156] I col (x,y)=I col_base (x,y)+I col_detail (x,y) (20).

[0157] Therefore, this invention employs a foggy scene restoration method based on polarization enhancement and residual total variation denoising. It processes polarization information based on the intensity distribution of polarization information, effectively enhancing the intensity of texture details in the polarization image. The residual inverse mapping-TV denoising method is used to denoise the polarization image, obtaining a low-noise polarization image while preserving image details. Taking full advantage of the good information preservation properties of polarization information, the polarization information is fused into the target information, effectively enhancing target details and clearly restoring target information in complex foggy scenes. It performs particularly well in scenes with both near and far targets, effectively restoring details of distant targets while significantly suppressing overprocessing of near targets. The image is converted to the LAB color space, and low-rank decomposition is used to suppress some high-frequency noise. A multi-scale fusion method is used to fuse the initial defogging image with the polarization image, effectively enhancing target information in the defogging result image. This invention possesses excellent target information restoration capabilities, enabling clearer restoration of target details in complex foggy environments. It exhibits significant advantages in image detail, clarity, and stability, demonstrating stronger adaptability and reliability.

[0158] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to be construed as...

[0159] Although the invention has been described in detail with reference to preferred embodiments, this invention is not intended to limit it.

[0160] Those skilled in the art should understand that they can still make modifications to the technical solutions of this invention.

[0161] Modifications or equivalent substitutions are made, but these modifications or equivalent substitutions cannot improve the modified technology.

[0162] The proposed solution deviates from the spirit and scope of the technical solution of this invention.

Claims

1. A method for fog scene restoration based on polarization enhancement and residual total variation denoising, characterized in that, The method comprises the following steps: Step S1, haze removal processing is performed on the polarized haze image by median filtering, polarized filter haze removal is realized, and a preliminary haze-removed image is obtained; Step S2, polarized degree detail information is obtained by using a Stokes vector, abnormal values of the polarized degree image are removed by mathematical statistical analysis, and the polarized image is enhanced in detail; Step S3, residual reflection map-TV denoising method is used to perform denoising processing on the polarized degree image, and a low-noise polarized degree image is obtained; Step S4, a multi-scale fusion method is used to fuse the polarized degree detail information into the polarized haze-removed image, target detail information is enhanced, and a final restored image is obtained, and the specific process is as follows: Step S41, a LatLRR model is introduced, image data is decomposed in low rank, and global structure and local significant features are revealed; Step S42, the significance map and gradient map features of the image low rank layer are double-weight fused, and the weighting formula is as follows: (14)； wherein, , is an image fusion weight; , is a saliency map of two images; , is a gradient information map of two images; is an adaptive coefficient for adjusting the proportion of saliency map and gradient map in the fusion weight, as follows: (15)； wherein, denotes the difference between the two saliency images at pixel point ; denotes the difference between the two gradient information images at pixel point ; is a neighborhood centered at pixel point , and the neighborhood average difference is taken as the difference of the pixel point. Step S43, the significance layer of the image is fused based on the improved Laplace energy function; Step S44, the low rank layer and the significance layer are superimposed to obtain the luminance map of the fused image, and local window pixel value consistency verification of the fused image is performed to obtain the final fused enhanced image.

2. The method of claim 1, wherein the method is characterized by, In step S1, the target scene is imaged by using the polarization imaging technology to obtain the foggy images under three polarization states of 0°, 60° and 120°, denoted as , , The Stokes vector is calculated as follows: (7)； wherein is the total light intensity information of the incident light; and are the polarization information of the incident light rays, respectively; Linear polarization degree of an image As follows: (8)。 3. The method of claim 2, wherein the method is characterized by, The three polarized images are filtered by median filtering, atmospheric light filtering based on atmospheric light characteristics, and the atmospheric light intensity under the corresponding polarization angle is obtained from 、 and 、 and . The sky region is selected by combining the high brightness and low saturation characteristics of the atmospheric light, and the 0.1% brightest pixel mean value of the selected sky region is taken as the atmospheric light intensity value at infinity of the image , as follows:​ (9)； In the formula, denotes the original image; denotes the image after polarization modulation, ; denotes the number of selected sky region pixel points; By using Stokes vector, the obtained , and are synthesized into atmospheric light , and based on atmospheric scattering model, combined with and , the preliminary defogging image is calculated.

4. The method of claim 1, wherein the method is characterized by, In step S2, the polarized degree detail information is obtained by using a Stokes vector, abnormal values of the polarized degree image are removed by setting a threshold value through mathematical statistical analysis, image boundary values are obtained, and then pixel value intensity normalization processing is performed on the image to realize polarized image detail enhancement.

5. The method of claim 1, wherein the method is characterized by, In step S3, residual reflection map-TV denoising method is used to perform denoising processing on the polarized degree image, and a low-noise polarized degree image is obtained, and the specific process is as follows: Step S31, the total variation TV algorithm model is as follows: (10)； wherein denotes the ideal noise-free image; denotes the energy function; is a linear blurring operator; denotes the initial noisy image; is a regularization parameter; Step S32, when the polarized degree image is denoised, the difference between the noise image and the denoised image is calculated as a residual image, and high and low double thresholds are used to divide the residual area; wherein, the threshold calculation method of the double threshold processing residual image is as follows: (11)； In the formula, represents a residual image; , respectively represent a divided high threshold value and a low threshold value; , represents a threshold parameter, and the residual region division is realized by adjusting the threshold parameter; Step S33, part of the detail information is restored by nonlinear change processing on the residual area, and the specific process is as follows: (12)； wherein represents the residual map containing the details information after processing; is a tuning parameter between 0 and 1 that controls the degree of preservation of noise and weak details.

6. The method of claim 1, wherein the method is characterized by, In step S41, two images are converted to the LAB color space, a LatLRR model is introduced, image data is decomposed in low rank, global structure and local significant features are revealed, and noise is separated from the source image information; wherein, the LatLRR model is as follows: (13)； wherein, is a low-rank coefficient; is a saliency coefficient; is a sparse noise part; is a balance coefficient; is a nuclear norm; denotes an image data matrix; denotes a low-rank layer of the image, i.e. a base layer, containing target and background details information of the whole image; denotes a saliency layer of the image, i.e. a detail layer, embodying local salient region and edge information.

7. The method of claim 6, wherein the method is characterized by, In step S42, the significance map and gradient map features of the image low rank layer are double-weight fused, and the base layer fusion result is as follows: （16）； In the formula, denotes the base layer fusion result; denotes the base layer of the defogged image; denotes the base layer of the enhanced polarization degree image.

8. The method of claim 7, wherein the method is characterized by, In step S42, the significance layer of the image is fused based on the improved Laplace energy function, and the specific process is as follows: NSML increases the calculation of all variable step length Laplace operator values in four directions on the diagonal line on the basis of calculating the variable step length Laplace operator value ML in the horizontal and vertical directions of each pixel point; wherein, the variable step length Laplace operator value ML is as follows: (17)； In the formula, represents a variable step Laplacian value of a pixel point; represents an image calculated by using a Laplacian operator; represents a variable step parameter of ML calculation; The improved Laplace energy function is as follows: (18)； wherein denotes the regional Laplacian energy; denotes the neighborhood size; Therefore, the detail layer is fused by using NSML, and the fusion result of the detail layer is as follows: (19)； In the formula, denotes the fusion result of the detail layer; is the detail layer of the defogging image; is the detail layer of the polarization degree image; is the Laplace energy sum of the defogging image; is the Laplace energy sum of the polarization degree image.

9. The method of claim 8, wherein the method is characterized by, In step S42, the base layer is superimposed with the detail layer to obtain a luminance map of the fusion image, the fusion image is subjected to local window pixel value consistency checking, and then the image is converted from the LAB color space to the RGB color space to obtain a final fusion enhanced image, as shown below: (20)。

Images