A method of visible light image despeckling

Abstract

The application discloses a visible light image despeckling method, and steps are as follows: estimating a visible light transmittance map and atmospheric light of a visible light image; using an infrared image to optimize the edge of the estimated visible light transmittance map; extracting edge information of the infrared image; establishing and solving a despeckling optimization model to output a despeckled visible light image. The application estimates the visible light transmittance map and the atmospheric light, and optimizes the edge of the visible light transmittance map based on the edge of the infrared image to obtain a visible light transmittance map with clear edges; then, edge information of the infrared image is extracted, the advantage that the infrared image can be clearly imaged under a scattering environment is fully utilized, the visible light image is optimized based on the area contrast information and the edge information of the infrared image, finally, the despeckling optimization model is established and solved, and the despeckling effect of the visible light image is enhanced, so that clear color imaging of the visible light image under a scattering scene is realized, and the clear color imaging through the scattering medium is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and digital image processing, in particular to a visible light image de-scattering method. BACKGROUND

[0002] Scattering phenomenon is widespread, for example, fog, cloud smoke and other scattering media scatter visible light, resulting in a serious decline in the imaging quality of visible light equipment. This brings great challenges to the completion of fire rescue, aviation, navigation, remote sensing and other tasks, and also limits the further development of computer vision technology landing fields such as automatic driving, intelligent transportation, robots and the like. Therefore, it is crucial to realize clear color imaging through scattering media. The existing visible light image defogging method estimates the transmittance map and the atmospheric light from a single visible light image, so as to solve the atmospheric scattering model and realize de-scattering, however, such method has great limitations in strong scattering environment such as thick fog: the loss of edge information of visible light image in thick fog scattering environment will lead to the failure of optimization for the initial estimated transmittance map; when the atmospheric light in the environment is uneven, it is easy to cause parameter misestimation of the scattering model, and reduce the de-scattering effect; the image defogging method cannot restore the image edge and detail information lost due to scattering.

[0003] Compared with visible light, infrared waveband has longer wavelength, and thus has stronger ability to resist the interference of scattering media. Therefore, in the scattering environment with fog or cloud smoke, infrared cameras can obtain clearer imaging results than visible light cameras. In recent years, with the gradual development of civil infrared technology, the cost of infrared sensors has gradually decreased, and infrared cameras have been more widely applied. Therefore, in the scattering environment, fusing the clear imaging infrared image to assist the visible light to realize de-scattering has great application prospect and potential.

[0004] At present, there are many works on the fusion method of visible light image and infrared image, which extracts more prominent areas in the two types of images for fusion, so as to achieve the effect of fusion enhancement. In recent years, many end-to-end fusion methods based on deep neural network have also been proposed. However, due to the difference between the imaging principles of infrared cameras and visible light cameras, there is a strong inconsistency problem between the pixels of the images. Therefore, in the scattering environment, the method based on image fusion directly fuses the pixel intensity, which often leads to distortion and light halo phenomenon. At the same time, such method does not essentially solve the problem of image quality reduction caused by scattering, which makes the fusion result generally have the phenomenon of low contrast, overall brightness is dark, and color saturation is not high. SUMMARY

[0005] In order to solve the technical problem that the existing image fusion method cannot solve the image quality reduction caused by scattering, the purpose of the present application is to provide a visible light image de-scattering method.

[0006] The technical problem of the present application is solved by the following technical solutions:

[0007] A visible light image de-scattering method, comprising the following steps:

[0008] S1, estimating a visible light transmittance map and atmospheric light for a visible light image;

[0009] S2, using an infrared image to perform edge optimization on the estimated visible light transmittance map;

[0010] S3, extracting edge information of the infrared image;

[0011] S4, establishing and solving a de-scattering optimization model, and outputting a de-scattered visible light image.

[0012] In some embodiments, in step S1, the visible light image in a scattering scene is represented as:

[0013] I VIS (x)=J VIS (x)t VIS (x)+A(1-t VIS (x));

[0014] wherein I VIS is a visible light image in a scattering scene acquired by a visible light camera, J VIS is a non-scattering visible light image, A represents atmospheric light caused by ambient light, t VIS (x) is a visible light transmittance map, and x is an image coordinate.

[0015] In some embodiments, the method of estimating the visible light transmittance map and the atmospheric light comprises a dark channel prior algorithm DCP, a color attenuation prior algorithm CAP, and a haze-line based algorithm Haze-lines.

[0016] In some embodiments, in step S2, the edge optimization of the estimated visible light transmittance map using the infrared image is represented as:

[0017]

[0018] wherein GuidedFilter(·) represents a guided filter algorithm, the output of the algorithm is the visible light transmittance map t VIS-refined that is edge-optimized, and the input parameters are an initially estimated visible light transmittance map an infrared image I IR , h g , and λ, h g is a window size of the guided filter, and ε represents a regularization parameter.

[0019] In some embodiments, in step S3, the method for extracting the edge information of the infrared image comprises a Haar wavelet, a log-Gabor filter, a shearlet, and a gradient operator.

[0020] Wherein, the gradient operator is used to extract the edge information of the infrared image, and is expressed as:

[0021]

[0022] Wherein, i represents the filter number, and in the gradient operator, 1≤i≤2, represents a horizontal gradient operator, represents a vertical gradient operator.

[0023] In some embodiments, in step S4, the established de-scattering optimization model comprises a guided contrast enhancement term G(·), a transmittance map optimization atmospheric scattering model term H(·), an image edge preservation term ED(·), and a color fidelity term CF(·).

[0024] In some embodiments, the contrast enhancement term G(·) is expressed as:

[0025]

[0026] Wherein, the weight ω(x, y) between the pixel x and the pixel y is expressed as:

[0027]

[0028] Wherein, X represents a set of all pixel positions of the image; J(x) represents the intensity of the visible light image J at the pixel x, J(y) represents the intensity of the visible light image J at the pixel y, R(x) represents the intensity of the infrared image R at the pixel x, R(y) represents the intensity of the infrared image R at the pixel y, I IR (x) represents the intensity of the long-wave infrared image at the pixel x; σ R and σ S are smoothing parameters;

[0029] The transmittance map optimization atmospheric scattering model term H(·) is expressed as:

[0030]

[0031] Wherein, represents the square of the 2-norm, I is a visible light scattering image obtained by a visible camera, t VIS-refined is a visible light transmittance map obtained by performing edge optimization in step S2, and A is atmospheric light.

[0032] The image edge preservation term ED(·) is expressed as:

[0033]

[0034] in, Edge represents the square of the 2-norm. i (I IR The edge information of the infrared image extracted in step S3 is called Edge. i (J) represents the edge of a visible light image;

[0035] The color fidelity item CF(·) is represented as:

[0036]

[0037] in, Represents the square of the 2-norm. This indicates that a window of size h is executed on image J. a Mean filtering operation.

[0038] In some embodiments, in step S4, solving the descattering optimization model is expressed as minimizing the following optimization problem:

[0039]

[0040] Where E(·) is the cost function of the optimization problem, J is the visible light image to be descattered, G(·) is the guided contrast enhancement term, H(·) is the atmospheric scattering model term for transmittance map optimization, ED(·) is the image edge preservation term, CF(·) is the color fidelity term, and the hyperparameters α, β, γ and δ represent the weights of each term in the cost function, and all of them should be greater than 0.

[0041] In some embodiments, the descattering optimization model is represented as a method for minimizing the following optimization problem, including the following steps:

[0042] S4-1. Acquire visible light images in a scattering scene (I) VIS Infrared images in scattering scenarios I IR k is the iteration number, J k J is the result of the k-th iteration. k-1 This represents the result of the (k-1)th iteration, and J is initialized. 0 Visible light image I in a scattering scene VIS ;

[0043] S4-2. Calculate the gradient term of the cost function E(·) for the optimization problem.

[0044] S4-3, through formula Update the visible light image to obtain the current iteration result J. k ,in, is the gradient step size, and

[0045] S4-4, calculating the current iteration result J k is the root mean square error of the previous iteration result J k-1

[0046] S4-5, if the RMSE (J k , J k-1 ) is less than a threshold value ∈, then stop iteration and output the de-scattered visible light image, otherwise jump to step S4-2 to calculate the gradient term Continue iteration.

[0047] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the visible light image de-scattering method.

[0048] The beneficial effects of the application compared with the prior art include:

[0049] The application estimates the visible light transmittance map and atmospheric light, optimizes the edge of the visible light transmittance map based on the edge of the infrared image, obtains a visible light transmittance map with a clearer edge, extracts the edge information of the infrared image, fully utilizes the advantage that the infrared image can clearly image in a scattering environment, optimizes the visible light image based on the regional contrast information and edge information of the infrared image, finally establishes and solves the de-scattering optimization model, enhances the de-scattering effect of the visible light image, and thus realizes clear color imaging of the visible light image in a scattering scene, which is of great significance to realize clear color imaging through a scattering medium.

[0050] Other beneficial effects of the embodiments of the application will be further described below. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a flowchart of the visible light image de-scattering method in the embodiments of the application;

[0052] Figure 2a is a visible light image schematic diagram in the embodiments of the application;

[0053] Figure 2b is an infrared image schematic diagram in the embodiments of the application;

[0054] Figure 2c is an initial estimated transmittance map in the embodiments of the application;

[0055] Figure 2d is a transmittance map optimized based on the visible light image in the embodiments of the application;

[0056] ​Figure 2e is Figure 2d is an enlarged view of the middle square part;

[0057] Figure 2f is a transmittance map optimized based on an infrared image in an embodiment of the present application;

[0058] Figure 2g is Figure 2f is an enlarged view of the middle square part;

[0059] Figure 3 is a flow chart of solving an infrared-guided de-scattering optimization model in an embodiment of the present application;

[0060] Figure 4a is a visible light image in a scattering scene one and an enlarged view thereof in an embodiment of the present application;

[0061] Figure 4b is an infrared image in a scattering scene one and an enlarged view thereof in an embodiment of the present application;

[0062] Figure 4c is a schematic diagram of a prior art DCP in a scattering scene one and an enlarged view thereof;

[0063] Figure 4d is a schematic diagram of a prior art Haze-lines in a scattering scene one and an enlarged view thereof;

[0064] Figure 4e is a schematic diagram of a prior art DLF in a scattering scene one and an enlarged view thereof;

[0065] Figure 4f is a schematic diagram of a prior art ResNet in a scattering scene one and an enlarged view thereof;

[0066] Figure 4g is a schematic diagram of de-scattering of a visible light image in a scattering scene one and an enlarged view thereof in an embodiment of the present application;

[0067] Figure 5a is a visible light image in a scattering scene two and an enlarged view thereof in an embodiment of the present application;

[0068] Figure 5b is an infrared image in a scattering scene two and an enlarged view thereof in an embodiment of the present application;

[0069] Figure 5c is a schematic diagram of a prior art DCP in a scattering scene two and an enlarged view thereof;

[0070] Figure 5d is a schematic diagram of a prior art Haze-lines in a scattering scene two and an enlarged view thereof;

[0071] Figure 5eis a schematic diagram of prior art DLF in scattering scene two and its enlarged view;

[0072] Figure 5f is a schematic diagram of prior art ResNet in scattering scene two and its enlarged view;

[0073] Figure 5g is a schematic diagram of visible light image de-scattering of the embodiment of the application in scattering scene two and its enlarged view; DETAILED DESCRIPTION

[0074] The application will be further described below with reference to the drawings and in conjunction with preferred embodiments. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0075] It should be noted that the left, right, up, down, top, bottom and the like in the embodiments are only relative concepts or are referenced to the normal use state of the product, and should not be considered as limiting.

[0076] Before introducing the embodiments of the application, the idea of the application is described as follows:

[0077] The prior art has the following defects: 1. The existing infrared enhancement algorithm does not consider the infrared radiation interference of the scattering medium itself and the attenuation interference of the scattering medium on light, and the enhancement result is unnatural; 2. A single infrared image is difficult to estimate the scene scattering information.

[0078] In view of the above defects of the prior art, the embodiments of the application consider the physical scattering imaging mechanism, and remove the infrared radiation interference of the scattering medium itself and the attenuation interference of the scattering medium on light; the system proposed by the embodiments of the application uses a visible light camera to obtain three-channel images, realizes scene scattering information estimation, and helps to realize infrared scattering model solving.

[0079] The embodiments of the application propose an infrared image guided visible light image de-scattering method. It fully utilizes the advantage that infrared images can be clearly imaged under a scattering scene, guides the visible light image based on the edge and regional contrast information of the infrared image, realizes clear color imaging of the visible light image under the scattering scene, i.e. visible light image de-scattering enhancement, by constructing and solving an infrared image guided de-scattering optimization model; the embodiments of the application guide the contrast enhancement of the visible light based on the regional contrast information of the infrared image; realize the transmittance map optimization and the edge preservation of the de-scattering result based on the edge of the infrared image; realize the color fidelity of the de-scattering result based on the gray world assumption, and finally realize the visible light image de-scattering enhancement. It has important significance for realizing clear color imaging through the scattering medium.

[0080] As Figure 1As shown, the embodiment of the present application proposes an infrared image guided visible light image despeckling method, and the steps are as follows:

[0081] S1: estimating a visible light transmittance map and an atmospheric light for the visible light image. The imaging model of the visible light in the scattering scene is as follows:

[0082] I VIS (x) = J VIS (x) t VIS (x) + A (1-t VIS (x) ) ; (1)

[0083] Wherein, I VIS (x) is the visible light image observed by the visible light camera in the scattering scene, t VIS (x) is the visible light transmittance map, A is the atmospheric light caused by the ambient light, J VIS (x) is the non-scattering visible light image, and x is the image coordinate.

[0084] The defogging algorithm estimates the image transmittance map t VIS and the atmospheric light A by using the statistical information of the R, G and B channels of the visible light. In the embodiment, the transmittance map t VIS (x) and the atmospheric light A are estimated by using the dark channel prior algorithm (DCP). It should be noted that the present application is not limited to using the DCP estimation, the color attenuation prior algorithm (CAP), the haze-line-based algorithm (Haze-lines) and other image defogging algorithms, and in a more preferred embodiment, a defogging algorithm with better performance can be used for transmittance map t VIS (x) and atmospheric light A estimation.

[0085] S2: using the infrared image to optimize the edges of the estimated visible light transmittance map;

[0086] Specifically, the edges of the estimated visible light transmittance map are optimized based on the infrared image, wherein the initial estimated transmittance map often needs to be optimized by the visible light image I VIS (x) to remove artifacts and refine the edges for further optimization.

[0087] In the thick fog scattering scene, the loss of edge information in the visible light image I VIS (x) will cause the optimization of the initial estimated transmittance map to fail, while the infrared image still has clear edges in the scattering scene. Therefore, the edges of the transmittance map can be optimized based on the edges of the infrared image, so that the edges of the transmittance map are clearer, thereby obtaining the optimized transmittance map t VIS-refined .

[0088] Therefore, with the guidance of the infrared edge information, the embodiment uses the infrared image to perform edge optimization on the estimated visible light transmittance map based on the guided filtering, and the expression is as follows:

[0089]

[0090] wherein GuidedFilter(·) represents a guided filtering algorithm, the output of the algorithm is the edge-optimized visible light transmittance map t VIS-refined , the input parameters are the initial estimated visible light transmittance map , the infrared image I IR , h g , and λ, h g is the window size of the guided filtering, and ε represents a regularization parameter.

[0091] As Figure 2a , Figures 2c-2e shown, the visible light image is used to optimize the transmittance map based on the guided filtering in a scattering scene. As Figure 2a shown, the visible light image and the initial estimated transmittance map as Figure 2c shown are used as input parameters of the guided filtering algorithm, and the transmittance map optimized based on the visible light image as Figure 2d and Figure 2e shown is obtained. The visible light image has a sharp decline in image clarity due to scattering interference, and the transmittance map optimized based on the visible light image is edge-blurred and cannot maintain the contour information.

[0092] As Figures 2b-2c , Figures 2f-2g shown, the infrared image is used to optimize the transmittance map based on the guided filtering. As Figure 2b shown, the infrared image and the initial estimated transmittance map as Figure 2c shown are used as input parameters of the guided filtering algorithm, and the transmittance map optimized based on the infrared image as Figure 2f and Figure 2g shown is obtained. Since the infrared image can be clearly imaged in a scattering scene, the transmittance map optimized based on the infrared image can remove the effective light halo and simultaneously transfer the edge information of the building in the infrared image. It should be noted that in the more preferred embodiment, an edge-preserving method including but not limited to a bilateral filtering algorithm, a guided filtering algorithm, and the like can be used to optimize the transmittance map guided by the infrared image.

[0093] S3: Extracting the edge information of the infrared image. The infrared image still has clear edges in the scattering scene. In order to obtain the edge information of the infrared image, many methods are used to extract the multi-scale and multi-direction edge information of the image. Therefore, in the embodiment, the methods including but not limited to Haar wavelet, log-Gabor filter, shearlet transform, gradient operator, etc. can be selected to extract the edge information of the infrared image. In the embodiment, the gradient operator is selected to extract the edge information of the infrared image, and the expression is as follows:

[0094]

[0095] wherein i represents the filter number, and in the gradient operator, 1≤i≤2, represents the horizontal direction gradient operator, represents the vertical direction gradient operator. In a more preferred embodiment, the edge representation methods such as Haar wavelet, log-Gabor filter, shearlet, etc. can be selected to represent the edge information of the infrared image.

[0096] S4: Establishing and solving the guided despeckling optimization model, outputting the despeckled visible light image, and realizing the despeckling of the visible light image. The infrared-guided visible light image despeckling can be represented as minimizing the following optimization problem, which is expressed as follows:

[0097]

[0098] wherein E(·) is the cost function of the optimization problem, J is the visible light image to be despeckled, and the despeckling optimization model includes the following four items: G(·) is the guided contrast enhancement item, which is based on the regional contrast information of the infrared image to guide the contrast enhancement of the visible light; H(·) is the atmospheric scattering model item of the optimized transmittance map t VIS-refined , so that the despeckling result conforms to the imaging model in the scattering scene; ED(·) is the image edge preservation item, which is based on the infrared image edge to realize the edge preservation of the despeckling result; CF(·) is the color fidelity item, which is based on the gray world assumption to realize the color fidelity of the despeckling result. The super parameters a, β, γ and δ represent the weights of each item in the cost function, and the values should all be greater than 0. In the embodiment, the super parameters are respectively set as a=0.4, β=0.3, γ=0.5, and δ=0.5.

[0099] The function of the guided contrast enhancement item G is to guide the enhancement of the image J based on the regional contrast information provided by the infrared image, which is expressed as:

[0100]

[0101] wherein the weight ω(x, y) between the pixel x and the pixel y is:

[0102]

[0103] where X denotes the set of all pixel positions of the image. J(x) denotes the intensity of the visible image J at pixel x, J(y) denotes the intensity of the visible image J at pixel y, R(x) denotes the intensity of the infrared image R at pixel x, R(y) denotes the intensity of the infrared image R at pixel y, I IR (x) denotes the intensity of the long-wave infrared image at pixel x. σ R and σ S are smoothing parameters.

[0104] H(·) is an atmospheric scattering model term based on the optimized transmittance map, based on the optimized transmittance map t VIS-refined , the de-scattering result conforms to the imaging model under the scattering scene, which is expressed as:

[0105]

[0106] where, denotes the square of the 2-norm, I is the visible light scattering image obtained by the visible camera, t VIS-refined is the visible light transmittance map optimized by the edge obtained in step S2, and A is the atmospheric light.

[0107] ED(·) is an edge preserving term based on the infrared image edge Edge i (I IR ), based on the infrared image edge, the edge preserving of the de-scattering result is realized, and its expression is as follows:

[0108]

[0109] where, denotes the square of the 2-norm, Edge i (I IR ) infrared image edge information is the infrared image edge information extracted in step S3, and Edge i (J) is the visible light image edge.

[0110] The color fidelity term CF(·) ensures that the de-scattering result is more natural and prevents serious color deviation. Referring to the gray world assumption, it is assumed that the average value of each channel in the local area of the defogging image represents the gray level. Its expression is as follows:

[0111]

[0112] where, denotes the square of the 2-norm, denotes that a mean filtering operation with a window size of h a is performed on the image J.

[0113] AsFigure 3 As shown, the descattering optimization model represented by Equation (4) can be solved iteratively by gradient descent by minimizing the following optimization problem, which includes the following steps:

[0114] S4-1. Acquire visible light images in a scattering scene (I) VIS Infrared images in scattering scenarios I IR k is the number of iterations, J k J is the result of the k-th iteration. k-1 This represents the result of the (k-1)th iteration, and J is initialized. 0 Visible light image I in a scattering scene VIS .

[0115] S4-2. Calculate the gradient term of the cost function E(·) of the optimization problem shown in formula (4).

[0116] S4-3, through formula Update the visible light image to obtain the current iteration result J k t is the gradient descent step size.

[0117] S4-4. Calculate the current iteration result J. k Compared with the result J of the previous iteration k-1 The root mean square error (RMSE);

[0118] S4-5, If RMSE(J k J k-1 If the value is less than the threshold ∈, then stop the iteration and output the descattered visible light image; otherwise, jump to the step of calculating the gradient term and continue the iteration.

[0119] The embodiments of the present invention have the following advantages: Since the edge information and regional contrast information of the infrared image are introduced in the descattering process, compared with the traditional visible light image dehazing method, the visible light image descattering method proposed in the embodiments of the present invention obtains more detailed information in the descattering result. Compared with the visible light and infrared image fusion method, this method does not directly fuse pixel intensity to avoid image distortion, and uses infrared information to achieve a more effective and natural descattering effect.

[0120] Experimental example:

[0121] This experimental example tests the method using visible light and infrared images under two scattering scenarios, and compares it with visible light image dehazing algorithms (DCP, Haze-lines) and visible light / infrared image fusion algorithms (DLF, ResNet) to verify the effectiveness of the proposed method. In this experimental example, the hyperparameters α, β, γ, δ, step size t, and weight smoothing parameter σ are used. R and σ S , guided filter parameter hg and ε and mean filter parameter h a are fixed as 0.4, 0.3, 0.5, 0.5, 0.3, 0.3, 250, 50, 0.01 and 600 respectively. The initial transmittance map of the visible image estimated by the Haze-lines method. The optimization solving process is performed on the three channels of the visible image respectively. The threshold value ε is 0.005, that is, when the root mean square error between the iteration result J k and the last iteration result J k-1 is less than 0.005, it is considered that the optimization problem converges, and the de-scattering visible image is output.

[0122] Figure 4a -g and Figure 5a -g respectively show the visible image, the infrared image, the visible image de-fogging algorithm (DCP, Haze-lines) processed image and the visible / infrared image fusion algorithm (DLF, ResNet) processed image of the scattering scene 1 and the scattering scene 2, and specifically show the processed image of the visible image de-scattering method and the comparative algorithm proposed in the embodiment of the present application. The subjective visual quality of the algorithm processing result shows that the method in the embodiment performs well in the scattering scene. Compared with the visible image de-fogging algorithm, the method in the embodiment fuses the edge information of the infrared to obtain better detail information, as shown in the enlarged detail. Compared with the visible / infrared image fusion algorithm, the method does not directly fuse the intensity of the image pixels. Considering the difference in imaging principle between the long-wave infrared and the visible light and the interference of the scattering scene on the imaging process, the method establishes an optimization problem, uses the edge information of the long-wave infrared and the regional contrast information for guidance to improve the image contrast and make the edge more obvious, so that a better de-scattering effect is achieved.

[0123] Compared with the prior art, the embodiment of the present application has the following advantages:

[0124] 1. Unlike the existing infrared enhancement algorithm, the embodiment of the present application considers the infrared radiation interference of the scattering medium itself and the light attenuation interference of the scattering medium, so that the processing result is more consistent with the physical imaging mechanism and distortion is avoided.

[0125] 2. The embodiment of the present application realizes the estimation of the scene scattering information by introducing a visible light camera. The visible light camera has a low cost, and the method has a low requirement on the resolution of the visible light camera, so the method is low in cost and has strong popularization value.

[0126] The above is further detailed description of the present application in combination with specific preferred embodiments, and cannot be deemed as limitation of the specific implementation of the present application to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of equivalent substitutions or obvious variations can be made, and the performance or use is the same, which should be deemed as falling within the protection scope of the present application.

Claims

1. A method of visible light image de-scattering, characterized in that, The method comprises the following steps: S1, estimating a visible light transmittance map and atmospheric light from a visible light image; S2, performing edge optimization on the estimated visible light transmittance map using an infrared image; S3, extracting edge information of the infrared image; S4, establishing and solving a despeckling optimization model to output a despeckled visible light image; In step S4, the established despeckling optimization model comprises a guided contrast enhancement term G(·), a transmittance map optimized atmospheric scattering model term H(·), an image edge preservation term ED(·), and a color fidelity term CF(·). The guided contrast enhancement term G(·) is expressed as: The weight ω(x, y) between pixels x and y is expressed as: where X represents the set of all pixel positions of the image; J(x) represents the intensity of the visible light image J at pixel x, J(y) represents the intensity of the visible light image J at pixel y, R(x) represents the intensity of the infrared image R at pixel x, R(y) represents the intensity of the infrared image R at pixel y, σ R and σ S are smoothing parameters; The transmittance map optimized atmospheric scattering model term H(·) is expressed as: wherein, represents the square of the 2-norm, I VIS is a visible light scattering image acquired by a visible camera, t VIS-refined is a visible light transmittance map obtained by performing edge optimization in step S2, and A is atmospheric light; The image edge preservation term ED(·) is expressed as: wherein, represents the square of the 2-norm, Edge i (I IR ) is the infrared image edge information extracted in step S3, Edge i (J) is the visible light image edge; The color fidelity term CF(·) is expressed as: wherein, denotes the square of the 2-norm, denotes a mean filtering operation with window size h a performed on the image J; In step S4, the solving of the despeckling optimization model is expressed as minimizing the following optimization problem: Wherein, E(·) is a cost function of the optimization problem, J is a visible light image to be despeckled, G(·) is a guided contrast enhancement term, H(·) is a transmittance map optimized atmospheric scattering model term, ED(·) is an image edge preservation term, CF(·) is a color fidelity term, and the hyperparameters α, β, γ, and δ represent the weights of the terms in the cost function, all of which are greater than 0. The solving method of the despeckling optimization model expressed as minimizing the following optimization problem comprises the following steps: S4-1, obtaining a visible light image I under a scattering scene VIS and an infrared image I under a scattering scene IR k is an iteration number, J k is a result of the kth iteration, J k-1 denotes a result of the (k-1)th iteration, J 0 is initialized as a visible light image I under a scattering scene VIS ; S4-2, calculating the gradient term ▽E of the cost function E(·) of the optimization problem; S4-3, by formula J k = J k-1 -t - DE update the visible light image to obtain the current iteration result J k wherein DE is a gradient term of a cost function E(·) of the optimization problem, and t is a gradient descent step. S4-4, compute current iteration result J k root mean square error RMSE of the previous iteration result J k-1 root mean square error RMSE of the previous iteration result J S4-5, if the RMSE(J k ,J k-1 ) is less than a threshold value, then stop the iteration and output the de-speckled visible light image, otherwise jump to step S4-2 to calculate the gradient term ▽E and continue the iteration.

2. The method of visible light image de-scattering of claim 1, wherein, In step S1, the visible light image in a scattering scene is expressed as: I VIS (x) = J VIS (x)t VIS (x) + A(1 - t VIS (x)) where I VIS is the visible light image acquired by the visible light camera under a scattering scene, J VIS is the non-scattering visible light image, A represents the atmospheric light caused by the ambient light, t VIS (x) is the visible light transmittance map, x is the image coordinate.

3. The method of visible light image de-scattering according to claim 1 or 2, wherein, The method for estimating a visible light transmittance map and atmospheric light comprises a dark channel prior algorithm DCP, a color attenuation prior algorithm CAP, and a haze-line based algorithm Haze-lines.

4. The method of visible light image de-scattering of claim 1, wherein, In step S2, the edge optimization on the estimated visible light transmittance map using an infrared image is expressed as: where GuidedFilter(·) denotes a guided filter algorithm, and the output is the edge-optimized visible light transmittance map t VIS-refined , with input parameters being the initial estimated visible light transmittance map , the infrared image I IR , h g , and λ, h g is the window size of the guided filter, and ε represents a regularization parameter.

5. The method of visible light image de-scattering of claim 1, wherein, In step S3, the extraction method of the edge information of the infrared image comprises Haar wavelet, log-Gabor filter, shearlet, and gradient operator. Wherein, the extraction of the edge information of the infrared image using the gradient operator is expressed as: Edge i (I IR )=▽ i I IR ; Wherein, i represents the filter number, in the gradient operator, 1≤i≤2, ▽1 represents a horizontal direction gradient operator, and ▽2 represents a vertical direction gradient operator.

6. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 5. The computer program is executed by a processor to implement the steps of the method according to any one of claims 1-5.

Images