Night image defogging method and device based on local block atmospheric light and color migration

Abstract

The invention provides a night image defogging method and device based on atmospheric light and color migration of local blocks, and the method comprises the steps: decomposing an input image into three channels in an RGB color space, and calculating the atmospheric light of each local block in each channel; atmospheric light for each local block is optimized in combination with brightness and spatial location. A depth map set is estimated in combination with the input image and the optimized atmospheric light. And carrying out weighted fusion on the depth image group, and carrying out multi-scale detail enhancement on a fused image to obtain a defogged grayscale image. And based on the image features of the input image, migrating the color of the input image to the defogged grey-scale image to obtain a defogged color image. The method breaks through the limitation of global constancy of beta values, and adopts a plurality of beta values to solve the depth map group. And selecting the depth map corresponding to one beta value as a reference to normalize the brightness of other depth maps in the depth map group. The method provided by the invention aims at defogging a single-frame image, does not depend on mass data, is efficient in calculation and is suitable for a complex haze scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a night image defogging method and device based on local block atmospheric light and color migration. BACKGROUND

[0002] Under the condition of fog, haze and other bad weather, due to the scattering effect of suspended particles in the atmosphere, the contrast of the image is reduced, the color is distorted, and the effect of the subsequent visual task is seriously affected. Therefore, image defogging technology is needed to eliminate the influence of fog and haze on image quality and improve the contrast and clarity of the image.

[0003] The existing defogging methods mainly include physical model-based and deep learning methods, but there are the following problems: (1) The physical model-based method such as dark channel prior and fog line prior depends on strong assumptions and is easy to fail in complex scenes. (2) The machine learning-based method needs a large amount of labeled data, and has poor generalization ability in real scenes, which may have overfitting problem and large hardware resource consumption.

[0004] In summary, there is an urgent need for a single-frame image defogging method that does not rely on massive data, is computationally efficient, and is suitable for complex fog and haze scenes. SUMMARY

[0005] To overcome the problems in the related art, the purpose of the present application is to provide a night image defogging method and device based on local block atmospheric light and color migration, which is suitable for single-frame image defogging, does not rely on massive data, is computationally efficient, and is suitable for complex fog and haze scenes.

[0006] The night image defogging method based on local block atmospheric light and color migration comprises: Obtaining an input image, and decomposing the input image into three channels in the RGB color space; Calculating the atmospheric light of each local block in each channel; Optimizing the atmospheric light of each local block in combination with brightness and spatial position to obtain optimized atmospheric light; Combining the input image and the optimized atmospheric light to estimate a depth map group; Weighted fusion of the depth map group to obtain a fused image; Multi-scale detail enhancement of the fused image to obtain a defogging grayscale image; Color migration of the input image to the defogging grayscale image based on the image features of the input image to obtain a defogging color image.

[0007] In the preferred technical solution of the present application, before calculating the atmospheric light of each local block in each channel, it further comprises: establishing an atmospheric scattering model, an input of the atmospheric scattering model including the transmittance, and an output of the atmospheric scattering model being the input image; modeling the attenuation of the transmittance to obtain a transmittance expression; substituting the transmittance expression into the atmospheric scattering model to obtain an adjusted atmospheric scattering model.

[0008] In the preferred technical solution of the present application, the calculation of the atmospheric light of each local block in each channel includes: based on the adjusted atmospheric scattering model, a first inequality relationship between the atmospheric light and the input image is established according to the following formula: ; wherein, is the cth channel of the input image, is the maximum value of the cth channel of the haze-free image, is the minimum value of the cth channel of the haze-free image, is the cth channel of the atmospheric light; if c=1, it represents the R channel; if c=2, it represents the G channel; if c=3, it represents the B channel; the maximum value and the minimum value of the haze-free image are estimated in the local block: ; ; wherein, avg is an average value operation, x is the pixel point serial number in the cth channel of the input image, p i is the ith local block; combining the first inequality relationship, the maximum value of the cth channel of the haze-free image, and the minimum value of the cth channel of the haze-free image, the atmospheric light of the cth channel of each local block is calculated: ; wherein, is the atmospheric light of the cth channel of the ith local block.

[0009] In the preferred technical solution of the present application, the optimization of the atmospheric light of each local block in combination with the brightness and the spatial position to obtain the optimized atmospheric light includes: the following formula is used to calculate the corresponding optimized atmospheric light according to the atmospheric light of each local block: ; ; ; ; wherein, the atmospheric light of the i-th local block of the c-th channel, the atmospheric light of the i-th local block of the c-th channel, the c-th channel of the input image, f is an atmospheric light optimization function, smooth is a smoothing function, and p is an argument of the smoothing function; d total the total distance, which is used to represent the brightness and spatial distance of the input image, l min the minimum distance, l max the maximum distance.

[0010] In the preferred technical solution of the present application, the combination of the input image and the optimized atmospheric light estimates a depth map set, comprising: traversing all haze concentrations; According to each haze concentration, the corresponding depth map is calculated using the following formula: wherein, the i-th depth map of the c-th channel, the haze concentration; all depth maps are combined into a depth map set.

[0011] In the preferred technical solution of the present application, the depth map set is weighted and fused to obtain a fused image, comprising: selecting a depth map from the depth map set as a brightness reference map; According to the brightness reference map, the brightness of other depth maps in the depth map set is normalized to obtain a plurality of normalized depth maps; Based on the normalized depth map, the corresponding fusion weight is calculated; Using the fusion weight, all the normalized depth maps are weighted and summed at the pixel level to obtain a fused image.

[0012] In the preferred technical solution of the present application, the fused image is subjected to multi-scale detail enhancement to obtain a dehazed grayscale image, comprising: Using different standard deviations, the fused image is subjected to multiple Gaussian filtering to obtain total different scale blur maps; wherein total≥2; Subtracting the k+1th blur map from the kth blur map, a plurality of scale details are obtained; wherein 1≤k≤total-1; Statistical local brightness of the blur map, and calculating the corresponding adaptive gain mapping according to each of the local brightness; Divide the adaptive gain mapping by the number of channels of the input image to obtain a scale gain; Using the scale gain, all the scale details are weighted and summed to obtain a fused detail.​ The fused image is then combined with the fused details to obtain the enhanced fused image; The fused image and the enhanced fused image are weighted and summed to obtain a dehazed grayscale image.

[0013] In a preferred embodiment of the present invention, the step of transferring the color of the input image to the dehazed grayscale image based on the image features of the input image to obtain a dehazed color image includes: Calculate the mean values ​​of the red-green color channels and the mean values ​​of the yellow-blue color channels of the input image; Calculate the standard deviation of the red-green color channel and the standard deviation of the yellow-blue color channel of the input image; The neutral deviation of the input image is calculated based on the mean values ​​of the red-green color channels and the mean values ​​of the yellow-blue color channels; Divide the neutral deviation by the adjustment coefficient to obtain the adjusted neutral deviation; use the minimum value between the preset compensation coefficient and the adjusted neutral deviation as the compensation coefficient; The negative of the product of the mean value of the red and green color channels of the input image and the compensation coefficient is used as the adjustment amount of the red and green color channels, and the negative of the product of the mean value of the yellow and blue color channels of the input image and the compensation coefficient is used as the adjustment amount of the yellow and blue color channels. Superpixel segmentation is performed on the dehazed grayscale image to obtain the first superpixel image; The red and green color channels are adjusted to compensate each superpixel in the first superpixel image to obtain the compensated red and green color channels. The yellow and blue color channels are adjusted to compensate each superpixel in the first superpixel image to obtain the compensated yellow and blue color channels. Using the L channel in the Lab color space of the dehazed grayscale image, guided filtering is performed on the compensated red-green color channel and the compensated yellow-blue color channel to obtain a dehazed color image.

[0014] In a preferred embodiment of the present invention, the smog concentration is 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 0.99, and 1.

[0015] A nighttime image dehazing device based on local patch atmospheric light and color migration includes: The channel decomposition module is used to acquire the input image and decompose the input image into three channels in the RGB color space; Atmospheric light calculation module, used to calculate atmospheric light for each local block in each channel; The atmospheric light optimization module is used to combine brightness and spatial location to optimize the atmospheric light of each local block, resulting in optimized atmospheric light. A depth map group estimation module is used to estimate a depth map group by combining the input image and the optimized atmospheric light. The depth map group fusion module is used to perform weighted fusion of the depth map groups to obtain a fused image; A multi-scale detail enhancement module is used to perform multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image; The color transfer module is used to transfer the colors of the input image to the dehazed grayscale image based on the image features of the input image, so as to obtain a dehazed color image.

[0016] The beneficial effects of this invention are as follows: This invention provides a nighttime image dehazing method based on local patch atmospheric light and color transfer. The method involves acquiring an input image. Color is a crucial factor in human image perception; directly using a grayscale image for this algorithm would lose some color harmony, negatively impacting subsequent colorization and image correction quality. Therefore, this invention decomposes the input image into three channels in the RGB color space to fully utilize its color information. Based on derived prior knowledge, atmospheric light is calculated for each local patch in each channel. Combining brightness and spatial location, the atmospheric light of each local patch is optimized to obtain optimized atmospheric light. This invention overcomes the limitation of globally constant β values ​​by using multiple β values ​​to solve for depth map groups. A depth map corresponding to a selected β value is used as a benchmark to normalize the brightness of other depth maps in the depth map group. Fusion weights are calculated, and the depth map group is weighted and fused at the pixel level to obtain a fused image. Multi-scale detail enhancement is performed on the fused image to obtain a dehazed grayscale image. This invention linearly fuses the fused image and the enhanced fused image, enhancing details at multiple scales while avoiding over-enhancement, thus preserving more natural characteristics of the real scene. Based on the image features of the input image, the colors of the input image are transferred to a dehazed grayscale image to obtain a dehazed color image, thus completing the colorization of the grayscale image and solving the color distortion problem of the original image. The method provided by this invention is for dehazing single-frame images, does not rely on massive amounts of data, is computationally efficient, and is applicable to complex haze scenarios. Attached Figure Description

[0017] Figure 1 This is a flowchart of the nighttime image dehazing method based on local block atmospheric light and color migration according to the present invention; Figure 2 This is a flowchart of the weighted fusion of depth map groups according to the present invention; Figure 3 This is a flowchart of the multi-scale detail enhancement of fused images according to the present invention; Figure 4 This is the first input image containing smog in this invention; Figure 5 yes Figure 4The result image after dehazing using the nighttime image dehazing method provided by this invention; Figure 6 This is the second input image containing smog in this invention; Figure 7 yes Figure 6 The image shows the result after dehazing using the nighttime image dehazing method provided by this invention. Detailed Implementation

[0018] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0019] Example 1 like Figure 1 As shown, this embodiment provides a nighttime image dehazing method based on local patch atmospheric light and color migration, including: S1: Obtain the input image and decompose the input image into three channels in the RGB color space; S2: Calculate the atmospheric light for each local block in each channel; S3: Combining brightness and spatial location, optimize the atmospheric light of each local block to obtain the optimized atmospheric light; S4: Combine the input image and the optimized atmospheric light to estimate the depth map group; S5: Perform weighted fusion on the depth map group to obtain a fused image; S6: Perform multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image; S7: Based on the image features of the input image, transfer the colors of the input image to the dehazed grayscale image to obtain a dehazed color image.

[0020] The input image is an RGB image, i.e., a color image. The input image is split into R, G, and B channels: R (red), G (green), and B (blue). Relevant features are calculated within each channel, fully utilizing the color information from different channels of the input image.

[0021] Before calculating the atmospheric light for each local block in each channel, the method further includes: S21': Establish an atmospheric scattering model, wherein the input of the atmospheric scattering model includes transmittance, and the output of the atmospheric scattering model is the input image; S22': Perform attenuation modeling on the transmittance to obtain the transmittance expression; S23': Substitute the transmittance expression into the atmospheric scattering model to obtain the adjusted atmospheric scattering model.

[0022] In the field of computer vision, the atmospheric scattering model is defined as follows: (1) Where I(x) is the input image, and A(x) is the global atmospheric light. For a clear image, t(x) represents the transmittance, and x represents the pixel index. The transmittance is modeled using the McCarney attenuation model as follows: (2) Where e is an exponential function, is the transmission attenuation coefficient, dis(x) is the transmission distance, and formula (2) represents the nonlinear relationship between the transmittance of the x-th pixel and the transmittance distance of the x-th pixel.

[0023] Expanding equation (2) to the second order using Taylor's formula, we get: (3) In this invention, . represents multiplication.

[0024] Substituting equation (3) into equation (1) of the atmospheric scattering model, we get: (4) Converting formula (4) into a quadratic function, we get formula (5), where, It is a quadratic term. J(x) - I(x) is a linear term, and J(x) = I(x) is a constant term. A(x).

[0025] (5) Solving formula (5) using the root-finding formula, i.e., Vieta's formulas, and discarding negative values, yields: (6) For formula (5) in the form of a quadratic function, a pattern is observed: the axis of symmetry of this quadratic function always lies in... The position, expressed mathematically, is: (7) At the position of the axis of symmetry, we have: (8); in, Let x be the x-coordinate of the axis of symmetry. Let y be the ordinate of the axis of symmetry. At the position of the axis of symmetry, the above quadratic function reaches its maximum value. max is the operation for finding the maximum value.

[0026] Substituting formula (8) into formula (1), we get: (9) Among them, I max This represents the maximum value of the input image.

[0027] In summary, this invention derives the following by Taylor expansion of the McCatney attenuation model: (10) This invention models the pixel values ​​of each local block in each channel of the input image as a quadratic function, with the horizontal axis being... The vertical axis is The significance of this prior is that there always exists a point within the local block whose x-coordinate is the median of the corresponding quadratic function and whose y-coordinate is the maximum value of the corresponding quadratic function. There exists a point with a x-coordinate... And the ordinate I max It is equal to the maximum value of the local block pixel value. The upper part of formula (10) is the prior formula for solving the depth map group later, and the lower part of formula (10) is the prior formula for solving atmospheric light.

[0028] The calculation of atmospheric light for each local block in each channel includes: Based on the adjusted atmospheric scattering model, a first inequality relationship between atmospheric light and the input image is established according to the following formula: (11) in, This refers to the c-th channel of the input image. Let be the maximum value of the c-th channel of the haze-free image. Let be the minimum value of the c-th channel of the haze-free image. This represents the c-th channel of atmospheric light; if c=1, it represents the R channel; if c=2, it represents the G channel; if c=3, it represents the B channel. Estimate the maximum and minimum values ​​of the haze-free image in local blocks: (12) (13) Where avg is the average value operation, x is the pixel index in the c-th channel of the input image, and p i For the i-th local block; It can be rewritten as , , , Here, a filtering mechanism is implemented. For a local block, when calculating the mean, this invention does not use all pixels for calculation. Instead, a threshold is set, and pixels above the threshold are set as valid pixels.

[0029] Combining the first inequality relationship, the maximum value of the c-th channel of the haze-free image, and the minimum value of the c-th channel of the haze-free image, calculate the atmospheric light in the c-th channel of each local block: (14) (15) in, For the atmospheric light of the i-th local block on the c-th channel, avg in formula (14) is the average value calculation, combined with the above The filtering process involves averaging all valid pixels in the local block.

[0030] use The atmospheric light A(x) of a local patch centered at point x can be estimated, where J refers to the dehazed or hazy image. The above estimate is the maximum value J of the input image with haze after dehazing. max It should not exceed the maximum value I of the input image. max The above conclusion also applies to each channel, that is, the maximum value of each channel after the input image is dehazed. It should not exceed the maximum value of the corresponding channel of the input image, that is... The reasoning is that the presence of haze in the scene increases the overall brightness of the image. Therefore, the maximum brightness of the image after dehazing is no greater than the maximum brightness of the image before dehazing. This conclusion also applies to local blocks.

[0031] Perform the above operation on each channel of the input image according to the R channel, G channel and B channel to obtain the atmospheric light of each local block in each channel, that is, three atmospheric lights, each atmospheric light corresponding to one channel.

[0032] This embodiment provides a nighttime image dehazing method based on local patch atmospheric light and color migration. The method involves acquiring an input image. Color is a crucial factor in human image perception; directly using a grayscale image for this algorithm would lose some color harmony, negatively impacting subsequent colorization and image correction quality. Therefore, this invention decomposes the input image into three channels in the RGB color space to fully utilize its color information. Based on derived prior knowledge, atmospheric light is calculated for each local patch in each channel. Combining brightness and spatial location, the atmospheric light of each local patch is optimized to obtain optimized atmospheric light. This invention overcomes the limitation of globally constant β values ​​by using multiple β values ​​to solve for depth map groups. A depth map corresponding to a selected β value is used as a benchmark to normalize the brightness of other depth maps in the depth map group. Fusion weights are calculated, and the depth map group is weighted and fused at the pixel level to obtain a fused image. Multi-scale detail enhancement is performed on the fused image to obtain a dehazed grayscale image. This invention linearly fuses the fused image and the enhanced fused image, enhancing details at multiple scales while avoiding over-enhancement, thus preserving more natural characteristics of the real scene. Based on the image features of the input image, the colors of the input image are transferred to a dehazed grayscale image to obtain a dehazed color image, thus completing the colorization of the grayscale image and solving the color distortion problem of the original image. The method provided by this invention is for dehazing single-frame images, does not rely on massive amounts of data, is computationally efficient, and is applicable to complex haze scenarios.

[0033] Example 2 like Figure 1 As shown, this embodiment provides a nighttime image dehazing method based on local patch atmospheric light and color migration. This embodiment describes the differences from Embodiment 1, based on Embodiment 1. The method includes: S1: Obtain the input image and decompose the input image into three channels in the RGB color space; S2: Calculate the atmospheric light for each local block in each channel; S3: Combining brightness and spatial location, optimize the atmospheric light of each local block to obtain the optimized atmospheric light; S4: Combine the input image and the optimized atmospheric light to estimate the depth map group; S5: Perform weighted fusion on the depth map group to obtain a fused image; S6: Perform multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image; S7: Based on the image features of the input image, transfer the colors of the input image to the dehazed grayscale image to obtain a dehazed color image.

[0034] The process of combining brightness and spatial location to optimize the atmospheric light of each local block yields optimized atmospheric light, including: The optimized atmospheric light is calculated based on the atmospheric light of each local block using the following formula: (16) (17) (18) (19) in, Optimized atmospheric light for the i-th local block of the c-th channel. For the atmospheric light of the i-th local block in the c-th channel, For the c-th channel of the input image, Given a grayscale image, f is the atmospheric light optimization function, smooth is the smoothing function, and p is the independent variable of the smoothing function; d total The total distance is used to characterize the brightness and spatial distance of the input image. min For the minimum distance, l max This represents the maximum distance.

[0035] The formula for calculating the total distance is as follows: (20) ;(twenty one) ;(twenty two) This represents the average value of a local block within a channel. This represents the spatial distance, specifically the distance between the center coordinates of the local block and its boundary. W represents the width of the local block, H represents the height of the local block, xo represents the x-coordinate of the center point of the local block, and yo represents the y-coordinate of the center point of the local block.

[0036] After normalizing the total distance, the normalized result is used as the independent variable and input into the smoothing function. The smoothing function is a cubic function that monotonically decreases when the distance is less than 0 and first increases and then decreases when the distance is greater than 0. When the independent variable is greater than 0, the smoothing function reaches its maximum value at x=1. In the atmospheric light estimation algorithm of this invention, the independent variable of the smoothing function is 0≤p≤1.

[0037] An optimized atmospheric light was calculated using the R, G, and B channels. Each channel's optimized atmospheric light is a single value; therefore, the final optimized atmospheric light is a vector composed of three values, each corresponding to one channel. The basis for this operation is that color is a crucial factor in human image perception. Directly using grayscale images results in the loss of color information, leading to poor overall image harmony and hindering the colorization and quality improvement of grayscale images.

[0038] For the optimized atmospheric light of the i-th local block in the c-th channel, this invention assumes that the atmospheric light is locally constant, and that each local block in an input image is... When combined, they form the atmospheric light of the entire image.

[0039] The step of combining the input image and the optimized atmospheric light to estimate the depth map set includes: S41: Traverse all smog concentrations; S42: Calculate the corresponding depth map based on each haze concentration using the following formula: ;(twenty three) in, For the i-th depth map of the c-th channel, The concentration of smog; S43: Combine all depth maps into a depth map group.

[0040] Many previous traditional methods will The value is set to be globally constant, but we analyze a real haze image where different degrees of haze exist. The values ​​are not globally constant, so different haze values ​​are designed and different depth maps are generated to obtain a group of depth maps, with haze concentrations of 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 0.99 and 1.

[0041] like Figure 2 As shown, the weighted fusion of the depth map groups to obtain the fused image includes: S51: Select one depth map from the depth map group as the brightness reference map; S52: Based on the brightness reference map, normalize the brightness of other depth maps in the depth map group to obtain multiple normalized depth maps; S53: Calculate the corresponding fusion weights based on the normalized depth map; S54: The fusion weights are used to perform pixel-level weighted summation on all the normalized depth maps to obtain the fused image.

[0042] To map the brightness of local blocks to the sensitive response range of the Sigmoid function and avoid low weight discrimination due to concentrated brightness, a suitable image needs to be selected from the depth map set as the brightness reference image. This embodiment selects... Taking a depth map with a brightness ratio of 0.7 as an example, brightness feature normalization is performed on other depth maps. The brightness of a depth map is closely related to the distance data it represents. The brightness of each pixel in the depth map corresponds to the distance of objects in the scene from the camera. A linear or non-linear mapping is typically used. Near objects (objects with small distances) are displayed with high brightness, such as light gray or white, while distant objects (objects with large distances) are displayed with low brightness, such as dark gray or black. The following formula is used to normalize the depth maps in the depth map group, excluding the brightness baseline map: ;(twenty four) in, This is the brightness bias coefficient. For the brightness gain coefficient, in this embodiment, mid=0.5. Taking a value of 10 as an example, the same set of mid and mid is used for all depth maps. L base This is a brightness reference diagram. For the k-th normalized depth map, the same set of mid and mid values ​​is used for the entire depth map. .

[0043] Calculation of fusion weights for a single normalized depth map: (25) Among them, w k Let be the k-th fusion weight, e be the exponential function, and k be the index of the fusion weight.

[0044] The following formula is used to perform a pixel-level weighted summation of the depth map group: (26) Where fused represents the fused image, and num represents the total number of depth maps contained in the depth map group.

[0045] This embodiment overcomes the limitation of a globally constant β value by employing multiple β values ​​and Vieta's theorem to solve for the depth map group. The range of β values ​​is 0.7-1, and the depth map with β=0.70 is selected as the brightness reference map to normalize the brightness. After calculating the fusion weights, pixel-level weighted fusion is performed on the depth map group to obtain the fused image.

[0046] Example 3 This embodiment provides a nighttime image dehazing method based on local patch atmospheric light and color migration. This embodiment describes the differences from Embodiment 1, and the method includes: S1: Obtain the input image and decompose the input image into three channels in the RGB color space; S2: Calculate the atmospheric light for each local block in each channel; S3: Combining brightness and spatial location, optimize the atmospheric light of each local block to obtain the optimized atmospheric light; S4: Combine the input image and the optimized atmospheric light to estimate the depth map group; S5: Perform weighted fusion on the depth map group to obtain a fused image; S6: Perform multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image; S7: Based on the image features of the input image, transfer the colors of the input image to the dehazed grayscale image to obtain a dehazed color image.

[0047] like Figure 3 As shown, the multi-scale detail enhancement of the fused image to obtain a dehazed grayscale image includes: S61: Apply Gaussian filtering to the fused image multiple times using different standard deviations to obtain total blur images of different scales; wherein, total ≥ 2; S62: Subtract the (k+1)th blurred image from the kth blurred image to obtain multiple scale details; where 1≤k≤total-1; S63: Statistically determine the local brightness of the blurred image, and calculate the corresponding adaptive gain mapping based on each local brightness; S64: Divide the adaptive gain mapping by the number of channels of the input image to obtain the scale gain; S65: The scale gain is used to perform a weighted summation of all the scale details to obtain the fused details; S66: Add the fused details to the fused image to obtain the enhanced fused image; S67: Perform a weighted summation on the fused image and the enhanced fused image to obtain a dehazed grayscale image.

[0048] The following formula is used to extract multiple blurred images: (27) Among them, blurred s Let represent the s-th blurred image, imgaussfilt represent Gaussian filtering, and fused represent the fused image. This represents the s-th standard deviation, when s=1. =0.5; when s=2, =1; when s=3 =2. This embodiment takes total=3 as an example, meaning it contains 3 blur images. The smoothing factor of the k-th blur image is less than that of the (k+1)-th blur image, meaning the k-th blur image contains more details and textures than the (k+1)-th blur image. Three scale details are calculated: the first scale detail is the first blur image minus the second blur image, the second scale detail is the first blur image minus the third blur image, and the third scale detail is the second blur image minus the third blur image. The number of blur images, the number of scale details, and the number of channels in the input image are all the same, 3.

[0049] The fused image is the result of fusing images of different depths. Gaussian blurring is performed on the fused image using different standard deviations to obtain multiple blurred images. These multiple blurred images form the Gaussian pyramid of the fused image.

[0050] The adaptive gain mapping is calculated using the following formula: (28) Where gainmap represents adaptive gain mapping, localbrightness represents local brightness, localbrightness is the adjustment of each element in the two-dimensional matrix fused to the interval [0,1], and localbrightness is a two-dimensional matrix.

[0051] Since a single-channel image ignores the overall color harmony of the scene and affects the coloring of the grayscale image, a color depth map is calculated based on a three-channel color atmospheric light map, combining Examples 1 and 2. The three-channel color depth map is then converted into a single-channel grayscale image, and formula (28) processes the single-channel image.

[0052] (29) (30) Where, diff1 = blurred1 - blurred2, diff1 represents the first scale detail, diff2 = blurred1 - blurred3, diff2 represents the second scale detail, and diff3 = blurred2 - blurred3, diff3 represents the third scale detail. "Enhanced" represents the enhanced fused image, and "fused" represents the fused image. The fused image is a single-channel image, represented as a two-dimensional matrix. s Indicates the s-th scale. s This represents the gain at the s-th scale. Each scale corresponds to a scale gain, which is determined by the scale. s Scaling and diffing the gainmap s This represents the s-th scale detail.

[0053] (31) Here, finalenhanced represents the dehazed grayscale image. The fusion coefficient is represented in this embodiment. Taking a value of 0.8 as an example, using linear fusion can avoid over-enhancement, allowing the image to retain more of the natural characteristics of the real scene.

[0054] The step of transferring the colors of the input image to the dehazed grayscale image based on the image features of the input image to obtain a dehazed color image includes: S71: Calculate the mean values ​​of the red-green color channels and the yellow-blue color channels of the input image; S72: Calculate the standard deviation of the red-green color channel and the standard deviation of the yellow-blue color channel of the input image; S73: Calculate the neutral deviation of the input image based on the mean values ​​of the red-green color channels and the mean values ​​of the yellow-blue color channels; S74: Divide the neutral deviation by the adjustment coefficient to obtain the adjusted neutral deviation; use the minimum value between the preset compensation coefficient and the adjusted neutral deviation as the compensation coefficient; S75: The negative of the product of the mean value of the red-green color channel and the compensation coefficient of the input image is used as the adjustment amount of the red-green color channel, and the negative of the product of the mean value of the yellow-blue color channel and the compensation coefficient of the input image is used as the adjustment amount of the yellow-blue color channel. S76: Perform superpixel segmentation on the dehazed grayscale image to obtain the first superpixel image; S77: Compensate each superpixel in the first superpixel image by adjusting the red and green color channels to obtain the compensated red and green color channels; S78: The yellow-blue color channel adjustment amount is used to compensate each superpixel in the first superpixel image to obtain the compensated yellow-blue color channel; S79: Using the L channel in the Lab color space of the dehazed grayscale image, perform guided filtering on the compensated red-green color channel and the compensated yellow-blue color channel to obtain a dehazed color image.

[0055] This embodiment uses the following steps to convert the input image from the RGB color space to the Lab color space: (1) RGB to XYZ: Normalize and inverse gamma correct the RGB values, and convert them to XYZ values ​​through 3×3 matrix multiplication. (2) XYZ normalization: Divide the XYZ values ​​by the reference white point to achieve normalization. (3) XYZ to Lab: Calculate the values ​​of L, a, and b through a piecewise function.

[0056] The RGB color space is based on the principle of device light emission, while the Lab color space is more in line with the characteristics of human eye perception. In the Lab color space, the L channel represents brightness, the a channel is used to represent red and green tones, and the b channel is used to represent yellow and blue tones.

[0057] The formula for calculating the neutral deviation is as follows: (32) Among them, D neutral Indicates the degree of neutral deviation, i.e., the degree of color cast, u a u represents the mean value of channel a (red and green) in Lab color space. b This represents the mean value of the b channel (yellow and blue) in Lab color space.

[0058] Superpixel segmentation reduces the number of basic units in image processing by clustering pixels into regions with similar features, while preserving key structural information, thereby improving the efficiency of color compensation and transfer. A superpixel is a perceptual unit between a pixel and a complete object, composed of pixels with similar color and texture, typically ranging in size from tens to hundreds of pixels. Since the algorithm of this invention uses the original, unprocessed input image as a reference image for colorization, the input image and the image to be colored, i.e., the dehazed grayscale image, should be semantically consistent, and the semantic segmentation results should also be consistent. Therefore, this invention performs superpixel segmentation simultaneously on the dehazed grayscale image and the input image. The compensation coefficient, red-green channel adjustment, and yellow-blue channel adjustment are calculated using the following formulas: (33) (34) (35) Where, compensation represents the compensation coefficient. This indicates the adjustment amount for the red and green color channels. This indicates the adjustment amount for the yellow and blue color channels. In this embodiment, the adjustment coefficient is 15, and the preset compensation coefficient is 0.8.

[0059] Let t be the k-th superpixel in the target image, i.e., the dehazed grayscale image. k , t k Compared to the reference image, i.e., the kth superpixel s of the input image k Correspondingly, t k The color channel values ​​in the Lab color space are: (36) (37) Among them, xo k For the k-th superpixel t in the dehazed grayscale image kThe x-axis, yo k For the k-th superpixel t in the dehazed grayscale image k The ordinate. For t k After compensation, the red and green color channels For t k After compensation, the yellow and blue color channels, u a,k Represents the k-th superpixel t k The mean of the red and green color channels, u b,k Represents the k-th superpixel t k The mean of the yellow and blue color channels.

[0060] The compensated red-green and yellow-blue color channels are guided by the L channel of the target image (i.e., the dehazed grayscale image). Finally, the colors in the input image are transferred to the dehazed grayscale image to complete the coloring and obtain the dehazed color image. Figure 4 This is the first input image with haze in this invention. The nighttime image dehazing method provided by this invention is used to... Figure 4 Defogging treatment is performed to obtain Figure 5 . Figure 6 This is the second input image containing haze in this invention. The nighttime image dehazing method provided by this invention is used to... Figure 6 Defogging treatment is performed to obtain Figure 7 .contrast Figure 4 and Figure 5 , Figure 6 and Figure 7 The nighttime image dehazing method provided by this invention can effectively enhance the contrast of nighttime haze images, highlight the outlines of targets and backgrounds in the images, and make the dehazed color images more consistent with the observation characteristics of the human eye, so that operators can more accurately and quickly distinguish targets from dehazed images.

[0061] This embodiment assumes that the reference image and the image to be colored are semantically identical, meaning the superpixel segmentation results should be consistent. Since the reference image in this embodiment is the input image, and the image to be colored is a dehazed grayscale image, the input image is a color image containing the target, background, and haze, while the dehazed grayscale image is a grayscale image containing the target and background. Therefore, the input image and the dehazed grayscale image represent the same scene, and the distribution of the target and background in that scene should be consistent. Superpixel segmentation is performed on the dehazed grayscale image to obtain a first superpixel image, and simultaneously, superpixel segmentation is performed on the input image to obtain a second superpixel image. Compensation coefficients and adjustment amounts for the two color channels are calculated based on the input image containing color information. Then, based on the adjustment amounts for the two color channels, the colors of the input image are transferred to the dehazed grayscale image to colorize it, reflecting the color information of the real scene.

[0062] This embodiment also provides a nighttime image dehazing device based on local patch atmospheric light and color migration, including: The channel decomposition module is used to acquire the input image and decompose the input image into three channels in the RGB color space; Atmospheric light calculation module, used to calculate atmospheric light for each local block in each channel; The atmospheric light optimization module is used to combine brightness and spatial location to optimize the atmospheric light of each local block, resulting in optimized atmospheric light. A depth map group estimation module is used to estimate a depth map group by combining the input image and the optimized atmospheric light. The depth map group fusion module is used to perform weighted fusion of the depth map groups to obtain a fused image; A multi-scale detail enhancement module is used to perform multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image; The color transfer module is used to transfer the colors of the input image to the dehazed grayscale image based on the image features of the input image, so as to obtain a dehazed color image.

[0063] A nighttime image dehazing device based on local block atmospheric light and color migration is used to implement a nighttime image dehazing method based on local block atmospheric light and color migration.

[0064] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0065] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A nighttime image dehazing method based on local patch atmospheric light and color transfer, characterized in that, include: Acquire the input image and decompose it into three channels in the RGB color space; Calculate the atmospheric light for each local block in each channel; By combining brightness and spatial location, the atmospheric light of each local block is optimized to obtain the optimized atmospheric light; By combining the input image and the optimized atmospheric light, a depth map group is estimated; The depth maps are weighted and fused to obtain a fused image; Multi-scale detail enhancement is performed on the fused image to obtain a dehazed grayscale image; Based on the image features of the input image, the colors of the input image are transferred to the dehazed grayscale image to obtain a dehazed color image.

2. The nighttime image dehazing method based on local patch atmospheric light and color migration according to claim 1, characterized in that, Before calculating the atmospheric light for each local block in each channel, the method further includes: An atmospheric scattering model is established, wherein the input of the atmospheric scattering model includes transmittance, and the output of the atmospheric scattering model is the input image; The transmittance is modeled by attenuation to obtain the transmittance expression; Substituting the transmittance expression into the atmospheric scattering model yields the adjusted atmospheric scattering model.

3. The nighttime image dehazing method based on local patch atmospheric light and color migration according to claim 2, characterized in that, The calculation of atmospheric light for each local block in each channel includes: Based on the adjusted atmospheric scattering model, a first inequality relationship between atmospheric light and the input image is established according to the following formula: ； in, This refers to the c-th channel of the input image. is the maximum value of the c-th channel of the haze-free image. Let be the minimum value of the c-th channel of the haze-free image. This represents the c-th channel of atmospheric light; if c=1, it represents the R channel; if c=2, it represents the G channel; if c=3, it represents the B channel. Estimate the maximum and minimum values ​​of the haze-free image in local blocks: ； ； Where avg is the average value operation, x is the pixel index in the c-th channel of the input image, and p i For the i-th local block; Combining the first inequality relationship, the maximum value of the c-th channel of the haze-free image, and the minimum value of the c-th channel of the haze-free image, calculate the atmospheric light in the c-th channel of each local block: ； in, Let be the atmospheric light of the i-th local block on the c-th channel.

4. The nighttime image dehazing method based on local patch atmospheric light and color migration according to claim 1, characterized in that, The process of combining brightness and spatial location to optimize the atmospheric light of each local block, resulting in optimized atmospheric light, includes: The optimized atmospheric light is calculated based on the atmospheric light of each local block using the following formula: ； ； ； ； in, Optimized atmospheric light for the i-th local block of the c-th channel. For the atmospheric light of the i-th local block in the c-th channel, Let be the c-th channel of the input image, f be the atmospheric light optimization function, smooth be the smoothing function, and p be the independent variable of the smoothing function; d total The total distance is used to characterize the brightness and spatial distance of the input image. min For the minimum distance, l max This represents the maximum distance.

5. The nighttime image dehazing method based on local patch atmospheric light and color migration according to claim 4, characterized in that, The step of combining the input image and the optimized atmospheric light to estimate the depth map set includes: Iterate through all smog concentrations; The corresponding depth map is calculated using the following formula based on each smog concentration: ； in, For the i-th depth map of the c-th channel, The concentration of smog; All depth maps are grouped into a depth map group.

6. The nighttime image dehazing method based on local block atmospheric light and color migration according to claim 1, characterized in that, The weighted fusion of the depth map groups to obtain the fused image includes: Select one depth map from the group of depth maps as the brightness reference map; Based on the brightness reference map, the other depth maps in the depth map group are normalized in brightness to obtain multiple normalized depth maps. The corresponding fusion weights are calculated based on the normalized depth map; The fusion weights are used to perform pixel-level weighted summation on all the normalized depth maps to obtain the fused image.

7. The nighttime image dehazing method based on local patch atmospheric light and color migration according to claim 1, characterized in that, The process of performing multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image includes: The fused image is subjected to Gaussian filtering multiple times using different standard deviations to obtain a total number of blurred images at different scales; wherein, total ≥ 2. Subtracting the (k+1)th blurred image from the kth blurred image yields multiple scale details; where 1≤k≤total-1; The local brightness of the fuzzy image is statistically analyzed, and an adaptive gain mapping is calculated for each local brightness. Divide the adaptive gain mapping by the number of channels in the input image to obtain the scale gain; The scale gain is used to perform a weighted summation of all the scale details to obtain the fused details. The fused image is then combined with the fused details to obtain the enhanced fused image; The fused image and the enhanced fused image are weighted and summed to obtain a dehazed grayscale image.

8. The nighttime image dehazing method based on local block atmospheric light and color migration according to claim 1, characterized in that, The step of transferring the colors of the input image to the dehazed grayscale image based on the image features of the input image to obtain a dehazed color image includes: Calculate the mean values ​​of the red-green color channels and the mean values ​​of the yellow-blue color channels of the input image; Calculate the standard deviation of the red-green color channel and the standard deviation of the yellow-blue color channel of the input image; The neutral deviation of the input image is calculated based on the mean values ​​of the red-green color channels and the mean values ​​of the yellow-blue color channels; Divide the neutral deviation by the adjustment coefficient to obtain the adjusted neutral deviation; use the minimum value between the preset compensation coefficient and the adjusted neutral deviation as the compensation coefficient; The negative of the product of the mean value of the red and green color channels of the input image and the compensation coefficient is used as the adjustment amount of the red and green color channels, and the negative of the product of the mean value of the yellow and blue color channels of the input image and the compensation coefficient is used as the adjustment amount of the yellow and blue color channels. Superpixel segmentation is performed on the dehazed grayscale image to obtain the first superpixel image; The red and green color channels are adjusted to compensate each superpixel in the first superpixel image to obtain the compensated red and green color channels. The yellow and blue color channels are adjusted to compensate each superpixel in the first superpixel image to obtain the compensated yellow and blue color channels. Using the L channel in the Lab color space of the dehazed grayscale image, guided filtering is performed on the compensated red-green color channel and the compensated yellow-blue color channel to obtain a dehazed color image.

9. The nighttime image dehazing method based on local block atmospheric light and color migration according to claim 5, characterized in that, The smog concentrations were 0.7, 0.75, 0.8, 0.85, 0.9, 0.95, 0.99, and 1.

10. A nighttime image dehazing device based on local patch atmospheric light and color migration, characterized in that, include: The channel decomposition module is used to acquire the input image and decompose the input image into three channels in the RGB color space; Atmospheric light calculation module, used to calculate atmospheric light for each local block in each channel; The atmospheric light optimization module is used to combine brightness and spatial location to optimize the atmospheric light of each local block, resulting in optimized atmospheric light. A depth map group estimation module is used to estimate a depth map group by combining the input image and the optimized atmospheric light. The depth map group fusion module is used to perform weighted fusion of the depth map groups to obtain a fused image; A multi-scale detail enhancement module is used to perform multi-scale detail enhancement on the fused image to obtain a dehazed grayscale image; The color transfer module is used to transfer the colors of the input image to the dehazed grayscale image based on the image features of the input image, so as to obtain a dehazed color image.

Images