A Single Image Dehazing Method Based on Light and Dark Region Segmentation

Abstract

The present invention discloses a single-image defogging method based on light and dark region segmentation in the field of image processing technology, aiming to solve the problems that the restored image in the prior art is prone to noise and color distortion in bright regions, and the color of the image after defogging by the dark channel prior algorithm is too dark. It includes obtaining a foggy image, performing smoothing processing on the image, preprocessing the image segmentation, calculating the global atmospheric light and transmittance, obtaining the defogged image, and enhancing its brightness; the present invention obtains the segmentation results of bright regions and non-bright regions, independently estimates the transmittance for each region, corrects the brightness of the brightest part of the defogged image, enhances the brightness of the remaining part, further improves the image effect, avoids noise and color distortion in bright regions of the restored image, avoids the drawback of the image being too dark after defogging, and solves the problem that the previous defogging algorithms are not suitable for multiple bright regions.

Description

Technical Field

[0001] The present invention relates to a single - image defogging method based on light and dark region segmentation, belonging to the technical field of image processing. Background Technique

[0002] At present, image defogging algorithms are mainly divided into three categories. The first category is the end - to - end image defogging algorithm, that is, obtaining a fog - free image based on deep learning. Although the end - to - end deep - learning defogging method can be unrestricted by prior knowledge, due to the problem of overfitting in deep learning, that is, it has good effects on the training set but greatly decreases on the test set, so the actual defogging effect may be far lower than the training effect. In addition, during the process of deep learning, the number of parameters to be trained designed by the neural network increases with the increase of the network depth, so it has high requirements for hardware. The second category is the method based on image enhancement. Such algorithms process foggy images based on the image feature information of low brightness and low contrast, without considering the reason of image degradation, directly enhancing the contrast of the image, highlighting the features and valuable information of the objects in the image, but it may cause the loss of some information in the image and make the image distorted. The third category is the method based on image restoration. This method studies the scattering effect of atmospheric suspended particles on light, establishes an atmospheric scattering model, understands the physical mechanism of image degradation, and restores the image before degradation. However, this method requires the support of prior knowledge, and different prior knowledge has different effects on image restoration, and different prior knowledge is only applicable to different scenarios. For example, through the observation of a large number of outdoor fog - free images, the dark - channel prior defogging algorithm was proposed, which achieved good results, but there are still defects.

[0003] When the existing single - image defogging method processes local pixel intensities close to the atmospheric light, such as in the light area, sky area, etc., the restored image is prone to noise and color distortion in the bright area, and the color of the image after defogging by the dark - channel prior algorithm is dark. Summary of the Invention

[0004] The purpose of the present invention is to overcome the deficiencies in the prior art, provide a single - image defogging method based on light and dark region segmentation, and solve the problems that when the single - image defogging method processes local pixel intensities close to the atmospheric light, such as in the light area, sky area, etc., the restored image is prone to noise and color distortion in the bright area, and the color of the image after defogging by the dark - channel prior algorithm is dark.

[0005] To solve the above - mentioned technical problems, the present invention is implemented by the following technical solutions:

[0006] The present invention provides a single - image defogging method based on light and dark region segmentation, including:

[0007] Step A: Obtain a foggy image;

[0008] Step B: Preprocess the acquired image;

[0009] Step C: Use the SLIC superpixel processing method to pre-segment the acquired image to obtain a number of superpixels;

[0010] Step D: According to the pre-segmentation result obtained in Step C, utilize the characteristic that the average brightness of the bright area is relatively large to divide the image into a bright area and a non-bright area, record the boundary information between the bright area and the non-bright area, and use the L component of the LAB color space to represent the brightness of the pixel;

[0011] Step E: Obtain the global atmospheric light estimation value within the bright area segmented in Step D;

[0012] Step F: Obtain the transmittance of the bright area and the non-bright area;

[0013] According to the transmittance of the bright area and the non-bright area, obtain the preliminary transmittance of the foggy image;

[0014] According to the preliminary transmittance of the foggy image, obtain the final transmittance;

[0015] Substitute the global atmospheric light estimation value and the final transmittance into the physical model of atmospheric light scattering to obtain the defogged image;

[0016] Step G: Perform brightness enhancement on the defogged image using the multi-scale Retinex algorithm with adaptive weights;

[0017] Step H: Statistically obtain the pixel area within the preset proportion of the L component for the original foggy image, use the brightness adjustment algorithm based on the HSV space in the corresponding area of the defogged image, and according to the adjustment result of this area, cover the corresponding area of the image adjusted in part of Step G to obtain the final defogged image.

[0018] Furthermore, the preprocessing of the acquired image includes:

[0019] Use a filter in the acquired image to highlight strong edges, and the principle of the filter is as follows:

[0020] ;

[0021] where, represents a pixel, represents a matrix, h is the height of the matrix, and w is the width of the matrix; each pixel is weighted with its adjacent pixels to obtain the filtered value , and the specific filtering formula is as follows:

[0022] ;

[0023] In the formula: N represents the neighborhood of a pixel ; are the coordinates within the N region; represents the weight coefficient during the filtering operation;

[0024] During the filtering process, the color information of the pixel is considered;

[0025] Define the difference between pixel points as:

[0026] ;

[0027] In the formula: A and D represent two different pixel points. Point A corresponds to , and point D corresponds to , represents the position of the pixel, that is, the coordinate position information, represents the information of the pixel value; is the pixel value information of point A, is the pixel value information of point D;

[0028] Construct the contribution value of the pixels within the neighborhood as:

[0029] ;

[0030] In the formula: q represents the pixel within the neighborhood of pixel point A, is the set coefficient, and the filtering formula is as follows:

[0031] ;

[0032] is the neighborhood of pixel A.

[0033] Furthermore, the SLIC superpixel processing method is used to pre-segment the obtained image, and obtaining several superpixels also includes:

[0034] According to the number of superpixels to be generated , at an equal-sized grid space interval for initialization; where represents the grid interval, and the initial size of each grid is , represents the area of the image, represents the number of superpixels after segmentation;

[0035] According to the divided grids, initialize each cluster center at the center of the grid, represented by , and then adjust the initialized cluster centers; where are the three components of the LAB color space, is the position of the pixel in the image.

[0036] Furthermore, according to the divided grid, initializing each cluster center at the center of the grid includes:

[0037] the foggy image is converted from the RGB space to the LAB space; where x and y both represent the pixel space coordinates;

[0038] Among them, converting the pixel from the RGB space to the XYZ space and then from the XYZ space to the LAB space, where the pixel point in the RGB space is represented by is represented by in the XYZ space, and the pixel point in the LAB space is represented by where is the position information of each pixel;

[0039] Adjusting the initialized cluster center includes:

[0040] Calculating the gradient map of the input foggy image and moving the cluster center to the position with the minimum gradient within the neighborhood, and this gradient map is directly calculated in the RGB color space of the foggy image; where n is a positive integer greater than 1;

[0041] Mapping the pixel in the RGB space to , the gradient of is , , obtaining the gradient map grad of the foggy image and moving the initial cluster center to the position with the minimum gradient within the neighborhood to complete the first correction of the cluster center;

[0042] Further adjusting the position of the cluster center, when adjusting, using the mean correction method with standard deviation constraint, the position of the cluster center after correction based on the first correction is , ; where represents the set of all pixels belonging to the i-th cluster center, represents the standard deviation of all pixel points in the i-th cluster center on the L component in the LAB space, is a constant, is the standard deviation constraint condition, is the final cluster center, is the L component of the pixel at position j and position i in the LAB space, is The number of medium pixels.

[0043] Furthermore, the SLIC superpixel processing method is adopted to perform segmentation preprocessing on the acquired image, and several superpixels are obtained, including:

[0044] Obtain a foggy image The clustering centers of, set a distance metric for two different pixels in the figure, and the distance metric will be calculated by weighting the spatial position distance and the LAB space distance;

[0045] The spatial position distance between pixels is ;

[0046] The distance of the pixel in the LAB space is ;

[0047] Foggy image The distance between pixels is ;

[0048] Normalize the distance , where: S is the maximum value of the spatial position distance, , , , , are the coordinates of two pixels respectively, m is the maximum value of the LAB space distance, and it is set as the statistical maximum value, is a constant representing the weight, and the range is ; is the A component of the pixel at position j and i in the LAB space; is the B component of the pixel at position j and i in the LAB space;

[0049] Normalize the distance , where: S is the maximum value of the spatial position distance, , , , , are the coordinates of two pixels respectively, m is the maximum value of the LAB space distance, and it is set as the statistical maximum value, is a constant representing the weight, and the range is ; is the A component of the pixel at position j and i in the LAB space; is the B component of the pixel at position j and i in the LAB space;

[0050] Obtain the distance metric, scan each pixel point i in the input image, and search for its neighborhood for the distance The smallest cluster center is found and the superpixel is assigned to this cluster center, and finally k superpixels are obtained.

[0051] Furthermore, based on the pre-segmentation result obtained in step C, taking advantage of the characteristic that the average brightness of the bright area is relatively large, the image is segmented into a bright area and a non-bright area, and the boundary information of the bright area and the non-bright area is recorded, including:

[0052] Superpixels with a relatively large average L component are classified as the bright area, and the remaining part is classified as the non-bright area. The classification condition is as follows:

[0053] ;

[0054] where represents the average L component of all pixel points in the i-th superpixel, reflecting the average brightness of this superpixel, represents the maximum value of the average brightness among all superpixels, is a dynamic proportionality coefficient used to control the critical value for classifying bright and dark superpixels; represents the bright area, represents the non-bright area;

[0055] The ratio of the gray values on both sides of the rightmost peak of the gray histogram is set as the dynamic proportionality coefficient , according to statistics, the gray values of the bright area are in the interval. The number of pixels on both sides of the rightmost peak in the interval is used as and . When then , otherwise, , where and both represent the number of pixels;

[0056] All superpixels are scanned, the image is segmented into a bright area and a non-bright area, and the boundary information of the bright area and the non-bright area is recorded.

[0057] Furthermore, the transmittance of the bright area and the non-bright area is obtained. Based on the transmittance of the bright area and the non-bright area, the initial transmittance of the foggy image is obtained, including:

[0058] For the non-bright area segmented in step D, the block-level dark channel prior algorithm is used to obtain the transmittance and the defogged image. Based on the defogged image, the halo area in it is obtained and used as the depth-of-field mutation area. The depth-of-field mutation area and the boundary information of the bright area and the non-bright area recorded in step D are set as the transmittance mutation retention area. Combining the transmittance of the bright area and the non-bright area, the initial transmittance of the foggy image is obtained;

[0059] Among them, the block-level dark channel prior algorithm is used to calculate the block-level dark channel map of the input foggy image , where is the color channel of RGB, is the neighborhood centered on the pixel ; is the minimum filtering; is a color channel of any image J; is to take the minimum value for each pixel;

[0060] Perform minimum filtering on the block-level dark channel map to obtain the traditional transmittance , combined with the obtained global atmospheric light, to obtain a preliminary defogged image, and retain the halo area in the defogged image as the depth-of-field mutation area; the transmittance of the bright area is determined to be 1;

[0061] Retain the transmittance of the part in the non-bright area obtained by the above traditional dark channel prior defogging algorithm ;

[0062] Merge the above-mentioned calculated bright area transmittance and non-bright area transmittance to obtain the preliminary transmittance .

[0063] Furthermore, according to the preliminary transmittance of the foggy image, the final transmittance includes:

[0064] Use guided filtering to smooth the transmittance of the part outside the transmittance mutation retention area, and keep the transmittance of the transmittance mutation retention area unchanged to obtain the final transmittance;

[0065] Among them, correct the obtained preliminary transmittance ;

[0066] Use guided filtering to smooth the part outside the transmittance mutation retention area of ;

[0067] Retain the part inside the transmittance mutation retention area;

[0068] Furthermore, substitute the global atmospheric light estimation value and the final transmittance into the physical model of atmospheric light scattering to obtain the defogged image, including:

[0069] The physical model of atmospheric light scattering in a foggy environment is:

[0070] ;

[0071] In the formula: , where y is the spatial coordinate of each pixel, is the dehazed image, is the transparency of the fog, that is, the ratio of the light reflected by the object passing through the fog to the camera, is the atmospheric light intensity, is the direct attenuation term, that is, the part of the light reflected by the object passing through the fog to the camera, is the scattered light caused by the ambient illumination received by the camera; is the foggy day image;

[0072] Substitute the calculated atmospheric light and the corrected transmittance into the physical model of atmospheric light scattering to obtain the equation, so as to calculate the dehazed image .

[0073] Furthermore, the multi-scale Retinex algorithm with adaptive weights includes:

[0074] The mathematical expression of the Retinex principle is: ;

[0075] The basic principle of the Retinex algorithm is as follows:

[0076] ;

[0077] where represents a certain channel in the original image R / G / B, and is the Retinex output of a certain channel in R / G / B, , is the Gaussian surround function, is the convolution operation, , K is the normalization factor to ensure , represents the scale parameter, which controls the details and color information of the enhanced image. The final algorithm result is:

[0078] ;

[0079] where S is a local area of the image, is the standard deviation of the pixel values in this local area, is the number of pixels in the area S, is the average value of the pixels in the area S, and are the control coefficients;

[0080] Calculate the flatness of each superpixel using the definition of the flatness index;

[0081] For flat superpixels, a smaller scale parameter is used, and for non-flat superpixels, a larger scale parameter is used. The scale parameter of the region is defined as follows:

[0082] ;

[0083] ;

[0084] ;

[0085] where is the optimal scale parameter of a certain superpixel, R is the number of superpixels, , , ; i is the region of the superpixel, and the value range is [1, R], represents the flatness index of the i-th region; is the maximum flatness index; is the minimum flatness index;

[0086] According to the optimal scale parameter of each obtained region, calculate the multi-scale weight, and the calculation method is as follows:

[0087] ;

[0088] where is the optimal scale parameter of a certain superpixel, , , ;

[0089] According to the aforementioned compensation factor , the adaptive weight is obtained, and finally the multi-scale Retinex algorithm with the improved adaptive weight is adopted:

[0090] ;

[0091] where represents the enhancement result of different color channels, E represents the total number of image blocks, K represents the number of scale parameters, is the weight corresponding to the scale parameter, n and k are position parameters, the value range of n is [1, E], and the value range of k is [1, K], is a certain channel at n, is the Gaussian surround function at k.

[0092] Compared with the prior art, the beneficial effects achieved by the present invention:

[0093] The single-image defogging method based on light and dark region segmentation smooths the original image, and uses the SLIC segmentation method combining the physical space and the LAB space weighted clustering for the smoothed image to obtain a pre-segmentation result. A dynamic threshold is used to classify the segmented regions to obtain the final segmentation results of the bright region and the non-bright region. On this basis, independent transmittance estimation is performed on each region, a simple proportional coefficient is used to correct the brightness of the brightest part of the defogged image, and the improved multi-scale Retinex algorithm with adaptive weights is used to enhance the brightness of the remaining part, further improving the image effect, avoiding noise and color distortion in the bright region of the restored image, and at the same time avoiding the disadvantage of the image being dark in color after defogging, and solving the problem that the previous defogging algorithms are not suitable for multiple bright regions. Description of the Drawings

[0094] Figure 1 is a flowchart of a single-image defogging method based on light and dark region segmentation according to an embodiment of the present invention;

[0095] Figure 2 is a schematic diagram of light and dark region segmentation according to an embodiment of the present invention;

[0096] Figure 3 is a schematic diagram of transmittance calculation according to an embodiment of the present invention;

[0097] Figure 4 is a schematic diagram of brightness enhancement according to an embodiment of the present invention. Detailed Embodiments

[0098] The technical solution of the present invention will be described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present application and the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0099] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0100] As Figures 1-4 shown, the present invention provides a single-image defogging method based on light and dark region segmentation, including step A: obtaining a foggy image

[0101] Step B: Preprocess the obtained image:

[0102] A filter is used to highlight strong edges in the acquired image. The principle of the filter is as follows:

[0103] ;

[0104] wherein, represents a pixel, represents a matrix, h is the height of the matrix, and w is the width of the matrix; in this process, each pixel is weighted with its adjacent pixels to obtain the filtered value , and the specific filtering formula is as follows:

[0105] In the formula: N represents the neighborhood of the pixel ; is the coordinate within the N region; represents the weight coefficient during the filtering operation;

[0106] In order to prevent partial loss of image boundary information during image smoothing, that is, to avoid blurring the image edge lines; during the filtering process, not only the spatial information of the pixels is considered, but also the color information of the pixels is considered. The farther the distance, the smaller the weight, and the greater the color difference, the smaller the weight. In this way, the image can be well smoothed and the strong edge information can be better protected;

[0107] Define the difference between pixel points as:

[0108] ;

[0109] In the formula: A and D represent two different pixel points. Point A corresponds to , and point D corresponds to , represents the position of the pixel, that is, the coordinate position information, represents the information of the pixel value; is the pixel value information of point A, is the pixel value information of point D;

[0110] When the positions of pixels A and D are more different, the difference is greater. When the pixel value information of pixels A and D is more different, the difference is greater. Therefore, construct the contribution value of the pixels within the neighborhood as:

[0111] ;

[0112] In the formula: q represents the pixel within the neighborhood of pixel point A, is the set coefficient, and the final filtering formula is as follows:

[0113] is the neighborhood of pixel A. When the pixel and when the pixel difference between them is very large, it will become very small, thus achieving the weakening of the influence of the filter value of point pair point. Therefore, during filtering, the phenomenon of blurred boundaries after filtering is greatly reduced, achieving the effect of highlighting strong edges.

[0114] Step C: Use the SLIC superpixel processing method to pre-segment the acquired image to obtain several superpixels;

[0115] Adopt the basic idea of k-means clustering and initialize according to the number of superpixels to be generated , with an equal-sized grid space interval; where, represents the grid interval, and the initial size of each grid is , represents the area of the image, represents the number of superpixels after segmentation;

[0116] According to the divided grid, initialize each cluster center at the center of the grid, represented by , and then adjust the initialized cluster center; where, are the three components of the LAB color space, is the position of the pixel in the image.

[0117] Convert the foggy image from the RGB space to the LAB space; where x and y both represent pixel space coordinates;

[0118] Among them, convert the pixel from the RGB space to the XYZ space, and then convert the pixel from the XYZ space to the LAB space. The pixel point in the RGB space is represented by , the pixel point in the XYZ space is represented by , and the pixel point in the LAB space is represented by , where is the position information of each pixel;

[0119] Adjusting the initialized cluster center includes:

[0120] Calculate the gradient map of the input foggy image , and move the cluster center to the position with the minimum gradient within the neighborhood. This gradient map is directly calculated in the RGB color space of the foggy image; where n is a positive integer greater than 1; optionally, n is 3;

[0121] Map the pixel in the RGB space to , the gradient of , where , , a foggy-day image is obtained The gradient map grad of is obtained, and the initial clustering center is moved to the position with the minimum gradient within the neighborhood, completing the first correction of the clustering center;

[0122] To reduce the situation of misclassification, the standard deviation is introduced to further adjust the position of the clustering center. When adjusting, the mean correction method with standard deviation constraint is adopted. The position of the clustering center after correction on the basis of the first correction is , ; where represents the set of all pixels belonging to the i-th clustering center, represents the standard deviation of all pixel points in the i-th clustering center on the L component in the LAB color space, is a constant, is the standard deviation constraint condition, is the final clustering center, is the L component of the pixel at position j and i in the LAB color space, is the number of pixels in.

[0123] The clustering center of the foggy-day image is obtained. A distance metric is set for two different pixels in the figure. The distance metric will be calculated by weighting the spatial position distance and the LAB color space distance; The spatial position distance between pixels is

[0124] ; ;

[0125] The distance of the pixel in the LAB color space is ;

[0126] The distance between pixels of the foggy-day image is ; ;

[0127] Normalize the distance , where: S is the maximum value of the spatial position distance, , , , , are the coordinates of two pixels respectively, and m is the maximum value of the LAB color space distance, which is set to the statistical maximum value, is a constant representing the weight, with the range ; is the A component of the pixel at position j and i in the LAB color space; is the B component of the pixel at position j and i in the LAB color space;

[0128] Obtain a distance metric, scan each pixel point i in the input image, and search for the clustering center with the smallest distance from it within its neighborhood, and assign it to this clustering center, finally obtaining k superpixels.

[0129] Step D: According to the pre-segmentation result obtained in Step C, utilize the characteristic that the average brightness of the bright region is relatively large, segment the image into a bright region and a non-bright region, and record the boundary information between the bright region and the non-bright region. Using the L component of the LAB color space to reflect the brightness of pixels includes:

[0130] The average value of the L component of the bright region of the haze-free and hazy images in the LAB space is relatively large, the gradient is relatively small, and it contains less information;

[0131] Since the L component of the LAB space exactly reflects the brightness of the image, classify the superpixels with a relatively large average L component as the bright region, and classify the remaining part as the non-bright region. The classification condition is as follows:

[0132] where represents the average L component of all pixel points in the i-th superpixel, reflecting the average brightness of this superpixel, represents the maximum value of the average brightness among all superpixels, is a dynamic proportionality coefficient used to control the critical value of bright and dark superpixel classification; represents the bright region, represents the non-bright region;

[0133] Set the ratio of the gray values on both sides of the rightmost peak of the gray histogram as the dynamic proportionality coefficient , according to statistics, the gray values of the bright region are in the interval. Therefore, the rightmost peak should be in this interval. Otherwise, it can be considered that there is no bright region. Take the pixel numbers on both sides of the rightmost peak in the interval as and , when then , otherwise, , where and both represent the number of pixels;

[0134] Scan all superpixels, set the pixels in the superpixels belonging to the bright region to 255, and set the pixels in the superpixels of the non-bright region to 0, obtaining the final bright and dark segmentation map, segment the image into a bright region and a non-bright region, and record the boundary information between the bright region and the non-bright region.

[0135] Step E: Obtaining the global atmospheric light estimate value within the bright region segmented in Step D includes:

[0136] Within the bright region segmented in Step D, perform global atmospheric light estimation; statistically calculate the average value of the grayscale values of the top 1% of the pixels with the highest brightness in the entire image, and set it as the estimated value of the global atmospheric light:

[0137] The traditional block-level dark channel prior is to extract the top 0.1% of the pixels with the highest brightness values in the block-level dark channel map, and use the maximum value of the corresponding points of these pixels in the original image as the value, and the principle is as follows:

[0138] ;

[0139] ;

[0140] where is the atmospheric attenuation coefficient, which is replaced by a fixed value, represents the scene depth from the scene to the camera at pixel , and the transmittance decreases as the scene depth increases; there are blank areas in the input image that are not specific scenes, such as blank areas like the sky, and the pixels in these areas are set as , that is, infinitely far from the camera. Therefore, whether in a fog-free or foggy scene, the transmittance of these areas approaches 0, and the image pixel value approaches , that is, extract the top 0.1% of the pixels with the highest brightness values in these areas, and use the maximum value of the corresponding points of the extracted pixels in the original image as the value to complete the estimation of the atmospheric light ;

[0141] Since the traditional method of atmospheric light estimation has problems such as overestimation and large estimation errors in bright regions with small depth of field, an improvement is made on the traditional method; statistically calculate the average value of the grayscale values of the top 1% of the pixels with the highest brightness in the entire image, and set it as the estimated value of A.

[0142] Step F: Obtaining the transmittance of the bright region and the non-bright region, and obtaining the preliminary transmittance of the foggy image based on the transmittance of the bright region and the non-bright region includes:

[0143] For the non-bright regions segmented in step D, the block-level dark channel prior algorithm is used to obtain the transmittance and the defogged image. Based on the defogged image, the halo region therein is obtained and used as the depth-of-field mutation region. The depth-of-field mutation region and the boundary information between the bright and non-bright regions recorded in step D are set as the transmittance mutation retention region. By combining the transmittance of the bright and non-bright regions, the preliminary transmittance of the foggy image is obtained; for the bright regions segmented in step D, the transmittance is determined to be 1.

[0144] Use the block-level dark channel prior algorithm to calculate the block-level dark channel map of the input foggy image , where is the color channel of RGB, is the neighborhood centered on the pixel ; is the minimum filtering; is a color channel of an arbitrary image J; is to take the minimum value for each pixel;

[0145] Perform minimum filtering on the block-level dark channel map to obtain the traditional transmittance , combine it with the obtained global atmospheric light to get the preliminary defogged image, and retain the halo region in the defogged image as the depth-of-field mutation region; the transmittance of the bright region is determined to be 1;

[0146] Retain the transmittance obtained by using the traditional dark channel prior defogging algorithm above for the part in the non-bright region ;

[0147] Merge the transmittance of the bright region calculated above and the transmittance of the non-bright region to obtain the preliminary transmittance .

[0148] Set the depth-of-field mutation region and the boundary information between the bright and non-bright regions recorded in step D as the transmittance mutation retention region: The transmittance mutation retention region means that during the subsequent process of processing the transmittance map, the transmittance at the boundary between the bright and dark regions and at the depth-of-field mutation is retained. This behavior is introduced based on the idea that the transmittance at the bright-dark boundary region and the depth-of-field mutation region is inherently discontinuous.

[0149] According to the preliminary transmittance of the foggy image, obtaining the final transmittance includes:

[0150] Use guided filtering to smooth the transmittance of the part outside the transmittance mutation retention region, and keep the transmittance of the transmittance mutation retention region unchanged. Obtaining the final transmittance includes: correcting the obtained preliminary transmittance ; use guided filtering for The part outside the transmittance mutation retention area is smoothed; retain the part within the transmittance mutation retention area; combining the above-mentioned processing, the final transmittance is obtained to prevent the color of the defogged image from being oversaturated and eliminate the white edges at the generation and light-dark boundary of the depth-of-field mutation area.

[0151] Substitute the global atmospheric light estimation value and the final transmittance into the physical model of atmospheric light scattering, and the defogged image obtained includes:

[0152] The physical model of atmospheric light scattering in a foggy environment is:

[0153] ;

[0154] In the formula: , y is the spatial coordinate of each pixel, is the defogged image, is the transparency of the fog, that is, the ratio of the light reflected by the object passing through the fog and reaching the camera, is the atmospheric light intensity, is the direct attenuation term, that is, the part of the light reflected by the object passing through the fog and reaching the camera, is the scattered light caused by the ambient illumination received by the camera; is the foggy day image;

[0155] Substitute the aforementioned calculated atmospheric light and the corrected transmittance into the physical model of atmospheric light scattering to obtain equation, so as to calculate the defogged image .

[0156] Step G: Perform brightness enhancement on the defogged image using the multi-scale Retinex algorithm with adaptive weights: Correct the logarithmic function in the original Retinex algorithm, use the superpixels obtained in step C as the basic unit of brightness enhancement, define the regional flatness index to represent the flatness of each superpixel in the image, calculate the flatness of each superpixel region and the optimal scale parameter, and calculate the multi-scale weights of the multi-scale Retinex according to the optimal scale parameter of each superpixel;

[0157] The multi-scale Retinex algorithm with adaptive weights includes:

[0158] The mathematical expression of the Retinex principle is: ;

[0159] The basic principle of the Retinex algorithm is as follows:

[0160] ;

[0161] where represents one of the R / G / B channels of the original image, is the Retinex output of a certain channel in R / G / B, , is the Gaussian surround function, is the convolution operation, , K is the normalization factor to ensure that , represents the scale parameter, which controls the details and color information of the enhanced image. The final algorithm result is:

[0162] ;

[0163] Since the low-light image contains many pixels with a pixel value of zero, directly using the logarithmic operation of the original Retinex algorithm will cause information loss. The original logarithm is corrected to avoid information loss. The general form of the corrected Retinex algorithm is:

[0164] ;

[0165] where is the compensation factor used to suppress information loss;

[0166] defines the regional flatness index as:

[0167] ;

[0168] ;

[0169] ;

[0170] where S is a local area of the image, is the standard deviation of the pixel values in this local area, is the number of pixels in the area S, is the average value of the pixels in the area S, and are control coefficients. The flatter the area, the larger the corresponding flatness index. Conversely, the flatter the coefficient, the smaller it is;

[0171] Calculate the flatness of each superpixel using the definition of the flatness index;

[0172] For flat superpixels, use a smaller scale parameter, and for non-flat superpixels, use a larger scale parameter. The scale parameter of the area is defined as follows:

[0173] ;

[0174] ;

[0175] where is the optimal scale parameter of a certain superpixel, N is the number of superpixels. Optionally, it is introduced that many researchers obtained through a large number of experiments in actual enhancement practice , , shows better effects in small-scale parameters, shows better effects in large-scale parameters. In addition, there is also ; i is the area of the superpixel, and the value range is [1, R], represents the flatness index of the i-th area; is the maximum flatness index; is the minimum flatness index;

[0176] According to the obtained optimal scale parameter of each area, calculate the multi-scale weight, and the calculation method is as follows:

[0177] ;

[0178] where is the optimal scale parameter of a certain superpixel, , , ;

[0179] According to the aforementioned compensation factor , the adaptive weight is obtained, and the final multi-scale Retinex algorithm using the improved adaptive weight is:

[0180] ;

[0181] where represents the enhancement results of different color channels, E represents the total number of image blocks, K represents the number of scale parameters, is the weight corresponding to the scale parameter, n and k are position parameters, the value range of n is [1, E], and the value range of k is [1, K], is a certain channel at n, is the Gaussian surround function at k.

[0182] Step H: Statistically obtain the pixel area within a preset proportion of the L component from the original foggy image, and use the HSV space brightness adjustment algorithm in the corresponding area of the defogged image. During adjustment, the adjustment is based on the current V value. The larger the V value, the greater the adjustment. According to the adjustment result of this area, cover part of the corresponding area of the image adjusted in Step G to obtain the final defogged image; Optionally, use an adjustment coefficient of 1.1. If the V value of a pixel is 100, it is adjusted to 110. If the V value of a pixel is 200, it is adjusted to 220, and the upper limit value is set to 245; Use the adjustment result of this area to cover part of the corresponding area adjusted by the improved adaptive weight multi-scale Retinex algorithm to obtain the final defogged result image. Optionally, the preset proportion is the top 1%.

[0183] The present invention uses a filter weighted by position and color to smooth the original image, adopts an SLIC segmentation method combining physical space and LAB space weighted clustering for the smoothed image to obtain a pre-segmentation result, classifies the segmented areas using a dynamic threshold to obtain the final segmentation result of bright areas and non-bright areas. On this basis, independent transmittance estimation is performed on bright areas, depth-of-field mutation and light-dark boundary areas, and other areas, uses a simple proportional coefficient to correct the brightness of the brightest part of the defogged image, and uses an improved adaptive weight multi-scale Retinex algorithm to enhance the brightness of the remaining part, further improving the image effect, avoiding noise and color distortion in the bright areas of the restored image, and at the same time avoiding the disadvantage of the image being dark in color after defogging, and solving the problem that the previous defogging algorithms are not suitable for multiple bright areas.

[0184] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0185] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementation in the process Figure 1One or more processes and / or boxes Figure 1 Apparatus for the functions specified in one or more boxes.

[0186] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions in the process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes.

[0187] These computer program instructions may also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions in the process Figure 1 One or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0188] The above is only the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A single-image defogging method based on light and dark region segmentation, characterized in that, it includes: Step A: Obtain a foggy image; Step B: Preprocess the obtained image; Step C: Use the SLIC superpixel processing method to perform pre-segmentation processing on the obtained image to obtain a number of superpixels; Step D: According to the pre-segmentation result obtained in Step C, utilize the characteristic that the average brightness of the bright region is relatively large to divide the image into a bright region and a non-bright region, and record the boundary information between the bright region and the non-bright region. Use the L component of the LAB color space to represent the brightness of the pixel; Step E: In the bright region segmented in Step D, obtain the global atmospheric light estimation value; Step F: Obtain the transmittance of the bright region and the non-bright region; According to the transmittance of the bright region and the non-bright region, obtain the preliminary transmittance of the foggy image; According to the preliminary transmittance of the foggy image, obtain the final transmittance; Substitute the global atmospheric light estimation value and the final transmittance into the physical model of atmospheric light scattering to obtain the defogged image; Step G: Perform brightness enhancement on the defogged image using the multi-scale Retinex algorithm with adaptive weights; Step H: Statistically obtain the pixel region within the preset proportion of the L component for the original foggy image, and use the brightness adjustment algorithm based on the HSV space in the corresponding region of the defogged image. According to the adjustment result of this region, cover the corresponding region of the image adjusted in part of Step G to obtain the final defogged image; Obtain the transmittance of the bright region and the non-bright region. According to the transmittance of the bright region and the non-bright region, obtaining the preliminary transmittance of the foggy image includes: For the non-bright region segmented in Step D, use the block-level dark channel prior algorithm to obtain the transmittance and the defogged image, and according to the defogged image, obtain the halo region therein, and use it as the depth-of-field mutation region. Set the depth-of-field mutation region and the boundary information between the bright region and the non-bright region recorded in Step D as the transmittance mutation retention region, and combine the transmittance of the bright region and the non-bright region to obtain the preliminary transmittance of the foggy image; Among them, the block-level dark channel prior algorithm is used to calculate the block-level dark channel map of the input foggy image , where is the color channel of RGB, is the neighborhood centered on the pixel ; is the minimum value filtering; is a color channel of any image J; is to take the minimum value for each pixel; Perform minimum filtering on the block-level dark channel map to obtain the traditional transmittance , combine with the obtained global atmospheric light to obtain a preliminary dehazed image, retain the halo area in the dehazed image as the depth-of-field mutation area; the transmittance of the bright area is determined to be 1; Keep the transmittance obtained by using the traditional dark channel prior defogging algorithm as above The part in the non-bright area ; Merge the transmittance of the bright area calculated above and the transmittance of the non-bright area to obtain the preliminary transmittance .

2. The single-image defogging method based on light and dark region segmentation according to claim 1, characterized in that, preprocessing the obtained image includes: Use a filter in the obtained image to highlight strong edges. The principle of the filter is as follows: ； Among them, represents a pixel, represents a matrix, where h is the height of the matrix and w is the width of the matrix; each pixel is weighted with its adjacent pixels to obtain a filtered value , and the specific filtering formula is as follows: ； Where: N represents a pixel neighborhood; is the coordinate within the N region; represents the weight coefficient during the filtering operation; During the filtering process, consider the color information of the pixels; Define the difference between pixel points as: ； Where: A and D represent two different pixel points. Point A corresponds to and point D corresponds to . represents the position of the pixel, i.e., the coordinate position information, represents the information of the pixel value; is the pixel value information of point A, is the pixel value information of point D; Construct the contribution value of the pixels within the neighborhood as: ； In the formula: q represents the pixels within the neighborhood of pixel point A, is the set coefficient, and the filtering formula is as follows: ； is the neighborhood of pixel A.

3. The single-image defogging method based on light and dark region segmentation according to claim 1, characterized in that, using the SLIC superpixel processing method to perform pre-segmentation processing on the obtained image to obtain a number of superpixels further includes: According to the number of superpixels to be generated , initialize according to the equal-size grid space interval ; among them, represents the interval of the grid, and the initial size of each grid is , represents the area of the image, represents the number of superpixels after segmentation; According to the divided grid, each cluster center is initialized at the center of the grid, denoted by , and then the initialized cluster centers are adjusted; among them, are the three components of the LAB color space, and is the position of the pixel in the image.

4. The single-image defogging method based on light and dark region segmentation according to claim 3, characterized in that, According to the divided grid, initialize each cluster center at the center of the grid includes: Convert the foggy image from the RGB color space to the LAB color space; where x and y both represent pixel spatial coordinates; Among them, the pixel is converted from the RGB space to the XYZ space, and then from the XYZ space to the LAB space. The pixel points in the RGB space are represented by , the pixel points in the XYZ space are represented by , and the pixel points in the LAB space are represented by . Among them, is the position information of each pixel; Adjusting the initialized cluster center includes: Calculate the input foggy image 's gradient map, and move the clustering center to the position with the minimum gradient within the neighborhood. This gradient map is directly calculated in the RGB color space of the foggy image; where n is a positive integer greater than 1; Map the RGB space pixels to , The gradient of is , where , Get the foggy image The gradient map grad of, move the initial clustering center to The position with the minimum gradient within the neighborhood to complete the first correction of the clustering center; Further adjust the position of the clustering center. When adjusting, the mean correction method with standard deviation constraint is adopted. The position of the clustering center after correction on the basis of the first correction is , ; where represents the set of all pixels belonging to the i-th clustering center, represents the standard deviation of all pixel points in the i-th clustering center on the L component in the LAB space, is a constant, is the standard deviation constraint condition, is the final clustering center, is the L component of the pixel at position j and position i in the LAB space, is the number of pixels in 5. The single-image defogging method based on light and dark region segmentation according to claim 4, characterized in that, The obtained image is segmented and preprocessed by using the SLIC superpixel processing method to obtain a number of superpixels, including: Obtain a foggy image The clustering centers are obtained, and a distance metric is set for two different pixels in the image. The distance metric will be calculated by weighting the spatial position distance and the LAB space distance; The spatial position distance between pixels is ; The distance of the pixel in the LAB space is ; Foggy day image The inter-pixel distance of ; Normalize the distance wherein: S is the maximum value of the spatial position distance, and and and are the coordinates of two pixels respectively, m is the maximum value of the LAB space distance, which is set to the statistical maximum value, is a constant representing the weight, with the range ; is the A component of the pixel in the LAB space at positions j and i; is the B component of the pixel in the LAB space at positions j and i;​ Obtain a distance metric, scan each pixel point i in the input image, and search within its neighborhood for the cluster center with the smallest distance from it and assign it to that cluster center, finally obtaining k superpixels.

6. A single-image defogging method based on light and dark region segmentation according to claim 1, characterized in that, According to the pre-segmentation result obtained in step C, taking advantage of the characteristic that the average brightness of the bright region is relatively large, the image is segmented into a bright region and a non-bright region, and the boundary information between the bright region and the non-bright region is recorded, including: Superpixels with a relatively large average L component are classified as the bright region, and the remaining part is classified as the non-bright region. The classification condition is as follows: ； where represents the average L component of all pixel points in the i-th superpixel, reflecting the average brightness of the superpixel, represents the maximum value of the average brightness among all superpixels, is a dynamic proportionality coefficient used to control the critical value for classifying bright and dark superpixels; represents the bright region, represents the non-bright region; Set the ratio of the grayscale values on both sides of the rightmost peak of the grayscale histogram as the dynamic proportionality coefficient , according to statistics, the grayscale values of the bright area are in interval, and the number of pixels on both sides of the rightmost peak in the interval is used as and , when time , otherwise, , where and both represent the number of pixels; Scan all superpixels, segment the image into a bright region and a non-bright region, and record the boundary information between the bright region and the non-bright region.

7. A single-image defogging method based on light and dark region segmentation according to claim 1, characterized in that, According to the initial transmittance of the foggy image, the final transmittance is obtained, including: Guided filtering is used to smooth the transmittance of the part outside the transmittance mutation retention area, and the transmittance of the transmittance mutation retention area remains unchanged to obtain the final transmittance; Among them, the obtained preliminary transmittance is corrected; Using guided filtering, perform smoothing on the part outside the transmittance mutation retention area; ​ Reserved Part within the transmittance mutation reservation area; Combined with the above processing of , the final transmittance is obtained.

8. A single-image defogging method based on light and dark region segmentation according to claim 7, characterized in that, Substitute the global atmospheric light estimation value and the final transmittance into the physical model of atmospheric light scattering to obtain the defogged image, including: The physical model of atmospheric light scattering in a foggy environment is: ； Wherein: , y is the spatial coordinate of each pixel, is the image after defogging, is the transparency of the fog, that is, the ratio of the light reflected by the object passing through the fog to the camera, is the atmospheric light intensity, is the direct attenuation term, that is, the part of the light reflected by the object passing through the fog to the camera, is the scattered light caused by the ambient illumination received by the camera; is the foggy day image; Substitute the previously calculated atmospheric light and the corrected transmittance into the physical model of atmospheric light scattering to obtain an equation, and thus calculate the defogged image .

9. A single-image defogging method based on light and dark region segmentation according to claim 1, characterized in that, The multi-scale Retinex algorithm with adaptive weights includes: The mathematical expression of the Retinex principle is: ; The basic principle of the Retinex algorithm is as follows: ； wherein represents a certain channel in the original image R / G / B, is the Retinex output of a certain channel in R / G / B, , is the Gaussian surround function, is the convolution operation, , K is the normalization factor to ensure that , represents the scale parameter, controlling the details and color information of the enhanced image. The final algorithm result is: ； The general form of the modified Retinex algorithm is: ； wherein is a compensation factor for suppressing information loss; Define the flatness index of the region as follows: ； ； ； Where S is a local area of the image, is the standard deviation of the pixel values of the local area, is the number of pixels in region S, is the average value of the pixels in region S, and are control coefficients; Calculate the flatness of each superpixel using the definition of the flatness index; For flat superpixels, use a smaller scale parameter, and for non-flat superpixels, use a larger scale parameter. The scale parameter of the region is defined as follows: ； ； ； Among them is the optimal scale parameter of a certain superpixel, R is the number of superpixels, , , ; i is the area of the superpixel, and the value range is [1, R], represents the flatness index of the i-th area; is the maximum flatness index; is the minimum flatness index; According to the obtained optimal scale parameter of each region, calculate the multi-scale weight, and the calculation method is as follows: ； ； wherein is the optimal scale parameter of a certain superpixel, , , ; According to the aforementioned compensation factor , an adaptive weight is obtained, and finally, an improved multi-scale Retinex algorithm with an adaptive weight is adopted: ； Among them represents the enhancement results of different color channels, E represents the total number of image blocks, K represents the number of scale parameters, is the weight value corresponding to the scale parameter, n and k are position parameters, the value range of n is [1, E], and the value range of k is [1, K], is a certain channel at n, is the Gaussian surround function at k.

Images