Image enhancement method, device, equipment and storage medium in severe weather

Abstract

The present application discloses an image enhancement method, device, equipment and storage medium under severe weather conditions, and relates to the field of image enhancement. The method reads an image frame and performs LAB color space conversion to determine the degraded image frame and the corresponding image type therein; if it is a dust image, it is converted into a grayscale image and a power operation is performed on the grayscale value to obtain a dust-free image; if it is a haze image, image edge filtering is performed and a transmittance image is extracted; when the filtered image contains a sky area, the transmittance value of the area is extracted and deviation correction is performed to obtain a corrected transmittance image; based on the corrected transmittance image, image defogging is performed to obtain a haze image. The scheme distinguishes the degraded image frames and the corresponding image types caused by severe weather according to the color cast factor and chromaticity, and adopts different image enhancement schemes for different image types, especially for haze images, gradient domain guided filtering and transmittance optimization of the sky area are performed to obtain a clearer defogging image.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image enhancement, and in particular to a method, device, equipment and storage medium for image enhancement in severe weather. Background Art

[0002] Bad weather such as haze and sandstorms can cause the light on the surface of objects to be absorbed and reflected by particles in the atmosphere, resulting in poor image quality, blurred details, and dim colors. In response to this problem of image quality degradation caused by natural phenomena, it is of universal practical significance to explore image processing.

[0003] Considering the low brightness and low contrast characteristics of the image, the image quality can be improved to a certain extent by using image enhancement methods. Typical examples include homomorphic filtering and Retinex algorithms. In recent years, defogging algorithms based on this principle have made great progress, such as improved algorithms based on bilateral filtering and guided filtering. Although these algorithms improve the image processing effect, they have certain shortcomings in terms of universality. Based on the classic dark channel prior method, people have developed a variety of complex defogging algorithms, but these algorithms cannot match a variety of severe weather conditions. In addition, when improving the defogging of a single image, there are phenomena such as blurred edges and halos, or when there are large areas of haze such as the sky, color distortion and other problems are prone to occur. Summary of the invention

[0004] The embodiments of the present application provide a method, device, equipment and storage medium for image enhancement in severe weather conditions, which solve the problem of reduced image quality and limited application scenarios in severe weather conditions.

[0005] In one aspect, the present application provides a method for image enhancement in severe weather, the method comprising:

[0006] Read the image frames of the video to be processed and perform LAB color space conversion to determine the degraded image frames and corresponding image types; the image types include dust images and haze images;

[0007] When the degraded image frame is a dust image, the dust image is converted into a grayscale image, a power operation is performed on the grayscale pixels on the grayscale image, and the calculated adjusted grayscale values ​​are mapped to form a dust-free image output;

[0008] When the degraded image frame is a haze image, performing image edge filtering on the haze image, and extracting a transmittance image of the filtered image;

[0009] In response to detecting that the filtered image includes a sky area, extracting a transmittance value of the sky area mapped in the transmittance image, performing deviation correction on the transmittance value of the sky area to obtain a corrected transmittance image;

[0010] The haze image is dehazed based on the corrected transmittance image to obtain a dehazed image.

[0011] Specifically, the reading of the image frames of the video to be processed and performing LAB color space conversion to determine the degraded image frames and the corresponding image types therein includes:

[0012] The read image frame is converted into LAB color space, and the chromaticity component, average chromaticity component and chromaticity center distance representing yellow on the LAB chromaticity plane are calculated according to the resolution size of the image frame;

[0013] The ratio of the average chromaticity to the chromaticity center distance is determined as a color cast factor, and when the color cast factor does not exceed a set threshold, it is determined as a normal image frame; when the color cast factor exceeds the set threshold, it is determined as a degraded image frame;

[0014] When the chrominance component representing yellow in the LAB chrominance plane of the degraded image frame is a positive number, the degraded image frame is determined to be a dust image.

[0015] Specifically, the method performs power operation on the grayscale pixels on the grayscale image and maps the calculated adjusted grayscale values ​​to form a dust-removed image output; including:

[0016] Extract the grayscale value of the grayscale image at the target pixel position, and use the grayscale value as the base of the power calculation to adjust the coefficient Perform power operation as an exponent, and use the calculated result as the adjusted grayscale value of the target position;

[0017] Remapping the adjusted grayscale value into the grayscale image to form an adjusted grayscale image;

[0018] The adjusted grayscale image is restored to obtain a dust-removed image.

[0019] Specifically, performing image edge filtering according to the haze image includes:

[0020] The haze image is filtered using a gradient domain guided filter, as shown below:

[0021]

[0022] Among them and They represent the coefficients of the filter obtained by linear regression with pixel j as the center, represents the pixel value of the output filtered image at position i, is the pixel value of the input haze image at position i, represents the window centered at pixel i;

[0023] Constructing the filter loss function , filter output with the goal of minimizing loss output, filter loss function It is expressed as follows:

[0024]

[0025]

[0026] Among them represents the edge-aware weighting function, represents the edge-aware weighting coefficient, Represents the relevant parameters of the e function, Represents the grayscale value of pixels within the analysis window with local differences, Represents the pixel value of the dark channel image of the haze image at position i.

[0027] Specifically, extracting the transmittance image of the filtered image includes:

[0028] Obtain a dark channel image of the haze image, and arrange the pixels in the dark channel image according to the pixel value, and set the pixel value ranked in the top 0.1% as the pixel threshold;

[0029] All pixel values ​​of the dark channel image are compared based on the pixel threshold to generate a binary mask image; points in the dark channel image that are less than the pixel threshold are set to 0, and points that are not less than the pixel threshold are set to 1;

[0030] Multiply the generated mask image bitwise with the haze image converted to HSV format, and output the brightness channel value;

[0031] All brightness channel values ​​are weighted averaged to obtain the atmospheric light component A;

[0032] Substitute the pixels of each point in the filtered image into the transmittance formula to calculate the transmittance of each point ;

[0033]

[0034] Among them represents the deblurring coefficient, c is a color channel of the image, is the dark channel image calculated based on the filtered image;

[0035] The transmittance image is formed by mapping the transmittance of each point of the filter image according to the pixel position.

[0036] Specifically, after performing image edge filtering on the haze image, the method further includes:

[0037] Extracting a binary image and a gradient image based on the filtered image, performing an AND operation on the binary image and the gradient image to obtain an edge image;

[0038] Edge tracking is performed on the edge image, and the area with the largest area is marked as the sky area.

[0039] Specifically, the deviation correction of the transmittance value of the sky area to obtain a corrected transmittance image includes:

[0040] The transmittance value of the sky area is corrected according to the following value:

[0041]

[0042] Where L1 and L2 are the transmittance thresholds for suppressing color distortion. is the corrected transmittance value after color correction of the sky area;

[0043] Remapping based on the corrected transmittance of the sky area and the transmittance of the non-sky area to form a corrected transmittance image;

[0044] The step of performing image defogging on the haze image based on the corrected transmittance image to obtain a defogged image includes:

[0045] Substitute the corrected transmittance image and haze image into the following formula:

[0046]

[0047] Among them It is a haze image. is the transmittance threshold, It is a dehazed image.

[0048] On the other hand, the present application provides an image enhancement device in severe weather, the device comprising:

[0049] An image determination module is used to read the image frames of the video to be processed and perform LAB color space conversion to determine the degraded image frames and corresponding image types; the image types include dust images and haze images;

[0050] An image dust removal module, for converting the dust image into a grayscale image when the degraded image frame is a dust image, performing power operations on the grayscale pixels on the grayscale image, and mapping the calculated adjusted grayscale values ​​to form a dust-removed image output;

[0051] An image extraction module, used for performing image edge filtering on the haze image when the degraded image frame is a haze image, and extracting a transmittance image of the filtered image;

[0052] An image correction module, configured to extract, in response to detecting that the filtered image includes a sky area, a transmittance value of the sky area mapped in the transmittance image, and perform deviation correction on the transmittance value of the sky area to obtain a corrected transmittance image;

[0053] The image defogging module is used to perform image defogging on the haze image based on the corrected transmittance image to obtain a defogging image.

[0054] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the image enhancement method in severe weather as described in the above aspect.

[0055] On the other hand, the present application provides a computer-readable storage medium, wherein the readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image enhancement method in severe weather as described in the above aspect.

[0056] The beneficial effects of the technical solution provided by the embodiment of the present application include at least: the present application extracts and analyzes the collected video to be processed frame by frame, uses the LAB color space conversion method, distinguishes the degraded image frames and corresponding image types caused by bad weather according to the color cast factor and chromaticity, and adopts different image enhancement schemes for different image types. For the dust image, the grayscale image is adjusted by power operation and remapping, and the dust-free image is restored, so that the contrast of some brighter images becomes appropriate, and the picture quality is reduced due to the yellowing of the sand and dust. For the haze image, the original defogging model is improved, and the image is preferentially replaced by the gradient domain guided filter instead of the minimum filter or the guided filter, which can significantly improve the image defogging effect, especially in the image edge retention and detail enhancement. Next, for the situation where the sky area contains a lot of haze, in order to improve the problem of image color imbalance, the sky area is marked, and the transmittance value of the area is optimized and adjusted, and finally the defogging operation is performed, so that a more realistic defogging image can be restored. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of an image enhancement method under severe weather conditions provided by an embodiment of the present application;

[0058] Figure 2 It is a schematic diagram of the algorithm for distinguishing frame quality, determining image type and performing image enhancement;

[0059] Figure 3 This is a flowchart of performing image edge filtering and transmittance optimization dehazing on a haze image;

[0060] Figure 4 It is the principle diagram for obtaining the dark channel image;

[0061] Figure 5 This is a structural block diagram of an image enhancement device for severe weather conditions provided by the present application. DETAILED DESCRIPTION

[0062] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0063] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0064] Figure 1 : is a flow chart of the image enhancement method in bad weather provided by an embodiment of the present application, which specifically includes the following steps:

[0065] S1, reading the image frames of the video to be processed and performing LAB color space conversion to determine the degraded image frames and corresponding image types; the image types include dust images and haze images;

[0066] The degraded image frames are image frames that are captured from the video to be processed and determined to have degraded image quality. The degraded image quality referred to in this application is mainly caused by bad weather, specifically dust images and haze images taken in dusty weather and haze weather.

[0067] First, the image quality needs to be analyzed using the color cast factor and chromaticity of the LAB color space as the criterion to determine the degraded image frame; at the same time, for the degraded image frame, it is also necessary to further determine which specific type of image it is. Because the dust image and the haze image in this application have their own unique characteristics on the screen, different processing methods are required for image enhancement and restoration.

[0068] S2, when the degraded image frame is a dust image, convert the dust image into a grayscale image, perform power operation on the grayscale pixels on the grayscale image, and map the calculated adjusted grayscale values ​​to form a dust-free image output;

[0069] When the degraded image frame is a sand and dust image, the image obtained under sandstorm weather conditions is usually yellowish brown. This is because dust and other particles have a biased reflection of light, which makes the effect of atmospheric light on the RGB three channels unevenly distributed. Therefore, the method of removing haze is not suitable for removing sand and dust from images. The method adopted in this application is to convert the original sand and dust image into a grayscale image. For the grayscale pixel at each position of the grayscale image, an exponent is set to perform a power operation based on this grayscale value, and the calculation result is used as the grayscale value to adjust the image. The specific expression is as follows:

[0070] A, extract the grayscale value of the grayscale image at the target pixel position, and use the grayscale value as the base to calculate the power to adjust the coefficient Perform power operation as an exponent, and use the calculated result as the adjusted grayscale value of the target position;

[0071] B, remapping the adjusted grayscale value into the grayscale image to form an adjusted grayscale image;

[0072] C, restore the adjusted grayscale image to obtain the dust-free image.

[0073] Through the grayscale mapping described above, the contrast of some brighter images can be made appropriate, reducing the situation where the image quality is affected by yellowing caused by sand and dust.

[0074] S3, when the degraded image frame is a haze image, performing image edge filtering on the haze image, and extracting a transmittance image of the filtered image;

[0075] The image quality of haze images is very different from that of dust images in terms of color. In the traditional defogging algorithm principle, the mathematical model of foggy and hazy images is directly used to remove them. In actual use, after calculating the transmittance map in the traditional solution, it is directly substituted into the calculation to solve the defogging image. However, in fact, when the transmittance value in the transmittance map is very small, it will cause the value of the defogging image to be too large, making the image whiter as a whole. When the transmittance map is too rough, the edges obtained by defogging are obviously inconsistent. In order to obtain a more refined transmittance map, minimum filtering or guided filtering is often used to obtain a better transmittance map. However, the defogging image obtained by this method is darker and the edges are blurred. When the framing object is close to the sky in a large range and is not covered by shadows, the effect is not good.

[0076] For this reason, the focus of this application in terms of defogging is to improve the filtering of the original haze image and optimize the transmittance. Gradient domain guided filtering is used for image edge filtering, because this method introduces the gradient constraint of the first-order edge. The edge-perceptible design constraint can better retain the edge. Using this algorithm to replace the minimum filter or guided filter can significantly improve the image defogging effect, especially in terms of image edge preservation and detail enhancement. The filtering method is used to obtain the filtered image, and then the transmittance image is extracted based on the filtered image, that is, a numerical array diagram composed of the transmittance values ​​of each point, which is used for subsequent defogging calculations.

[0077] S4, in response to detecting that the filtered image includes a sky area, extracting the transmittance value of the sky area mapped in the transmittance image, performing deviation correction on the transmittance value of the sky area, and obtaining a corrected transmittance image;

[0078] Because the defogging principle requires the use of transmittance / image for defogging, and the transmittance estimation is prone to inaccurate estimation when there are many haze areas such as the sky or when there is a large area of ​​haze in the image, resulting in color deviation or color distortion. For this reason, when improving this application, the sky area with higher brightness values ​​and smooth pixel features is first identified, and then the transmittance value of the area is appropriately adjusted and optimized to suppress color deviation and color distortion. After the transmittance value of the sky area is corrected for deviation, a corrected transmittance image is obtained. The picture after defogging using the corrected transmittance image is more realistic in color.

[0079] S5, performing image defogging on the haze image based on the corrected transmittance image to obtain a defogging image.

[0080] In the field of machine vision and computer image processing, the mathematical model describing foggy and hazy images can be expressed as follows:

[0081]

[0082] in is the image to be dehazed, is the restored dehazed image, parameters is the global atmospheric light component, Here, it can specifically refer to the transmittance value after adjustment and optimization.

[0083] In summary, the present application extracts and analyzes the collected video to be processed frame by frame, uses the LAB color space conversion method, distinguishes the degraded image frames and corresponding image types caused by bad weather according to the color cast factor and chromaticity, and adopts different image enhancement schemes for different image types. For dust images, grayscale adjustment is performed by using a grayscale image for power operation and remapping, and the dust-removed image is restored to make the contrast of some brighter images appropriate, reducing the situation where the picture quality is affected by the yellowing of the sand and dust. For haze images, improvements are made on the original defogging model, and the image is preferentially replaced by the gradient domain guided filter instead of the minimum filter or the guided filter, which can significantly improve the image defogging effect, especially in terms of image edge retention and detail enhancement; then, for the situation where the sky area contains a lot of haze, in order to improve the problem of image color imbalance, the sky area is marked, and the transmittance value of the area is optimized and adjusted, and finally the defogging operation is performed, so that a more realistic defogging image can be restored.

[0084] Figure 2 This is a schematic diagram of the algorithm for distinguishing frame quality, determining image type, and performing image enhancement. The specific judgment process can be summarized as follows:

[0085] a. Perform LAB color space conversion on the read image frame, and calculate the chromaticity component, average chromaticity component and chromaticity center distance representing yellow on the LAB chromaticity plane according to the resolution size of the image frame;

[0086] b. Determine the color cast factor according to the ratio of the average chromaticity to the chromaticity center distance. When the color cast factor does not exceed the set threshold, determine it as a normal image frame; when the color cast factor exceeds the set threshold, determine it as a degraded image frame;

[0087] c. When the chrominance component representing yellow in the LAB chrominance plane of the degraded image frame is a positive number, the degraded image frame is determined to be a dust image.

[0088] The LAB color space is a color model determined by the CIE organization that theoretically includes all colors visible to the human eye. Image color cast detection is to measure the degree of deviation between two colors. The color cast of an image is not only related to the image's chromaticity average value, but also to the image's chromaticity distribution characteristics. In the LAB color space, the concept of an equivalent circle is used, and the ratio of the image's average chromaticity D and the chromaticity center distance M, that is, the color cast factor K, is used to measure the degree of color cast of an image. The calculation formula is as follows:

[0089]

[0090]

[0091]

[0092]

[0093] Where a and b represent the A and B components of the LAB color space, and M and N represent the length and width of the image; and Respectively represent the red and yellow components on the LAB chromaticity plane; if If the color cast factor K is greater than 0, the image is judged to be yellowish, and it is determined to be a sandstorm weather, otherwise it is a haze image. The color cast factor threshold set in the embodiment of the present application is 1.35. If the color cast factor K is greater than 1.35, it is judged to be a degraded and blurred image, otherwise it is a clear image.

[0094] It is particularly noted that in this solution, the processing of dust images and haze images does not strictly distinguish the order, and the corresponding image enhancement operations can also be performed simultaneously.

[0095] Figure 3 This is a flowchart of performing image edge filtering and transmittance optimization defogging on a haze image, which may include the following steps:

[0096] 1. Edge filtering

[0097] The filtering uses a gradient domain guided filter to filter the haze image, which is expressed as follows:

[0098]

[0099] Among them and They represent the coefficients of the filter obtained by linear regression with pixel j as the center, represents the pixel value of the output filtered image at position i, is the pixel value of the input haze image at position i, represents the window centered at pixel i;

[0100] The loss function E defined by traditional filtering is expressed as follows:

[0101]

[0102] The weighting coefficient Usually a regular term to prevent Too large a setting, Represents the guide image of the filter. This filtering scheme is better than bilateral filtering in terms of the ability to avoid gradient reversal, but since the regularization coefficient is a fixed value, it is inevitable to lose edge pixels. In some embodiments, the improvement is also expressed as:

[0103]

[0104] Among them is the edge-aware weighting coefficient, which calculates the local changes of pixels in a 3×3 window and performs a specific analysis in combination with pixel variance. This weighting coefficient is used to reflect the importance of the current pixel to the entire filtered image. This method can effectively reduce boundary aperture blur, but the zero-order intensity constraint set above obtains the desired pixel value and gradient domain constraint, thereby performing pixel smoothing in the spatial domain. These edge preservation methods cannot obtain good results in some cases. To overcome these problems, the present application additionally defines an analysis window with local differences and sets a large-capacity filter window, which is usually set to a multiple of 8. Such a weight function can determine the importance of each pixel to the entire image. In order to better retain edge pixels, the present application constructs a filtering loss function , filter output with the goal of minimizing loss output, filter loss function It is expressed as follows:

[0105]

[0106]

[0107] Among them represents the edge-aware weighting function, represents the edge-aware weighting coefficient, Represents the relevant parameters of the e function, Represents the grayscale value of pixels within the analysis window with local differences, Represents the pixel value of the dark channel image of the haze image at position i.

[0108] From the above formula, we can find that if the pixel is on the edge of the image, then The value of is close to 1, and if the pixel is in a flat area, the value is close to 0. This is the core concept of the gradient domain guided filter.

[0109] 2. Extracting Transmittance Image

[0110] 1) Obtain the dark channel image of the haze image, and arrange the pixels in the dark channel image according to the pixel value, and set the pixel value ranked in the top 0.1% as the pixel threshold;

[0111] Figure 4 This is the principle diagram for obtaining the dark channel image. In this step, the original haze image needs to be converted into RGB format first, and then the point with the lowest pixel value in the three channels is extracted according to the pixel position, and it is used as the pixel point of the dark channel image at that position. The dark channel image can be obtained by traversing all pixel positions. Then, according to the image size, it is flattened into a one-dimensional vector, and the first n (one thousandth of the total pixels) largest elements are extracted and sorted to find the nth largest value in this group of elements, which is the pixel threshold.

[0112] 2) Compare all pixel values ​​of the dark channel image based on the pixel threshold to generate a binary mask image; points in the dark channel image that are less than the pixel threshold are set to 0, and points that are not less than the pixel threshold are set to 1;

[0113] 3) Multiply the generated mask image bit by bit with the haze image converted to HSV format, and output the brightness channel value;

[0114] The above steps require initializing an all-zero matrix matrix with the same size as the dark channel image, which is used to create a binary mask and traverse each pixel of the dark channel image. If the pixel value is greater than or equal to the pixel threshold, the corresponding position of the matrix is ​​set to 1, otherwise it is set to 0.

[0115] 4) Perform weighted average of all brightness channel values ​​to obtain the atmospheric light component A;

[0116] Apply the mask matrix to the HSV version of the haze image, retaining only the third channel (brightness channel, i.e., V in Hue-Saturation-Value), and convert the result into an unsigned 8-bit integer image C, then calculate the average value of C to obtain the atmospheric light component A.

[0117] 5) Substitute the pixels of each point of the filtered image into the transmittance formula to calculate the transmittance of each point ;

[0118]

[0119] Among them represents the deblurring coefficient, c is a color channel of the image, is the dark channel image calculated based on the filtered image;

[0120] 6) The transmittance of each point in the filtered image is mapped according to the pixel position to form a transmittance image.

[0121] 3. Transmittance Optimization

[0122] 1) After obtaining the filtered image Afterwards, a binary image and a gradient image are extracted based on the filtered image, and the binary image and the gradient image are ANDed to obtain an edge image;

[0123] 2) Perform edge tracking on the edge image and mark the area with the largest area as the sky area.

[0124] The core of this step is to identify the sky area with higher brightness value and smooth pixel features, and then perform sky area correction.

[0125] 3) Correct the transmittance value of the sky area according to the following values:

[0126]

[0127] L1 and L2 are the transmittance thresholds for suppressing color distortion. L1 is usually set to 220, and L2 is usually set to 180. It is the corrected transmittance value after color correction of the sky area. The transmittance image is corrected using this segmented mapping function to suppress the color deviation problem of the sky or large-area haze image.

[0128] The corrected transmittance image is formed by remapping based on the corrected transmittance of the sky area and the transmittance of the non-sky area; the corrected transmittance image and the haze image are substituted into the following formula:

[0129]

[0130] Among them It is a haze image. is the transmittance threshold, It is a dehazed image.

[0131] In summary, the present application utilizes a gradient domain guided filtering algorithm to improve edge preservation characteristics, performs transmittance compensation for the sky or large areas of haze, and eliminates the color distortion problem caused by defogging of large areas such as the sky.

[0132] Figure 5 This is a structural block diagram of an image enhancement device for severe weather provided by the present application, the device comprising:

[0133] The image determination module 510 is used to read the image frames of the video to be processed and perform LAB color space conversion to determine the degraded image frames and corresponding image types; the image types include dust images and haze images;

[0134] The image dust removal module 520 is used for converting the dust image into a grayscale image when the degraded image frame is a dust image, performing power operation on the grayscale pixels on the grayscale image, and mapping the calculated adjusted grayscale values ​​to form a dust-removed image output;

[0135] An image extraction module 530, configured to perform image edge filtering on the haze image and extract a transmittance image of the filtered image when the degraded image frame is a haze image;

[0136] An image correction module 540 is configured to, in response to detecting that the filtered image includes a sky area, extract a transmittance value of the sky area mapped in the transmittance image, perform deviation correction on the transmittance value of the sky area, and obtain a corrected transmittance image;

[0137] The image defogging module 450 is used to perform image defogging on the haze image based on the corrected transmittance image to obtain a defogging image.

[0138] Furthermore, the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the image enhancement method in severe weather described in any of the above embodiments.

[0139] Furthermore, the present application also provides a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the image enhancement method in severe weather described in any of the above embodiments.

[0140] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A method for image enhancement in bad weather, characterized in that: The method comprises: Read the image frames of the video to be processed and perform LAB color space conversion to determine the degraded image frames and corresponding image types; the image types include dust images and haze images; When the degraded image frame is a dust image, the dust image is converted into a grayscale image, a power operation is performed on the grayscale pixels on the grayscale image, and the calculated adjusted grayscale values ​​are mapped to form a dust-free image output; When the degraded image frame is a haze image, the haze image is subjected to image edge filtering, and a transmittance image of the filtered image is extracted; specifically, the haze image is filtered using a gradient domain guided filter, which is expressed as follows: Among them and They represent the coefficients of the filter obtained by linear regression with pixel j as the center, Represents the pixel value of the output filtered image at position i, is the pixel value of the input haze image at position i, represents the window centered at pixel i; Constructing the filter loss function , filter output with the goal of minimizing loss output, filter loss function It is expressed as follows: Among them represents the edge-aware weighting function, represents the edge-aware weighting coefficient, Represents the relevant parameters of the e function, Represents the grayscale value of pixels within the analysis window with local differences, Represents the pixel value of the dark channel image at position i of the haze image; Extracting a binary image and a gradient image based on the filtered image, performing an AND operation on the binary image and the gradient image to obtain an edge image; Performing edge tracking on the edge image, and marking the area with the largest area as the sky area; In response to detecting that the filtered image includes a sky area, extracting the transmittance value of the sky area mapped in the transmittance image, performing deviation correction on the transmittance value of the sky area, and obtaining a corrected transmittance image; specifically, correcting the transmittance value of the sky area according to the following values: Where L1 and L2 are the transmittance thresholds for suppressing color distortion. is the corrected transmittance value after color correction of the sky area; Remapping based on the corrected transmittance of the sky area and the transmittance of the non-sky area to form a corrected transmittance image; The haze image is dehazed based on the corrected transmittance image to obtain a dehazed image.

2. The method according to claim 1, characterized in that The step of reading the image frames of the video to be processed and performing LAB color space conversion to determine the degraded image frames and corresponding image types includes: The read image frame is converted into LAB color space, and the chromaticity component, average chromaticity component and chromaticity center distance representing yellow on the LAB chromaticity plane are calculated according to the resolution size of the image frame; The ratio of the average chromaticity to the chromaticity center distance is determined as a color cast factor, and when the color cast factor does not exceed a set threshold, it is determined as a normal image frame; when the color cast factor exceeds the set threshold, it is determined as a degraded image frame; When the chrominance component representing yellow in the LAB chrominance plane of the degraded image frame is a positive number, the degraded image frame is determined to be a dust image.

3. The method according to claim 1, characterized in that The method performs power operation on the grayscale pixels on the grayscale image and maps the calculated adjusted grayscale values ​​to form a dust-removed image output; comprising: Extracting the grayscale value of the grayscale image at the target pixel position, performing power calculation with the grayscale value as the base and the adjustment coefficient as the exponent, and using the calculation result as the adjusted grayscale value of the target position; Remapping the adjusted grayscale value into the grayscale image to form an adjusted grayscale image; The adjusted grayscale image is restored to obtain a dust-removed image.

4. The method according to claim 2, characterized in that: The step of extracting a transmittance image of the filtered image comprises: Obtain a dark channel image of the haze image, and arrange the pixels in the dark channel image according to the pixel value, and set the pixel value ranked in the top 0.1% as the pixel threshold; All pixel values ​​of the dark channel image are compared based on the pixel threshold to generate a binary mask image; points in the dark channel image that are less than the pixel threshold are set to 0, and points that are not less than the pixel threshold are set to 1; Multiply the generated mask image bitwise with the haze image converted to HSV format, and output the brightness channel value; All brightness channel values ​​are weighted averaged to obtain the atmospheric light component A; Substitute the pixels of each point in the filtered image into the transmittance formula to calculate the transmittance of each point ; Among them represents the deblurring coefficient, c is a color channel of the image, is the dark channel image calculated based on the filtered image; The transmittance image is formed by mapping the transmittance of each point of the filter image according to the pixel position.

5. The method according to claim 1, characterized in that The step of performing image defogging on the haze image based on the corrected transmittance image to obtain a defogged image includes: Substitute the corrected transmittance image and haze image into the following formula: Among them It is a haze image. is the transmittance threshold, It is a dehazed image.

6. An image enhancement device for severe weather conditions, characterized in that: The device comprises: An image determination module is used to read the image frames of the video to be processed and perform LAB color space conversion to determine the degraded image frames and corresponding image types; the image types include dust images and haze images; An image dust removal module, for converting the dust image into a grayscale image when the degraded image frame is a dust image, performing power operations on the grayscale pixels on the grayscale image, and mapping the calculated adjusted grayscale values ​​to form a dust-removed image output; The image extraction module is used for performing image edge filtering on the haze image when the degraded image frame is a haze image, and extracting a transmittance image of the filtered image; specifically, the haze image is filtered using a gradient domain guided filter, which is expressed as follows: Among them and They represent the coefficients of the filter obtained by linear regression with pixel j as the center, Represents the pixel value of the output filtered image at position i, is the pixel value of the input haze image at position i, represents the window centered at pixel i; Constructing the filter loss function , filter output with the goal of minimizing loss output, filter loss function It is expressed as follows: Among them represents the edge-aware weighting function, represents the edge-aware weighting coefficient, Represents the relevant parameters of the e function, Represents the grayscale value of pixels within the analysis window with local differences, Represents the pixel value of the dark channel image at position i of the haze image; Extracting a binary image and a gradient image based on the filtered image, performing an AND operation on the binary image and the gradient image to obtain an edge image; Performing edge tracking on the edge image, and marking the area with the largest area as the sky area; The image correction module is used for, in response to detecting that the filtered image contains a sky area, extracting the transmittance value of the sky area mapped in the transmittance image, performing deviation correction on the transmittance value of the sky area, and obtaining a corrected transmittance image; specifically, correcting the transmittance value of the sky area according to the following values: Where L1 and L2 are the transmittance thresholds for suppressing color distortion. is the corrected transmittance value after color correction of the sky area; Remapping based on the corrected transmittance of the sky area and the transmittance of the non-sky area to form a corrected transmittance image; The image defogging module is used to perform image defogging on the haze image based on the corrected transmittance image to obtain a defogging image.

7. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the image enhancement method in severe weather as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the image enhancement method in severe weather as described in any one of claims 1 to 5.

Images