Image defogging method, device, terminal, base station and chip

Abstract

The application relates to an image defogging method and device, a terminal, a base station and a chip, which can be used in the technical field of image processing. The method comprises the following steps: obtaining a first down-sampling image and a second down-sampling image based on a to-be-processed image; obtaining atmospheric light information and transmittance information based on the second down-sampling image; obtaining residual information based on the atmospheric light information, the transmittance information and the first down-sampling image; the residual information is used for representing image difference information before and after defogging; and the to-be-processed image and the residual information are fused to obtain a defogged image corresponding to the to-be-processed image. The method can improve the image defogging effect.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image dehazing method, apparatus, terminal, base station, and chip. Background Technology

[0002] With the development of photography technology, night scene shooting has become an important function of mobile phone cameras. However, night scene images often produce a foggy feeling due to complex light sources and long exposure times, resulting in reduced contrast and saturation, which seriously affects the image quality.

[0003] In related technologies, dehazing algorithms based on atmospheric degradation models are mainly used for image dehazing. However, these methods are prone to producing layering phenomena when adjusting the color and contrast of the image, resulting in poor image dehazing effects. Summary of the Invention

[0004] Therefore, it is necessary to provide an image dehazing method, apparatus, terminal, base station, and chip that can improve the image dehazing effect in response to the above-mentioned technical problems.

[0005] Firstly, this application provides an image dehazing method. The method includes:

[0006] Based on the image to be processed, a first downsampled image and a second downsampled image are obtained;

[0007] Based on the second downsampled image, atmospheric light information and transmittance information are obtained;

[0008] Based on the atmospheric light information, the transmittance information, and the first downsampled image, residual information is obtained; the residual information is used to characterize the image difference information before and after defogging.

[0009] The image to be processed and the residual information are fused to obtain the dehazed image corresponding to the image to be processed.

[0010] In one embodiment, the residual information is a residual map;

[0011] The residual information obtained based on the atmospheric light information, the transmittance information, and the first downsampled image includes:

[0012] The atmospheric light information and the transmittance information are upsampled respectively to obtain upsampled atmospheric light information and upsampled transmittance information.

[0013] Based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image, a basic dehazed image is obtained; the target first downsampled image is obtained by merging the first downsampled image.

[0014] The residual image is obtained based on the base dehazed image and the target first downsampled image.

[0015] In one embodiment, obtaining a first downsampled image and a second downsampled image based on the image to be processed includes:

[0016] The image to be processed is split into multiple channel images;

[0017] The multiple channel images are subjected to a first downsampling process to obtain a first downsampled image, and the multiple channel images are subjected to a second downsampling process to obtain a second downsampled image.

[0018] In one embodiment, obtaining atmospheric light information and transmittance information based on the second downsampled image includes:

[0019] Based on the second downsampled image, a target second downsampled image and a dark channel image are obtained; the target second downsampled image is obtained by averaging the second downsampled image.

[0020] Based on the second downsampled image of the target, the atmospheric light information is obtained, and based on the dark channel image, the transmittance information is obtained.

[0021] In one embodiment, the atmospheric light information is an atmospheric light map;

[0022] The process of obtaining the atmospheric light information based on the second downsampled image of the target includes:

[0023] The second downsampled image of the target is subjected to maximum value filtering to obtain the initial atmospheric light map;

[0024] The initial atmospheric light map is subjected to guided filtering to obtain the atmospheric light map.

[0025] In one embodiment, the transmittance information is a transmittance map;

[0026] The process of obtaining the transmittance information based on the dark channel map includes:

[0027] Based on the dark channel map and the atmospheric light information, an initial transmittance map is obtained;

[0028] The initial transmittance map is subjected to guided filtering to obtain the transmittance map.

[0029] Secondly, this application also provides an image dehazing apparatus. The apparatus includes:

[0030] The image determination module is used to obtain a first downsampled image and a second downsampled image based on the image to be processed;

[0031] The information determination module is used to obtain atmospheric light information and transmittance information based on the second downsampled image;

[0032] The residual determination module is used to obtain residual information based on the atmospheric light information, the transmittance information, and the first downsampled image; the residual information is used to characterize the image difference information before and after defogging.

[0033] The information processing module is used to fuse the image to be processed and the residual information to obtain the dehazed image corresponding to the image to be processed.

[0034] Thirdly, this application also provides a terminal. The terminal includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0035] Based on the image to be processed, a first downsampled image and a second downsampled image are obtained;

[0036] Based on the second downsampled image, atmospheric light information and transmittance information are obtained;

[0037] Based on the atmospheric light information, the transmittance information, and the first downsampled image, residual information is obtained; the residual information is used to characterize the image difference information before and after defogging.

[0038] The image to be processed and the residual information are fused to obtain the dehazed image corresponding to the image to be processed.

[0039] Fourthly, this application also provides a base station. The base station includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0040] Based on the image to be processed, a first downsampled image and a second downsampled image are obtained;

[0041] Based on the second downsampled image, atmospheric light information and transmittance information are obtained;

[0042] Based on the atmospheric light information, the transmittance information, and the first downsampled image, residual information is obtained; the residual information is used to characterize the image difference information before and after defogging.

[0043] The image to be processed and the residual information are fused to obtain the dehazed image corresponding to the image to be processed.

[0044] Fifthly, this application also provides a chip. The chip includes at least one processor, which, when executing the computer program, performs the following steps:

[0045] Based on the image to be processed, a first downsampled image and a second downsampled image are obtained;

[0046] Based on the second downsampled image, atmospheric light information and transmittance information are obtained;

[0047] Based on the atmospheric light information, the transmittance information, and the first downsampled image, residual information is obtained; the residual information is used to characterize the image difference information before and after defogging.

[0048] The image to be processed and the residual information are fused to obtain the dehazed image corresponding to the image to be processed.

[0049] The aforementioned image dehazing method, apparatus, terminal, base station, and chip obtain a first downsampled image and a second downsampled image based on the image to be processed; obtain atmospheric light information and transmittance information based on the second downsampled image; obtain residual information based on the atmospheric light information, the transmittance information, and the first downsampled image; the residual information is used to characterize the image difference information before and after dehazing; and perform fusion processing on the image to be processed and the residual information to obtain the dehazed image corresponding to the image to be processed. This scheme reduces computational complexity and preserves key image features by downsampling the image to be processed at different ratios to obtain a first downsampled image and a second downsampled image. Obtaining atmospheric light and transmittance information from the second downsampled image helps accurately estimate the fog distribution in the image, providing accurate parameters for subsequent image dehazing. Residual information derived from atmospheric light, transmittance, and the first downsampled image accurately characterizes the differences between the image before and after dehazing, allowing for better control of the degree and scope of dehazing. Finally, fusing the image to be processed and the residual information helps achieve dehazing while preserving the original details and texture information of the image, thus improving the accuracy and quality of image dehazing and ultimately enhancing the overall dehazing effect. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating an image dehazing method in one embodiment;

[0052] Figure 2This is a flowchart illustrating an image dehazing method in another embodiment;

[0053] Figure 3 This is a flowchart illustrating the preprocessing steps in one embodiment;

[0054] Figure 4 This is a flowchart illustrating the image dehazing method in yet another embodiment;

[0055] Figure 5 This is a structural block diagram of an image dehazing device in one embodiment;

[0056] Figure 6 This is an internal structural diagram of a computer device in one embodiment;

[0057] Figure 7 This is an internal structural diagram of another computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] It should be noted that the terms "comprising" and "having," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusion. The term "multiple" as used in this application refers to two or more. The term "and / or" as used in this application refers to one of the solutions, or any combination of multiple solutions.

[0060] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0061] In one exemplary embodiment, such as Figure 1 As shown, an image dehazing method is provided. This embodiment illustrates the application of this method to an electronic device, where the electronic device can refer to any device including memory, such as a terminal, base station, server, or one or more other devices. The electronic device may have a chip deployed on it, and the method can be executed through the chip. Furthermore, this method can also be applied to chip modules, virtual devices, and storage media. For example, a virtual device may include a chip, and the various modules within the virtual device may include software and / or hardware.

[0062] In this embodiment, the method may include the following steps:

[0063] Step S101: Based on the image to be processed, obtain the first downsampled image and the second downsampled image;

[0064] Step S102: Based on the second downsampled image, atmospheric light information and transmittance information are obtained;

[0065] Step S103: Based on atmospheric light information, transmittance information, and the first downsampled image, residual information is obtained; the residual information is used to characterize the image difference information before and after defogging.

[0066] Step S104: Perform fusion processing on the image to be processed and the residual information to obtain the dehazed image corresponding to the image to be processed.

[0067] The image to be processed can be a raw image processed from the image sensor of a mobile phone camera, digital camera, scanner, or film scanner. This raw image file contains data processed from the image sensor, which has not yet been processed, printed, or used for editing, and typically has a wide color gamut.

[0068] The first downsampled image can be a four-channel image obtained by downsampling the image to be processed by 2 times. The four channels include the R (red) channel, GR (green-red) channel, GB (green-blue) channel, and B (blue) channel.

[0069] Among them, the Bayer array is one of the main technologies for enabling CCD (charge-coupled device) or CMOS (complementary metal-oxide-semiconductor) sensors to capture color images. The Bayer array is a 2×2 array, which is usually composed of 2 green (G channel), 1 blue (B channel) and 1 red (R channel) pixels, and uses a 1 red 2 green 1 blue arrangement to convert grayscale information into color information.

[0070] The second downsampled image can be a four-channel image obtained by downsampling the image to be processed by a factor of 4.

[0071] Among them, atmospheric light information can be an atmospheric light map, which can be called atmospheric light map A.

[0072] Among them, the transmittance information can be a transmittance map, which can be called transmittance map t.

[0073] The residual information can be a residual map, such as a residual map R' obtained by subtracting the hazy image from the dehazed image at a 1 / 2 scale. This residual map R' is then refined by a fast guided filter and upsampled by a factor of 2 to obtain the residual map R. This residual map R contains the difference information before and after dehazing.

[0074] Among them, fast guided filtering can be a fast implementation of guided filtering. This fast implementation performs fast guided filtering after image downsampling, and outputs the filtered result at the original size, with the output resolution being higher than the input resolution.

[0075] The dehazed image can be obtained by fusing the image to be processed and the residual information. This fusing process involves adding the residual image R back to the original input image to be processed to obtain the original dehazed image file.

[0076] Optionally, the electronic device acquires an image to be processed, which is an image in the original image file format; based on the image to be processed, the electronic device obtains a first downsampled image and a second downsampled image, both of which are smaller than the size of the image to be processed; based on the second downsampled image, the electronic device obtains atmospheric light information and transmittance information, whereby the atmospheric light information is used to characterize the atmospheric light distribution in the image and the transmittance information is used to characterize the attenuation of light after passing through fog; based on the atmospheric light information, transmittance information, and the first downsampled image, the electronic device obtains residual information, which is used to characterize the image difference information before and after dehazing; the electronic device performs fusion processing on the image to be processed and the residual information to obtain a dehazed image corresponding to the image to be processed, wherein the contrast and saturation of the dehazed image are improved compared to the image to be processed.

[0077] For example, an electronic device acquires an image to be processed, which is an image in raw image file format processed from the image sensor of a mobile phone camera, digital camera, scanner, or film scanner. This image contains data that has not yet been processed, printed, or used for editing. The electronic device splits the image to be processed into four channels—R channel, GR channel, GB channel, and B channel—according to the Bayer array of the image sensor. Then, it performs a first downsampling process and a second downsampling process on each of these four channels, obtaining a first downsampled image and a second downsampled image. The downsampling factor of the first downsampling process is less than that of the second downsampling process; for example, the first downsampling process is 2x downsampling, and the second downsampling process is 4x downsampling. The electronic device then averages the second downsampled image to obtain the target image. The first downsampled image is processed by performing minimum value processing to obtain a dark channel image. Atmospheric light information is obtained by performing maximum value filtering and guided filtering on the target second downsampled image. Transmittance information is obtained based on the dark channel image and atmospheric light information. This atmospheric light information is replaced with an atmospheric light map instead of the globally unique atmospheric light value to adapt to complex light sources in night scene mode. The electronic device performs upsampling processing on the atmospheric light information and transmittance information respectively. The first downsampled image is then merged to obtain the target first downsampled image. Based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image, a base dehazed image is obtained. Residual information is obtained based on the base dehazed image and the target first downsampled image. The electronic device then adds the residual information back to the image to be processed to obtain the corresponding dehazed image.

[0078] In the above image dehazing method, a first downsampled image and a second downsampled image are obtained based on the image to be processed; atmospheric light information and transmittance information are obtained based on the second downsampled image; residual information is obtained based on the atmospheric light information, transmittance information and the first downsampled image; the residual information is used to characterize the image difference information before and after dehazing; the image to be processed and the residual information are fused to obtain the dehazed image corresponding to the image to be processed. This scheme reduces computational complexity and preserves key image features by downsampling the image to be processed at different ratios to obtain a first downsampled image and a second downsampled image. Obtaining atmospheric light and transmittance information from the second downsampled image helps accurately estimate the fog distribution in the image, providing accurate parameters for subsequent image dehazing. Residual information derived from atmospheric light, transmittance, and the first downsampled image accurately characterizes the differences between the image before and after dehazing, allowing for better control of the degree and scope of dehazing. Finally, fusing the image to be processed and the residual information helps achieve dehazing while preserving the original details and texture information of the image, thus improving the accuracy and quality of image dehazing and ultimately enhancing the overall dehazing effect.

[0079] In an exemplary embodiment, step S103 above, obtaining residual information based on atmospheric light information, transmittance information, and the first downsampled image, specifically includes the following: performing upsampling processing on the atmospheric light information and transmittance information respectively to obtain upsampled atmospheric light information and upsampled transmittance information; obtaining a basic dehazed image based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image; obtaining the target first downsampled image by merging the first downsampled image; and obtaining a residual map based on the basic dehazed image and the target first downsampled image.

[0080] The residual information is a residual map, which can be the residual map R' obtained by subtracting the hazy image from the dehazed image at a half-scale.

[0081] Upsampling can be a process of enlarging the size of an image. This enlargement is achieved by interpolating the pixels of the original image. Interpolation methods include nearest neighbor interpolation and bilinear interpolation.

[0082] The upsampled atmospheric light information can be an atmospheric light map A obtained by upsampling the atmospheric light information by 4 times. The width and height of the atmospheric light map A are magnified to 4 times that of the atmospheric light information. For example, when the size of the atmospheric light information is 500 in width and 375 in height, the size of the upsampled atmospheric light information is 2000 in width and 1500 in height.

[0083] The upsampled transmittance information can be a transmittance map t obtained by upsampling the transmittance information by 4 times. The width and height of the transmittance map t are magnified to 4 times that of the transmittance information. For example, when the transmittance information size is 500 wide and 375 high, the size of the upsampled transmittance information is 2000 wide and 1500 high.

[0084] The target first downsampled image can be image I' obtained by merging the first downsampled image. This merging process involves stitching the first downsampled image back into the original image file format according to the Bayer array, resulting in an image with the same size as the first downsampled image.

[0085] The base dehazed image can be a file J' that is half the size of the original image after dehazing.

[0086] Optionally, the electronic device upsamples the atmospheric light information and transmittance information respectively to obtain upsampled atmospheric light information and upsampled transmittance information; the electronic device merges the first downsampled image to obtain a target first downsampled image, which is in the format of the original image file; based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image, the electronic device obtains a base dehazed image, the size of which is the same as the size of the target first downsampled image; based on the base dehazed image and the target first downsampled image, the electronic device obtains a residual map, the size of which is the same as the size of the image to be processed, and the residual map is used to characterize the image difference information before and after dehazing.

[0087] For example, the electronic device upsamples the atmospheric light information and transmittance information by a factor of 4, enlarging the width and height of the atmospheric light information and transmittance information to four times their original size, resulting in upsampled atmospheric light information and upsampled transmittance information. The size of the upsampled atmospheric light information and upsampled transmittance information is the same as the size of the first downsampled image. The electronic device then stitches the first downsampled image back into the original image file format using a Bayer array to obtain the target first downsampled image. Based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image, the electronic device obtains a basic dehazed image. Specifically, this can be (target first downsampled image - (1 - upsampled transmittance information) × upsampled atmospheric light information) divided by the upsampled transmittance information, resulting in a dehazed original image file that is half the size of the original image file, which serves as the basic dehazed image. The electronic device subtracts the target first downsampled image from the basic dehazed image to obtain the original... Figure 2 A residual map R' of half the size is obtained, which contains the difference information before and after dehazing at half the scale. Then, the residual map R' is refined by fast guided filtering and the output after 2x upsampling is obtained. The residual map R with the same size as the image to be processed is used as the residual information, where the guide map of fast guided filtering is the image to be processed.

[0088] The technical solution provided in this embodiment, by upsampling the atmospheric light information and transmittance information, helps to match the size of the atmospheric light information and transmittance information with the size of the first downsampled image. This facilitates the accurate calculation of the base dehazed image based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image. By obtaining a residual map based on the base dehazed image and the target first downsampled image, it is beneficial to extract the difference information before and after dehazing, thereby reducing computational complexity and improving the efficiency of image dehazing.

[0089] In an exemplary embodiment, step S101 above, which obtains a first downsampled image and a second downsampled image based on the image to be processed, specifically includes the following: splitting the image to be processed into multiple channel images; performing a first downsampling process on the multiple channel images to obtain a first downsampled image; and performing a second downsampling process on the multiple channel images to obtain a second downsampled image.

[0090] The multiple channel images can be four channel images obtained by splitting the image to be processed according to the Bayer array of the image sensor: R channel, GR channel, GB channel, and B channel. Each channel image has the same size. For example, when the image to be processed is 4000 pixels wide and 3000 pixels high, the size of each channel image after splitting is 2000 pixels wide and 1500 pixels high.

[0091] The first downsampling process can be to downsample multiple channel images by a factor of 2.

[0092] The second downsampling process can be to downsample multiple channel images by a factor of 4.

[0093] Optionally, the electronic device splits the image to be processed into four channels—R, GR, GB, and B—based on the Bayer array of the image sensor. Each channel image has the same size after splitting. The electronic device then performs a 2x downsampling process on each of the four channels, reducing the width and height of each channel image to twice its original size, resulting in four downsampled channels: R, GR, GB, and B, which serve as the first downsampled image. Finally, the electronic device performs a 4x downsampling process on each of the four channels, reducing the width and height of each channel image to four times its original size, resulting in four downsampled channels: R, GR, GB, and B, which serve as the second downsampled image.

[0094] The technical solution provided in this embodiment, by splitting the image to be processed into multiple channel images, facilitates the separate processing of different color channel information of the image to be processed, thereby helping to preserve the color features of the image to be processed; by performing first downsampling processing and second downsampling processing on multiple channel images, it is beneficial to obtain downsampling results of the image to be processed at different scales, thereby helping to reduce the computational complexity of subsequent processing and improve the processing speed, and improve the efficiency of image dehazing.

[0095] In an exemplary embodiment, step S102 above, obtaining atmospheric light information and transmittance information based on the second downsampled image, specifically includes the following: obtaining a target second downsampled image and a dark channel image based on the second downsampled image; the target second downsampled image is obtained by averaging the second downsampled image; atmospheric light information is obtained based on the target second downsampled image, and transmittance information is obtained based on the dark channel image.

[0096] The target second downsampled image can be an image obtained by averaging the four channels of the second downsampled image (which can be called a Y-image).

[0097] The averaging process can be the process of averaging the pixel values ​​of the four channels (R, GR, GB, and B) of the second downsampled image.

[0098] The dark channel image can be obtained by performing minimum value filtering on the minimum value image. This dark channel image is obtained by traversing the four channels of the second downsampled image, finding the minimum value in the four channels at the same pixel position, and then performing minimum value filtering on the minimum value image. This dark channel image is used to calculate transmittance information.

[0099] The minimum value image can be obtained by traversing the four channels of the second downsampled image and finding the minimum value among the four channels at the same pixel position. The size of this image is the same as the size of each channel of the second downsampled image. For example, when the size of each channel of the second downsampled image is 500 in width and 375 in height, the size of the minimum value image is 500 in width and 375 in height.

[0100] Optionally, the electronic device averages the four channels (R, GR, GB, and B) of the second downsampled image. It adds the pixel values ​​of the four channels at the same pixel location and divides the sum by 4 to obtain the average value for that pixel, thus obtaining the target second downsampled image. The electronic device then iterates through the four channels (R, GR, GB, and B) of the second downsampled image, finding the minimum value among these four channels at the same pixel location, obtaining a minimum value image. Finally, it performs minimum value filtering on the minimum value image, i.e., iterates through the entire minimum value image and filters the minimum value for each pixel. The value is updated to the minimum value in the neighborhood of the pixel, resulting in the dark channel map. The electronic device performs maximum value filtering on the second downsampled image of the target, that is, it traverses the entire second downsampled image of the target and updates the value of each pixel to the maximum value in the neighborhood of that pixel, resulting in the maximum value map. Then, the maximum value map is refined using guided filtering to obtain atmospheric light information, where the guided map is the second downsampled image of the target. The electronic device obtains an initial transmittance map based on the dark channel map and atmospheric light information, and refines the initial transmittance map using guided filtering to obtain a transmittance map as transmittance information, where the guided map is the second downsampled image of the target.

[0101] The technical solution provided in this embodiment obtains the target second downsampled image by averaging the second downsampled image, which is beneficial for integrating information from multiple channels to obtain a more accurate image feature representation; obtaining the dark channel map based on the second downsampled image is beneficial for extracting dark channel feature information in the image, thus providing an accurate basis for calculating transmittance information; obtaining atmospheric light information based on the target second downsampled image is beneficial for accurately estimating the atmospheric light distribution in the image for complex light sources in night scene mode, thus improving the accuracy and adaptability of image dehazing.

[0102] In an exemplary embodiment, the above steps, based on the second downsampled image of the target, to obtain atmospheric light information, specifically include the following: performing maximum value filtering on the second downsampled image of the target to obtain an initial atmospheric light map; and performing guided filtering on the initial atmospheric light map to obtain an atmospheric light map.

[0103] Among them, atmospheric light information is atmospheric light map.

[0104] The maximum value filtering process can be an image filtering operation that iterates through the second downsampled image of the target and updates the value of each pixel in the second downsampled image of the target to the maximum value in the neighborhood of that pixel.

[0105] The initial atmospheric light map can be a maximum value map obtained by performing maximum value filtering on the second downsampled image of the target. The value of each pixel in the maximum value map is the maximum value in the neighborhood of that pixel.

[0106] Guided filtering can be performed by filtering the target image using a guide image, so that the final output image is generally similar to the target image, but the texture is similar to the guide image. For example, when performing guided filtering on the initial atmospheric light map, the guide image is the second downsampled image of the target image, and the target image is the initial atmospheric light map, thus refining the initial atmospheric light map to obtain the atmospheric light map.

[0107] Optionally, the electronic device performs maximum value filtering on the target second downsampled image, that is, it traverses the entire target second downsampled image and updates the value of each pixel in the target second downsampled image to the maximum value in the neighborhood of that pixel, thereby obtaining an initial atmospheric light map. The size of the initial atmospheric light map is the same as the size of the target second downsampled image. The electronic device performs guided filtering on the initial atmospheric light map, using the target second downsampled image as the guide image and the initial atmospheric light map as the target image. The guided filtering refines the initial atmospheric light map, making the output image generally similar to the initial atmospheric light map, but the texture part is similar to the target second downsampled image, thus obtaining an atmospheric light map. The size of the atmospheric light map is the same as the size of the initial atmospheric light map. The atmospheric light map replaces the globally unique atmospheric light value to adapt to the complex light source situation in the night scene mode, thereby improving the image dehazing adaptability to complex light source situations in the night scene image.

[0108] The technical solution provided in this embodiment obtains an initial atmospheric light map by performing maximum value filtering on the second downsampled image of the target, which is beneficial for extracting the maximum brightness information in the second downsampled image of the target to estimate the atmospheric light distribution; by performing guided filtering on the initial atmospheric light map to obtain an atmospheric light map, it is beneficial for obtaining a more accurate atmospheric light map, and using the atmospheric light map to replace the globally unique atmospheric light value can adapt to the situation of complex light sources in night scene mode, thereby improving the image dehazing adaptability to night scene images.

[0109] In an exemplary embodiment, the above steps, based on the dark channel map, obtain transmittance information, specifically including the following: based on the dark channel map and atmospheric light information, obtain an initial transmittance map; perform guided filtering on the initial transmittance map to obtain the transmittance map.

[0110] The transmittance information is presented as a transmittance map.

[0111] Optionally, the electronic device obtains an initial transmittance map based on the dark channel image and atmospheric light information. The size of the initial transmittance map is the same as the size of the dark channel image and atmospheric light information. The electronic device performs guided filtering on the initial transmittance map, using the second downsampled image of the target as the guide image and the initial transmittance map as the target image. The initial transmittance map is refined by guided filtering so that the output image is generally similar to the initial transmittance map, but the texture is similar to the second downsampled image of the target, thus obtaining a transmittance map with the same size as the initial transmittance map.

[0112] For example, the electronic device calculates an initial transmittance map based on the dark channel image and atmospheric light information according to the transmittance calculation method. This calculation method means that transmittance equals 1 - (dark channel image divided by atmospheric light information). The size of the initial transmittance map is the same as the size of the dark channel image and the atmospheric light information. The electronic device performs guided filtering on the initial transmittance map, using the target second downsampled image as the guide image and the initial transmittance map as the target image. By filtering the initial transmittance map through guided filtering, the output image is generally similar to the initial transmittance map, but the texture is similar to the target second downsampled image, thus obtaining a transmittance map with the same size as the initial transmittance map.

[0113] The technical solution provided in this embodiment obtains an initial transmittance map based on the dark channel map and atmospheric light information, which is beneficial for accurately calculating transmittance information and thus providing accurate transmittance parameters for subsequent image dehazing. By performing guided filtering on the initial transmittance map to obtain a more accurate transmittance map, the accuracy of transmittance estimation is improved, thereby improving the accuracy and effect of image dehazing.

[0114] The following is an example illustrating the image dehazing method provided in this application. This example demonstrates the application of this method to an electronic device.

[0115] When shooting at night, due to the low ambient light, the camera automatically lengthens the exposure time to absorb more light and make the image brighter. However, the light sources at night are complex, with varying intensity, direction, and color. Excessive exposure time can cause the image to appear hazy or foggy, resulting in lower contrast and saturation, color shifts, and distortion, making it difficult to capture the true atmosphere of the night scene. Image dehazing algorithms can remove the fog from night scene images, making them clearer and more transparent.

[0116] Dehazing algorithms can be divided into image enhancement-based dehazing and image restoration-based dehazing. Image enhancement-based dehazing algorithms aim to remove image noise and improve image contrast to restore a clear, fog-free image. One method is CLAHE (Contrast-Limited Adaptive Histogram Equalization). Image restoration-based dehazing algorithms are primarily based on atmospheric degradation models, performing dehazing accordingly. Algorithms include those based on dark channels. Dehazing based on atmospheric degradation models generally outperforms image enhancement-based algorithms. Therefore, current dehazing algorithms calculate the dark channels of RGB (red, green, blue) images and combine this with a scattering model of the foggy image for dehazing. In computer vision, a foggy image degradation model is used to describe the impact of severe weather conditions such as fog and haze on images. The expression is:

[0117] I(x)=J(x)t(x)+A(1-t(x)).

[0118] Where I represents the foggy image, J represents the fog-free image, t represents transmittance, and A represents atmospheric light. Therefore, the dehazing algorithm based on the fog degradation model works in the RGB domain, solving for the transmittance and global atmospheric light from the known foggy image, and finally calculating the fog-free image J. However, unlike daytime, nighttime light sources are complex. During the day, atmospheric light mainly comes from sunlight, while at night it comes from various lights, making the estimation of A more complex.

[0119] However, RGB images processed by ISP (Image Signal Processing) have a bit width of 8 bits. When adjusting their color and contrast, they are prone to layering. But when processing in the raw (original image) domain with a higher bit width, the computational load increases and the processing time increases. Moreover, when solving J according to the above formula, there is a division calculation, which is difficult to optimize. Secondly, when converting the raw domain to the RGB domain for de-mosaicing, it is easy to produce a mosaic-like grid distribution at the edges of objects, which is especially obvious in backlit night scenes. Finally, the dehazing algorithm based on the dark channel will reduce the brightness of the image, which is obviously unreasonable for night scene images.

[0120] This embodiment processes the input raw image; the main process is as follows: Figure 2 The process includes input image, preprocessing, dehazing, residual image, post-processing, and output image. The following explanation uses a 12MB raw image (4000 pixels wide and 3000 pixels high) as an example.

[0121] Step 1: Preprocess the input raw image I. Refer to [the relevant documentation / reference]. Figure 3 :

[0122] Based on the Bayer array of the image sensor, the original image is split into four channels: R channel, GR channel, GB channel, and B channel. Each channel is 2000×1500 in size.

[0123] The four channels were downsampled by 2x (1000×750) and 4x (500×375) respectively.

[0124] The result of the 2x downsampling is stitched back to the original image file format (2000×1500) using the Bayer array, and denoted as I', which is used when calculating the residual map at the end;

[0125] The average of the four channels downsampled by 4 times is used to obtain a Y-image (500×375), which is then used to calculate the atmospheric light image A.

[0126] For example, refer to Figure 3 The process includes the following steps: input the original image I, split it into four channels according to the Bayer array to obtain the R channel, GR channel, GB channel and B channel image, then perform half downsampling and quarter downsampling, merge the four channels according to the Bayer array to obtain I', and average the four channels to obtain the Y image.

[0127] Step 2: Based on the four channels obtained from the quarter-sampled sample in the previous step, solve for atmospheric light A in the image fog degradation model:

[0128] The above Y-image is subjected to maximum value filtering, that is, the entire Y-image is traversed and the value of each pixel is updated to the maximum value in its neighborhood;

[0129] Then, the maximum value map is refined by guided filtering. The guided map is a Y map, resulting in atmospheric light map A (500×375).

[0130] Step 3: Solve for the transmittance t in the image fog degradation model:

[0131] Traverse the four channels of the image after one-quarter sampling in step 1, find the minimum value of the four channels at the same pixel location, and obtain a minimum value image. Then, perform minimum value filtering on this minimum value image, that is, traverse the entire minimum value image and update the value of each pixel to the minimum value in its neighborhood, to obtain the dark channel image I. dark ;

[0132] Calculate the transmittance map t: t = 1 - (I dark / A).

[0133] The transmittance map t above is refined using guided filtering to obtain the final transmittance map t, which has a size of 500×375.

[0134] Step 4: Perform dehazing calculations and obtain the residual map:

[0135] The transmittance map t and atmospheric light map A above were upsampled by 4 times, and the size after upsampling was 2000×1500.

[0136] Substituting t, A, and I' from step 1 into the following formula, we obtain the raw image J' (2000×1500) of half size after dehazing: J'=(I'-(1-t)×A) / t.

[0137] Calculation yields the original Figure 2 The half-scale residual image R': This is calculated by subtracting the hazy image I' from the dehazed image J' at half the scale. This difference contains the information about the differences before and after dehazing. Then, a fast guided filter is used to refine it, and the output after 2x upsampling is obtained. The guided image is the original input image, and the final residual image R is obtained. The size of the residual image R is 4000×3000. The calculation method is: R'=J'-I'.

[0138] The information difference after dehazing is added back to the original input image to obtain the dehazed image J. The calculation method is: J=I+R.

[0139] Step 5: Brighten the dehazed image, where w is a value from 0 to 1, representing the brightening intensity. A larger value results in greater brightening. This can be adjusted according to different scenarios and needs. The brightening information is calculated as follows: J light =J+w×J×(1-J)

[0140] For example, the overall process of this embodiment is as follows: Figure 4 The process includes: inputting the original image, preprocessing it to obtain a half-downsampled I' image (the first downsampled image of the target) and a quarter-downsampled Y image (the second downsampled image of the target). Subsequently, after obtaining the atmospheric light map A and the transmittance map t, upsampling is performed to obtain a half-sized dehazed image and a half-sized residual image. Fast guided filtering, residual addition processing, and image brightening are then performed to finally output the image (dehazed image). Specifically, the atmospheric light map A and the transmittance map t are upsampled and then used together with the first downsampled image of the target to calculate the half-sized dehazed image. The half-sized dehazed image and the first downsampled image of the target are used to calculate the half-sized residual image. The half-sized residual image is then processed by fast guided filtering and the residual is added back to the original image. Finally, image brightening is performed to obtain the dehazed image.

[0141] The technical solution provided in this embodiment obtains a first downsampled image and a second downsampled image by downsampling the image to be processed at different multiples. This helps reduce computational complexity while preserving key image features. By obtaining atmospheric light information and transmittance information based on the second downsampled image, it is beneficial to accurately estimate the fog distribution in the image, thus providing accurate parameter basis for subsequent image dehazing. By obtaining residual information based on atmospheric light information, transmittance information, and the first downsampled image, it is beneficial to accurately characterize the differences between the image before and after dehazing, thus helping to control the degree and scope of dehazing. By fusing the image to be processed and the residual information, it is beneficial to achieve the dehazing effect while preserving the original details and texture information of the image to be processed, thereby improving the accuracy and quality of image dehazing and ultimately enhancing the image dehazing effect.

[0142] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0143] Based on the same inventive concept, this application also provides an image dehazing apparatus for implementing the image dehazing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image dehazing apparatus embodiments provided below can be found in the limitations of the image dehazing method described above, and will not be repeated here.

[0144] In one exemplary embodiment, such as Figure 5 As shown, an image dehazing device 500 is provided, which may include:

[0145] The image determination module 501 is used to obtain a first downsampled image and a second downsampled image based on the image to be processed;

[0146] The information determination module 502 is used to obtain atmospheric light information and transmittance information based on the second downsampled image;

[0147] The residual determination module 503 is used to obtain residual information based on atmospheric light information, transmittance information and the first downsampled image; the residual information is used to characterize the image difference information before and after defogging.

[0148] The information processing module 504 is used to fuse the image to be processed and the residual information to obtain the dehazed image corresponding to the image to be processed.

[0149] In an exemplary embodiment, the residual information is a residual map; the residual determination module 503 is further configured to perform upsampling processing on the atmospheric light information and the transmittance information respectively to obtain upsampled atmospheric light information and upsampled transmittance information; based on the upsampled atmospheric light information, the upsampled transmittance information and the target first downsampled image, a basic dehazed image is obtained; the target first downsampled image is obtained by merging the first downsampled image; based on the basic dehazed image and the target first downsampled image, a residual map is obtained.

[0150] In an exemplary embodiment, the image determination module 501 is further configured to split the image to be processed into multiple channel images; perform a first downsampling process on the multiple channel images to obtain a first downsampled image; and perform a second downsampling process on the multiple channel images to obtain a second downsampled image.

[0151] In an exemplary embodiment, the information determination module 502 is further configured to obtain a target second downsampled image and a dark channel image based on the second downsampled image; the target second downsampled image is obtained by averaging the second downsampled image; atmospheric light information is obtained based on the target second downsampled image, and transmittance information is obtained based on the dark channel image.

[0152] In an exemplary embodiment, the atmospheric light information is an atmospheric light map; the information determination module 502 is further configured to perform maximum value filtering on the second downsampled image of the target to obtain an initial atmospheric light map; and to perform guided filtering on the initial atmospheric light map to obtain another atmospheric light map.

[0153] In an exemplary embodiment, the transmittance information is a transmittance map; the information determination module 502 is further configured to obtain an initial transmittance map based on the dark channel map and atmospheric light information; and to perform guided filtering on the initial transmittance map to obtain the transmittance map.

[0154] Each module in the aforementioned image dehazing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0155] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory may include non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database can store images. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an image dehazing method.

[0156] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image dehazing method. The display unit of the computer device forms a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0157] Those skilled in the art will understand that Figure 6 and Figure 7The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0158] In one exemplary embodiment, a terminal is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0159] In one exemplary embodiment, a base station is provided, including a memory and a processor, the memory storing a computer program, which the processor executes to implement the steps in the above-described method embodiments.

[0160] In one exemplary embodiment, a chip is provided, including at least one processor that executes a computer program to implement the steps in the above-described method embodiments.

[0161] In one exemplary embodiment, a chip module coupled to a memory is provided for implementing the steps in the above method embodiments when executing a computer program stored in the memory.

[0162] In one exemplary embodiment, a server is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0163] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above-described method embodiments.

[0164] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0165] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0166] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image dehazing method, characterized in that, The method includes: Based on the image to be processed, a first downsampled image and a second downsampled image are obtained; Based on the second downsampled image, atmospheric light information and transmittance information are obtained; The atmospheric light information and the transmittance information are upsampled respectively to obtain upsampled atmospheric light information and upsampled transmittance information. Based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image, a basic dehazed image is obtained; the target first downsampled image is obtained by merging the first downsampled image; the merging process is to stitch the first downsampled image back into the original image file format according to the Bayer array to obtain an image with the same size as the first downsampled image; Based on the base dehazed image and the target first downsampled image, a residual map is obtained; the residual map is used to characterize the image difference information before and after dehazing. The image to be processed and the residual image are fused to obtain the dehazed image corresponding to the image to be processed.

2. The method according to claim 1, characterized in that, The process of obtaining a first downsampled image and a second downsampled image based on the image to be processed includes: The image to be processed is split into multiple channel images; The multiple channel images are subjected to a first downsampling process to obtain a first downsampled image, and the multiple channel images are subjected to a second downsampling process to obtain a second downsampled image.

3. The method according to claim 1, characterized in that, The atmospheric light information and transmittance information obtained based on the second downsampled image include: Based on the second downsampled image, a target second downsampled image and a dark channel image are obtained; the target second downsampled image is obtained by averaging the second downsampled image. Based on the second downsampled image of the target, the atmospheric light information is obtained, and based on the dark channel image, the transmittance information is obtained.

4. The method according to claim 3, characterized in that, The atmospheric light information is an atmospheric light map; The process of obtaining the atmospheric light information based on the second downsampled image of the target includes: The second downsampled image of the target is subjected to maximum value filtering to obtain the initial atmospheric light map; The initial atmospheric light map is subjected to guided filtering to obtain the atmospheric light map.

5. The method according to claim 3, characterized in that, The transmittance information is a transmittance map; The process of obtaining the transmittance information based on the dark channel map includes: Based on the dark channel map and the atmospheric light information, an initial transmittance map is obtained; The initial transmittance map is subjected to guided filtering to obtain the transmittance map.

6. An image dehazing device, characterized in that, The device includes: The image determination module is used to obtain a first downsampled image and a second downsampled image based on the image to be processed; The information determination module is used to obtain atmospheric light information and transmittance information based on the second downsampled image; The residual determination module is used to upsample the atmospheric light information and the transmittance information respectively to obtain upsampled atmospheric light information and upsampled transmittance information; based on the upsampled atmospheric light information, the upsampled transmittance information, and the target first downsampled image, a basic dehazed image is obtained; the target first downsampled image is obtained by merging the first downsampled image; the merging process is to stitch the first downsampled image back to the original image file format according to the Bayer array to obtain an image with the same size as the first downsampled image; based on the basic dehazed image and the target first downsampled image, a residual map is obtained; the residual map is used to characterize the image difference information before and after dehazing; The information processing module is used to perform fusion processing on the image to be processed and the residual image to obtain the dehazed image corresponding to the image to be processed.

7. The apparatus according to claim 6, characterized in that, The image determination module is further configured to split the image to be processed into multiple channel images; perform a first downsampling process on the multiple channel images to obtain a first downsampled image; and perform a second downsampling process on the multiple channel images to obtain a second downsampled image.

8. A terminal comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A base station, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

10. A chip, characterized in that, The chip includes at least one processor, which executes a computer program to implement the steps of the method according to any one of claims 1 to 5.

Images