Image defogging method and device and computer readable storage medium
By performing defogging in the brightness domain, the problem of image format limitation in the prior art is solved, and defogging treatment of various image domains is realized, thereby improving user experience.
Patent Information
- Application Number
- CN202310757577.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-25
- Publication Date
- 2024-12-31
AI Technical Summary
The existing defog removal method based on dark channel priors has relatively limited requirements for image formats and can only be applied to the RGB domain, resulting in the defog removal of images in other domains, which affects the user's shooting experience.
By converting the original image into an image in the brightness domain and performing defogging processing in the brightness domain, defogging images in the brightness domain are obtained, thereby realizing defogging processing of the original image, reducing the defogging processing of the image format Restrictions have improved the general applicability of defogging treatment.
Defogging in the brightness domain can improve the effect of defogging, realize defogging treatment of images in other domains (for example, the RAW domain), and enhance the universal applicability and effect of defogging.
Smart Images
Figure CN119228683A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing technology, and in particular to an image defogging method, device and computer-readable storage medium. Background Art
[0002] In order to make the terminal device have a higher screen-to-body ratio, make the screen more complete and have better visual effects, developers can embed the camera under the mobile phone screen to avoid the problem of setting the camera on the screen and reducing the screen-to-body ratio. However, due to the obstruction of the screen, the light sensitivity of the under-screen camera is much lower than that of a normal camera, so the images taken by the under-screen camera are relatively gray, which seriously affects the user's shooting experience.
[0003] At present, the images taken by the under-screen camera are usually processed by the defogging method based on the dark channel prior to restore the image clarity and thus ensure the user's shooting experience. However, the above defogging method based on the dark channel prior has relatively limited requirements on the image format and can only be applied to the red green blue (RGB) domain, which makes it impossible to defog images in other domains. Summary of the invention
[0004] The embodiments of the present application provide an image defogging method, an apparatus, and a computer-readable storage medium, which achieve the purpose of improving the universal applicability of defogging processing and the defogging effect.
[0005] In one aspect, an image defogging method is provided, comprising: obtaining an original image and converting the original image into a first brightness image; determining a brightness histogram of the first brightness image based on pixel values of pixels in the first brightness image, wherein the brightness histogram is used to indicate the number of pixels corresponding to different pixel values in the first brightness image;
[0006] Determine a bottom fog estimation value based on a brightness histogram of the first brightness image, where the bottom fog estimation value is used to characterize the degree of influence of the dustiness of the original image on the brightness;
[0007] Based on the bottom fog estimation value, the first brightness image is defogged to obtain a second brightness image;
[0008] Based on the second brightness image, a defogging image corresponding to the original image is obtained.
[0009] On the other hand, an electronic device is provided, comprising: a memory and a processor; the memory and the processor are coupled; the memory is used to store a computer program; and the image defogging method of any of the above embodiments is implemented when the processor executes the computer program.
[0010] On the other hand, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the image defogging method of any of the above embodiments is implemented.
[0011] On the other hand, a computer program product is provided, which includes computer program instructions, and when the computer program instructions are executed by a processor, the image defogging method of any of the above embodiments is implemented.
[0012] The image defogging method provided in the embodiment of the present application does not need to restrict the format of the original image. It only needs to convert the original image into an image in the brightness domain (i.e., the first brightness image), and perform defogging on the image in the brightness domain to obtain a defogging image in the brightness domain (i.e., the second brightness image). Based on the defogging image in the brightness domain, a defogging image corresponding to the original image is obtained. In this way, on the basis of realizing the defogging of the original image, the restriction of the defogging process on the image format is alleviated, and the defogging of images in other domains (e.g., the raw (RAW) domain) is realized, thereby improving the universal applicability of the defogging process. In addition, since the image in the brightness domain is a grayscale image, defogging the image in the brightness domain will not have a significant impact on the color of each pixel in the image, which can improve the effect of the defogging process. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order to more clearly illustrate the technical solution in the present application, the drawings required for use in some embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only drawings of some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0014] Figure 1 A schematic diagram of the structure of an under-screen camera system provided in some embodiments of the present application;
[0015] Figure 2 A flowchart of a method for processing an image captured by an under-screen camera provided in some embodiments of the present application;
[0016] Figure 3 A schematic diagram of a process of an image defogging method provided in some embodiments of the present application;
[0017] Figure 4 A schematic diagram of a Bayer array provided for some embodiments of the present application;
[0018] Figure 5 A schematic diagram of a histogram provided for some embodiments of the present application;
[0019] Figure 6A schematic diagram of another histogram provided for some embodiments of the present application;
[0020] Figure 7 A schematic diagram of a flow chart of another image defogging method provided in some embodiments of the present application;
[0021] Figure 8 An example diagram of an image provided in some embodiments of the present application;
[0022] Fig. 9 Another example diagram of an image provided for some embodiments of the present application;
[0023] Fig.10 Another example diagram of an image provided for some embodiments of the present application;
[0024] Fig.11 Another example diagram of an image provided for some embodiments of the present application;
[0025] Fig.12 An example diagram of a second brightness image provided in some embodiments of the present application;
[0026] Fig.13 An example diagram of a third brightness image provided in some embodiments of the present application;
[0027] Fig.14 A schematic diagram of a flow chart of another image defogging method provided in some embodiments of the present application;
[0028] Fig.15 A schematic diagram of the structure of an electronic device provided in some embodiments of the present application;
[0029] Fig.16 A schematic structural diagram of another electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION
[0030] The following will be combined with the drawings in this application to clearly and completely describe the technical solutions in this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0031] It should be noted that, in this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0032] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.
[0033] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "multiple" means two or more.
[0034] In order to make the terminal device have a higher screen-to-body ratio, make the screen more complete and have better visual effects, developers can embed the camera under the mobile phone screen to avoid the problem of setting the camera on the screen and reducing the screen-to-body ratio. However, due to the obstruction of the screen, the light sensitivity of the under-screen camera is much lower than that of a normal camera, so the images taken by the under-screen camera are relatively gray, which seriously affects the user's shooting experience.
[0035] At present, the images taken by the under-screen camera are usually processed by the defogging method based on the dark channel prior to restore the image clarity and thus ensure the user's shooting experience. However, the above defogging method based on the dark channel prior has relatively limited requirements on the image format and can only be applied in the RGB domain, which makes it impossible to defog images in other domains.
[0036] The image defogging method provided in the embodiment of the present application does not need to restrict the format of the original image. It only needs to convert the original image into an image in the brightness domain (i.e., the first brightness image), and perform defogging on the image in the brightness domain to obtain a defogging image in the brightness domain (i.e., the second brightness image). Based on the defogging image in the brightness domain, a defogging image corresponding to the original image is obtained. In this way, on the basis of achieving the defogging of the original image, the restriction of the defogging process on the image format is alleviated, and the defogging of images in other domains (e.g., RAW domain) is achieved, thereby improving the universal applicability of the defogging process. In addition, since the image in the brightness domain is a grayscale image, defogging the image in the brightness domain will not have a significant impact on the color of each pixel in the image, which can improve the effect of the defogging process.
[0037] For example, Figure 1, a schematic diagram of the structure of an under-screen camera system 10 provided in an embodiment of the present application is shown. The under-screen camera system 10 may include a denoising module 101, a high dynamic range imaging (HDR) module 102, a tone mapping module 103, a defogging module 104, and a glare module 105.
[0038] The denoising module 101 is mainly used to denoise the image by using a denoising technique to solve the noise problem of the image. Exemplarily, the denoising technique may include multi-frame denoising and / or artificial intelligence (AI) denoising technique.
[0039] The HDR module 102 is mainly used to solve the problem of high image exposure. The HDR module 102 is mainly used in images including bright and dark areas.
[0040] The tone mapping module 103 is mainly used to compress the HDR image so that the compressed image can meet the format requirements of the RAW domain.
[0041] The defogging module 104 is mainly used to perform defogging processing on the image to obtain a fog-free image.
[0042] Specifically, the defogging module 104 obtains the original image from the tone mapping module 103, converts the original image into a first brightness image, determines the brightness histogram of the first brightness image based on the pixel values of the pixels in the first brightness image, determines the bottom fog estimation value based on the brightness histogram of the first brightness image, performs defogging on the first brightness image based on the bottom fog estimation value to obtain a second brightness image, and obtains a defogging image corresponding to the original image based on the second brightness image. The brightness histogram is used to indicate the number of pixels corresponding to different pixel values in the first brightness image. The bottom fog estimation value is used to characterize the degree of influence of the grayness of the original image on the brightness.
[0043] The glare module 105 is mainly used to process the mirror-like light spots generated by the diffraction of light in the image.
[0044] As a way to achieve this, Figure 2 As shown, taking the image format as RAW format as an example, the following is a complete process of processing the image captured by the under-screen camera based on the above-mentioned under-screen camera system: the denoising module 101 can receive multiple RAW format images, denoise the above-mentioned multiple RAW format images based on the noise reduction technology, and send the denoised multiple RAW format images to the HDR module 102. The above-mentioned multiple RAW format images include N normally exposed images and M low-exposure images. N and M are both positive integers.
[0045] The HDR module 102 receives the multiple RAW format images after denoising from the denoising module 101 , fuses the multiple RAW format images after denoising to obtain an HDR image, and sends the HDR image to the tone mapping module 103 .
[0046] It should be noted that, since the multiple RAW format images after the above denoising process have different exposures, the HDR module 102 can fuse the multiple RAW format images after the denoising process to obtain an HDR image, which can avoid the problem that a single RAW format image after the denoising process cannot meet the requirements.
[0047] The tone mapping module 103 receives the HDR image from the HDR module 102 , compresses the HDR image to obtain a RAW image that meets the RAW format requirements, and sends the RAW format image to the defogging module 104 .
[0048] The defogging module 104 receives the RAW format image from the tone mapping module 103 , performs defogging on the RAW format image to obtain a RAW haze-free image, and sends the RAW haze-free image to the glare module 105 .
[0049] The glare module 105 receives the fog-free image in RAW format from the fog removal module 104 , and removes light spots (eg, mirror-like light spots) in the fog-free image in RAW format to obtain a fog-free image after light spots are removed.
[0050] It should be noted that Figure 1 This is just an exemplary framework diagram. Figure 1 The number of devices included in the Figure 1 In addition to the functional devices shown, the under-screen camera system may also include other devices, and this application does not impose any restrictions on this.
[0051] The application scenarios of the embodiments of the present application are not limited. The system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. It is known to those skilled in the art that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.
[0052] It should be noted that the following Figure 3 , Figure 7 ,as well as Fig.14 The methods shown can all be executed by a defogging module that supports the defogging function. Figure 3 , Figure 7,as well as Fig.14 The method shown may also be executed by an image processor, which is not limited in this application.
[0053] Figure 3 A flowchart of an image defogging method provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the method may include:
[0054] S301, acquiring an original image, and converting the original image into a first brightness image.
[0055] In one implementation, Figure 4 As shown, the original image can be arranged in a Bayer array. The Bayer array can include multiple 2×2 arrays. The 2×2 arrays are composed of 4 pixels. The 4 pixels can correspond to different channels respectively.
[0056] Exemplarily, the above channels may include a red (red, R) channel, a green red (green red, Gr) channel, a green blue (green blue, Gb) channel, and a blue (blue, B) channel. The above is only an exemplary description of the channel, and the above channels may also include other channels, and the present application does not impose any limitation on this.
[0057] In some embodiments, the implementation process of converting the original image into the first brightness image may be: based on a grayscale conversion formula, converting the original image into the first brightness image. The grayscale conversion formula is shown in the following formula 1:
[0058] Y0=0.299×R+0.587×0.5(Gr+Gb)+0.114×B Formula 1
[0059] Where Y0 is the grayscale value of a pixel in the first brightness image, R is the red channel value corresponding to the pixel located at the 1st column and the 1st row in the 2×2 array. Gr is the green channel value corresponding to the pixel located at the 2nd column and the 1st row in the 2×2 array. Gb is the green channel value corresponding to the pixel located at the 1st column and the 2nd row in the 2×2 array. B is the blue channel value corresponding to the pixel located at the 2nd column and the 2nd row in the 2×2 array.
[0060] It should be noted that, based on the above formula 1, the four pixels in the original image can be converted into one grayscale pixel (i.e., one pixel of the first brightness image), which is equivalent to downsampling the original image, so that the number of pixels in the first brightness image is equivalent to one quarter of the number of pixels in the original image.
[0061] It is understandable that if the original image is directly dehazed, problems such as color cast will occur in the dehazed image.
[0062] In one implementation, before converting the original image into the first brightness image, the original image may be subjected to at least one of black and white level correction, white balance, lens shading correction, demosaicing, and gamma conversion to make the original image have higher image quality.
[0063] In some examples, the original image may be an original image in RAW format. The above is only an exemplary description of the format of the original image, and the original image may also be an original image in other formats, and the present application does not impose any limitation on this.
[0064] S302: Determine a brightness histogram of the first brightness image based on pixel values of pixels in the first brightness image.
[0065] The brightness histogram is used to indicate the number of pixels corresponding to different pixel values in the first brightness image.
[0066] For example, Figure 5 An example of a histogram of a first brightness image is shown. Figure 5 As shown, the horizontal axis is used to represent the pixel value range corresponding to the pixel point in the first brightness image. The vertical axis is used to represent the number of pixel points corresponding to each pixel value. For example, the number of pixel points with a pixel value of 0 in the first brightness image is approximately 0. For another example, the number of pixel points with a pixel value of 36 in the first brightness image is approximately 50. For another example, the number of pixel points with a pixel value of 64 in the first brightness image is approximately 5000.
[0067] In some examples, the pixel values recorded in the embodiments of the present application may be normalized pixel values, that is, the ratio of the actual pixel value to the maximum pixel value. In this case, the normalized pixel value is within [0, 1]. Alternatively, the pixel values recorded in the embodiments of the present application may be actual pixel values.
[0068] S303: Determine a bottom fog estimation value based on a brightness histogram of the first brightness image.
[0069] The bottom fog estimation value is used to characterize the influence of the grayness of the original image on the brightness.
[0070] In some examples, the reasons for the graying of the original image may include at least one of the following: under-screen camera, environment, and camera parameter configuration error. The above is only an exemplary description of the reasons for the graying of the original image. The reasons for the graying of the original image may also include other reasons (for example, the camera is dirty), and this application does not impose any limitation on this.
[0071] In one embodiment, starting from the minimum pixel value of the brightness histogram, the number of pixel points corresponding to the pixel values in the brightness histogram are accumulated in order from small to large, and when the accumulated sum of the number of pixel points corresponding to the above pixel values is greater than or equal to a preset proportion of the total number of pixel points of the first brightness image for the first time, the accumulated maximum pixel value is obtained as the base fog estimation value.
[0072] As an example, in the brightness histogram, the number of pixels with a pixel value of 0 is 0, the number of pixels with a pixel value of 1 is 30, the number of pixels with a pixel value of 2 is 40, the number of pixels with a pixel value of 3 is 100, the total number of pixels of the first brightness image is 2000, and the preset ratio is 5%: in the brightness histogram, the number of pixels with pixel values in the range of 0 to 3 is 170, and the preset ratio of the total number of pixels of the first brightness image is 100, so the base fog estimation value can be determined to be 3.
[0073] It should be noted that the above preset ratio can be determined based on a preset threshold, for example, the preset ratio is determined to be 3%. The above is only an exemplary description of the preset range, and the present application does not impose any limitation on this.
[0074] Combination Figure 5 It can be seen that the sum of the number of pixels corresponding to the pixel values in the range of 0 to 36 is greater than or equal to the preset ratio of the total number of pixels of the first brightness image for the first time. In this case, the bottom fog estimation value can be determined as 36. In another embodiment, the ratio of the accumulated maximum pixel value to the maximum value in the pixel value range can also be determined as the bottom fog estimation value. In this embodiment, in the process of defogging the first brightness image based on the bottom fog estimation value, it is also necessary to convert the above-mentioned bottom fog estimation value into the accumulated maximum pixel value, and defog the first brightness image based on the accumulated maximum pixel value.
[0075] S304: Based on the base fog estimation value, perform defogging processing on the first brightness image to obtain a second brightness image.
[0076] In one embodiment, the implementation process of the above S304 may be: performing a difference operation between the pixel value of the pixel point in the first brightness image and the base fog estimation value to obtain the defogging pixel value of the pixel point in the first brightness image, and performing correction processing on the defogging pixel value of the pixel point in the first brightness image based on the correction coefficient to obtain the corrected pixel value of the pixel point in the first brightness image. The correction coefficient is equal to the ratio between the maximum value in the pixel value range and the maximum value of the defogging pixel value of the pixel point in the first brightness image. Based on the corrected pixel value of the pixel point in the first brightness image, the second brightness image is obtained.
[0077] It is understandable that in the process of defogging the first brightness image, it is necessary to perform difference calculation on the pixel values of the pixels in the first brightness image, which will lose some pixels with higher pixel values, and further affect the pixel value distribution of each pixel in the first brightness image to a certain extent, resulting in that each pixel in the first brightness image cannot be distributed relatively evenly in the brightness histogram. In view of this, the defogging pixel values of the pixels in the first brightness image are corrected so that each pixel in the first brightness image can be distributed relatively evenly in the brightness histogram.
[0078] For example, Figure 6 An example of a histogram of the second brightness image is shown. Figure 5 For the histogram shown in Figure 6 In the histogram shown, the number of pixel points corresponding to each pixel value is distributed relatively evenly, and it is rare that the number of pixel points corresponding to a pixel value is extremely large or extremely small.
[0079] S305 . Obtain a defogging image corresponding to the original image based on the second brightness image.
[0080] As an implementation method, the implementation process of the above S305 can be: based on the pixel value of each pixel in the first brightness image and the pixel value of each pixel in the second brightness image, the ratio value of each pixel in the second brightness image can be determined, and based on the ratio value of each pixel, the second brightness image is expanded to obtain an initial defogging image. The initial defogging image can be processed by GAMMA inverse transformation to obtain a defogging image.
[0081] For example, the above ratio value may satisfy the following formula 2:
[0082] Ratio=Y3 / Y0 Formula 2
[0083] Wherein, Ratio is the ratio value of a pixel, Y3 is the pixel value of the pixel in the second brightness image, and Y0 is the pixel value of the pixel in the first brightness image.
[0084] In some embodiments, the defogging image may be an image in RAW format. The above is only an exemplary description of the format of the defogging image, and the defogging image may also be a defogging image in other formats, and the present application does not impose any limitation on this.
[0085] It can be understood that, when both the defogging image and the original image are in RAW format, the original image in RAW format is converted into an original image in the brightness domain, and the image is defogged in the brightness domain to obtain a defogged image in the brightness domain. The defogged image in the brightness domain can be converted into a defogged image in RAW format, so that the defogging process of the RAW domain image can be achieved.
[0086] The image defogging method provided in the embodiment of the present application does not need to restrict the format of the original image. It only needs to convert the original image into an image in the brightness domain (i.e., the first brightness image), and perform defogging on the image in the brightness domain to obtain a defogging image in the brightness domain (i.e., the second brightness image). Based on the defogging image in the brightness domain, a defogging image corresponding to the original image is obtained. In this way, on the basis of achieving the defogging of the original image, the restriction of the defogging process on the image format is alleviated, and the defogging of images in other domains (e.g., RAW domain) is achieved, thereby improving the universal applicability of the defogging process. In addition, since the image in the brightness domain is a grayscale image, defogging the image in the brightness domain will not have a significant impact on the color of each pixel in the image, which can improve the effect of the defogging process.
[0087] Combination Figure 3 ,like Figure 7 As shown, the embodiment of the present application provides an image defogging method, which includes the following steps:
[0088] S701 . Perform brightness compensation processing on the second brightness image to obtain a third brightness image.
[0089] In one embodiment, a brightness compensation coefficient of a pixel in the second brightness image may be determined based on the pixel value of the pixel in the second brightness image. The brightness compensation coefficient and the pixel value of the pixel in the second brightness image satisfy a monotonically decreasing relationship. Based on the brightness compensation coefficient of the pixel in the second brightness image, brightness compensation may be performed on the pixel value of the pixel in the second brightness image to obtain a compensated pixel value of the pixel in the second brightness image, and based on the compensated pixel value of the pixel in the second brightness image, a third brightness image may be obtained.
[0090] In another embodiment, the brightness compensation coefficient of the pixel in the second brightness image can be determined based on the pixel value of the pixel in the second brightness image. The brightness compensation coefficient and the pixel value satisfy a monotonically decreasing relationship. The pixel value of the pixel in the partial image can be brightness compensated based on the brightness compensation coefficient of the pixel in the second brightness image to obtain the compensated pixel value of the pixel in the partial image, and the third brightness image can be obtained based on the compensated pixel value of the pixel in the partial image and the pixel value of the pixel in the image other than the partial image in the second brightness image.
[0091] As an example, the above compensated pixel value may satisfy the following formula 3:
[0092] Y2=Pow(Y1,C) Formula 3
[0093] Wherein, Y2 is the compensated pixel value of the pixel point in the second brightness image, Y1 is the pixel value of the pixel point in the second brightness image, and C is the brightness compensation coefficient.
[0094] As an implementation method, the brightness compensation coefficient can be determined based on the bottom fog estimation value and the parameter adjustment factor, wherein the parameter adjustment factor is used to achieve different brightness compensation requirements.
[0095] As an example, the brightness compensation coefficient may satisfy the following formula 4:
[0096] C = Pow (1.0-T, F) Formula 4
[0097] Where, T is the estimated value of bottom fog and F is the parameter adjustment factor.
[0098] It is understandable that the parameter adjustment factor is usually determined by the ISP platform based on the brightness requirement of the second brightness image to be compensated. For the same image, if the parameter adjustment factor is larger, the brightness compensation is performed based on the parameter adjustment factor, and the pixel value of the brightness compensated image is higher.
[0099] For example, Figures 8 to 11 As shown, Figure 8 The image shown corresponds to a parameter adjustment factor greater than Fig. 9 The image shown corresponds to the parameter adjustment factor, and Figure 8 The image shown is brighter than Fig. 9 The image brightness is shown. Fig. 9 The image shown corresponds to a parameter adjustment factor greater than Fig.10 The image shown corresponds to the parameter adjustment factor, and Fig. 9 The image shown is brighter than Fig.10 The image brightness is shown. Fig.10 The image shown corresponds to a parameter adjustment factor greater than Fig.11 The image shown corresponds to the parameter adjustment factor, and Fig.10 The image shown is brighter than Fig.11 The image brightness is shown.
[0100] In an example, the parameter adjustment factor may be greater than 0. The above is only an exemplary description of the parameter adjustment factor, and the parameter adjustment factor may also be greater than 1, and the present application does not impose any limitation on this.
[0101] It can be understood that due to the dehazing operation provided in the embodiment of the present application, it is necessary to perform a difference operation on the pixel value of the pixel point in the first brightness image and the base fog estimation value, so the pixel value of the second brightness image obtained in this way will be reduced, especially some pixels with lower pixel values in the first brightness image. Therefore, brightness compensation of the second brightness image can increase the pixel value of the second brightness image to avoid the problem of unclear image due to low pixel values.
[0102] S702: Perform contrast adjustment on the third brightness image to obtain a fourth brightness image.
[0103] In one embodiment, the third brightness image is divided into a plurality of brightness sub-images (for example, M×N sub-images, where M and N are positive integers). Contrast adjustment is performed on the plurality of brightness sub-images respectively. A plurality of adjusted brightness sub-images are obtained. The fourth brightness image is formed by the plurality of adjusted brightness sub-images.
[0104] As an example, contrast adjustment may be performed on multiple brightness sub-images based on a specified histogram equalization (contrast limited adaptive histogram equalization, CLAHE) algorithm.
[0105] It is understandable that in order to avoid the problem of unclear image due to low pixel value, brightness compensation is performed on the second brightness image to obtain the third brightness image. However, the brightness compensation operation will cause a certain degree of fog to appear in the third brightness image again. Based on this, the contrast of the third brightness image (or a part of the third brightness image) can be adjusted to improve the clarity of the image.
[0106] For example, Figure 12 to Figure 13 As shown, Fig.12 is an example of the second brightness image, Fig.13 is an example of the third brightness image. Fig.13 Compared to Fig.12 For example, it has higher contrast and clarity.
[0107] S703: Convert the fourth brightness image into a defogging image corresponding to the original image.
[0108] It should be noted that the implementation process of S703 can be understood by referring to the above S305, which will not be repeated here.
[0109] In an embodiment of the present application, the second brightness image can be subjected to brightness compensation processing to obtain a third brightness image, and the third brightness image can be subjected to contrast adjustment to obtain a fourth brightness image. Since the brightness compensation processing can increase the brightness of the second brightness image to avoid the problem of unclear image due to low pixel values, and the contrast adjustment processing can increase the contrast of the third brightness image to ensure the clarity of the image, the fourth brightness image obtained after the brightness compensation processing and the contrast adjustment processing can be clearer. In this way, the fourth brightness image obtained after the brightness compensation processing and the contrast adjustment processing is converted into a defogging image, which can further improve the clarity of the defogging image.
[0110] Combination Figure 3 ,like Fig.14 As shown, the embodiment of the present application provides an image defogging method, which includes the following steps:
[0111] S1401: Perform brightness enhancement processing on the defogging image to obtain a defogging image after the brightness enhancement processing.
[0112] In one embodiment, an initial brightening coefficient is obtained, and based on the initial brightening coefficient, the defogging image is brightened to obtain a first image. When the pixel value of the first image is less than the pixel value of the original image, the initial brightening coefficient is adjusted, and the defogging image is brightened based on the new initial brightening coefficient to obtain an updated first image. When the pixel value of the first image is greater than or equal to the pixel value of the original image, the first image is determined to be a defogging image after brightness enhancement processing.
[0113] It can be understood that the initial brightening coefficient is continuously adjusted based on the original image (i.e., the original image) so that the pixel value of the dehazed image adjusted based on the initial brightening coefficient (i.e., the dehazed image after brightening processing) can be closer to the pixel value of the original image, thereby ensuring the clarity of the dehazed image after brightening processing.
[0114] S1402: Fusing the defogging image after the brightness enhancement process and the defogging image before the brightness enhancement process to obtain a fused defogging image.
[0115] In one implementation, the implementation process of S1402 may be: the defogged image after brightness enhancement processing and the defogged image before brightness enhancement processing may be fused through a Laplacian pyramid to obtain a fused defogged image.
[0116] It is understandable that performing brightness enhancement processing on the defogging image will make all pixels in the defogging image brightened indiscriminately, which is likely to cause overexposure in some areas of the defogging image after brightness enhancement processing. In view of this, the defogging image after brightness enhancement processing and the defogging image before brightness enhancement processing are fused so that the pixel values of each area in the fused defogging image can be within the normal range, thereby solving the problem of overexposure in some areas of the defogging image after brightness enhancement processing.
[0117] In some embodiments, the fused dehazed image may be subjected to at least one inverse process of black and white level correction and white balance to obtain a target image, and the target image is sent to the ISP platform.
[0118] In an embodiment of the present application, the dehazed image is brightened to obtain a brightened dehazed image, and the dehazed image after the brightness enhancement process and the dehazed image before the brightness enhancement process are fused to obtain a fused dehazed image, which can further adaptively improve the pixel value of the dehazed image, thereby further improving the clarity of the dehazed image.
[0119] It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to each function. It should be easily appreciated by those skilled in the art that, in conjunction with the algorithm steps of each example described in the embodiments of the present application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.
[0120] The embodiment of the present application can divide the functional modules of the electronic device according to the above method embodiment. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one functional module. The above integrated module can be implemented in the form of hardware or software. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.
[0121] In one example, the electronic device may be an electronic device with an under-screen camera. The above is only an exemplary description of an electronic device, and the electronic device may also be other electronic devices, and this application does not impose any limitation on this.
[0122] Fig.15 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, and the electronic device can execute the image defogging method provided in the above method embodiment. Fig.15 As shown, the electronic device 150 includes: a communication unit 1501 and a processing unit 1502 .
[0123] The communication unit 1501 is used to obtain an original image and convert the original image into a first brightness image;
[0124] The processing unit 1502 is used to determine a brightness histogram of the first brightness image based on the pixel values of the pixels in the first brightness image, where the brightness histogram is used to indicate the number of pixels corresponding to different pixel values in the first brightness image;
[0125] The processing unit 1502 is further configured to determine a bottom fog estimation value based on a brightness histogram of the first brightness image, where the bottom fog estimation value is used to characterize the degree of influence of the gray haze on the brightness of the original image;
[0126] The processing unit 1502 is further configured to perform defogging processing on the first brightness image based on the bottom fog estimation value to obtain a second brightness image;
[0127] The processing unit 1502 is further configured to obtain a defogging image corresponding to the original image based on the second brightness image.
[0128] In some embodiments, the processing unit 1502 is also used to accumulate the number of pixel points corresponding to the pixel values in the brightness histogram, starting from the minimum pixel value of the brightness histogram, in order from small to large pixel values; when the accumulated sum of the number of pixel points corresponding to the pixel value is greater than or equal to a preset proportion of the total number of pixel points of the first brightness image for the first time, the communication unit 1501 is also used to obtain the accumulated maximum pixel value as the base fog estimation value.
[0129] In some embodiments, the processing unit 1502 is further used to perform a difference operation between the pixel value of the pixel point in the first brightness image and the base fog estimation value to obtain the defogged pixel value of the pixel point in the first brightness image; the processing unit 1502 is further used to perform correction processing on the defogged pixel value of the pixel point in the first brightness image based on the correction coefficient to obtain the corrected pixel value of the pixel point in the first brightness image; the correction coefficient is equal to the ratio between the maximum value in the pixel value range and the maximum value of the defogged pixel value of the pixel point in the first brightness image; the processing unit 1502 is further used to obtain the second brightness image based on the corrected pixel value of the pixel point in the first brightness image.
[0130] In some embodiments, the processing unit 1502 is further used to perform brightness compensation processing on the second brightness image to obtain a third brightness image; the processing unit 1502 is further used to perform contrast adjustment on the third brightness image to obtain a fourth brightness image; the processing unit 1502 is further used to convert the fourth brightness image into a dehazed image.
[0131] In some embodiments, the processing unit 1502 is further used to determine a brightness compensation coefficient of a pixel in the second brightness image based on the pixel value of the pixel in the second brightness image, and the brightness compensation coefficient and the pixel value of the pixel in the second brightness image satisfy a monotonically decreasing relationship; the processing unit 1502 is further used to perform brightness compensation on the pixel value of the pixel in the second brightness image based on the brightness compensation coefficient of the pixel in the second brightness image to obtain the compensated pixel value of the pixel in the second brightness image; the processing unit 1502 is further used to obtain a third brightness image based on the compensated pixel value of the pixel in the second brightness image.
[0132] In some embodiments, the processing unit 1502 is further used to divide the third brightness image into multiple brightness sub-images; the processing unit 1502 is further used to adjust the contrast of the multiple brightness sub-images respectively; obtain the adjusted multiple brightness sub-images; the processing unit 1502 is further used to construct a fourth brightness image with the adjusted multiple brightness sub-images.
[0133] In some embodiments, the processing unit 1502 is further used to perform brightness enhancement processing on the dehazed image to obtain a dehazed image after brightness enhancement processing; the processing unit 1502 is further used to fuse the dehazed image after brightness enhancement processing and the dehazed image before brightness enhancement processing to obtain a fused dehazed image.
[0134] In some embodiments, both the original image and the dehazed image are images in RAW format.
[0135] When implementing it specifically, Figure 1 The equipment in Fig.16 The structure shown, or including Fig.16 Parts shown. Fig.16 FIG. 1 is a schematic diagram of the composition of an electronic device 160 provided in an embodiment of the present application. Fig.16 As shown, the electronic device 160 may include a processor 1601, a communication line 1602, a communication interface 1603, and a memory 1604. The processor 1601, the memory 1604, and the communication interface 1603 may be connected via the communication line 1602.
[0136] The processor 1601 is a CPU, a general processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 1601 may also be other devices with processing functions, such as circuits, devices, or software modules, without limitation.
[0137] The communication line 1602 is used to transmit information between the components included in the electronic device 160 .
[0138] The communication interface 1603 is used to communicate with other devices or other communication networks. The other communication networks may be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 1603 may be a module, a circuit, a communication interface or any device capable of realizing communication.
[0139] The memory 1604 is used to store instructions, where the instructions may be computer programs.
[0140] Among them, the memory 1604 can be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage devices, etc., without limitation.
[0141] It should be noted that the memory 1604 can exist independently of the processor 1601, or can be integrated with the processor 1601. The memory 1604 can be used to store instructions or program codes or some data, etc. The memory 1604 can be located in the electronic device 160, or can be located outside the electronic device 160, without limitation. The processor 1601 is used to execute the instructions stored in the memory 1604 to implement the image defogging method provided in the following embodiments of the present application.
[0142] In one example, the processor 1601 may include one or more CPUs, for example, CPU0 and CPU1.
[0143] As an implementation, the electronic device 160 includes multiple processors.
[0144] As an implementation, the electronic device 160 further includes an output device and an input device. For example, the input device is a keyboard, a mouse, a microphone, a joystick, and the like, and the output device is a display screen, a speaker, and the like.
[0145] It should be noted that the electronic device 160 may be a desktop computer, a portable computer, a network server, a mobile phone, a tablet computer, a wireless terminal, an embedded device, a chip system, or a Fig.16 In addition, Fig.16 The composition shown in the Figure 1 as well as Fig.16 The limitations of each device in Fig.16 In addition to the parts shown, Fig.16 The various devices shown may include more or fewer components than shown, or combine certain components, or arrange the components differently.
[0146] In the embodiments of the present application, the chip system may be composed of a chip, or may include a chip and other discrete devices. Some embodiments of the present application provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium), which stores computer program instructions. When the computer program instructions are executed on a computer, the computer executes the image defogging method of any of the above embodiments.
[0147] Exemplarily, the above-mentioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks or magnetic tapes, etc.), optical disks (e.g., compact disks (CD), digital versatile disks (DVD), etc.), smart cards and flash memory devices (e.g., erasable programmable read-only memory (EPROM), cards, sticks or key drives, etc.). The various computer-readable storage media described in this application may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.
[0148] An embodiment of the present application provides a computer program product comprising instructions. When the computer program product is run on a computer, the computer is enabled to execute the image defogging method of any one of the above embodiments.
[0149] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An image defogging method, characterized in that: The method comprises: Acquire an original image, and convert the original image into a first brightness image; Determine a brightness histogram of the first brightness image based on pixel values of pixels in the first brightness image, wherein the brightness histogram is used to indicate the number of pixels corresponding to different pixel values in the first brightness image; Determine a bottom fog estimation value based on a brightness histogram of the first brightness image, where the bottom fog estimation value is used to characterize the degree of influence of the grayness of the original image on the brightness; Based on the bottom fog estimation value, performing defogging processing on the first brightness image to obtain a second brightness image; Based on the second brightness image, a defogging image corresponding to the original image is obtained.
2. The method according to claim 1, characterized in that The step of determining a bottom fog estimation value based on a brightness histogram of the first brightness image includes: Starting from the minimum pixel value of the brightness histogram, the number of pixel points corresponding to the pixel values in the brightness histogram is accumulated in order of pixel values from small to large; When the accumulated sum of the number of pixel points corresponding to the pixel value is greater than or equal to a preset ratio of the total number of pixel points of the first brightness image for the first time, the accumulated maximum pixel value is obtained as the base fog estimation value.
3. The method according to claim 1, characterized in that The step of performing defogging on the first brightness image based on the bottom fog estimation value to obtain a second brightness image includes: Performing a difference operation on a pixel value of a pixel point in the first brightness image and the base fog estimation value to obtain a defogging pixel value of the pixel point in the first brightness image; Based on the correction coefficient, the defogging pixel value of the pixel point in the first brightness image is corrected to obtain the corrected pixel value of the pixel point in the first brightness image; the correction coefficient is equal to the ratio between the maximum value in the pixel value range and the maximum value of the defogging pixel value of the pixel point in the first brightness image; The second brightness image is obtained based on the corrected pixel values of the pixels in the first brightness image.
4. The method according to claim 3, characterized in that: The obtaining, based on the second brightness image, a defogging image corresponding to the original image includes: performing brightness compensation processing on the second brightness image to obtain a third brightness image; Performing contrast adjustment on the third brightness image to obtain a fourth brightness image; The fourth brightness image is converted into the dehazed image.
5. The method according to claim 4, characterized in that The performing brightness compensation processing on the second brightness image to obtain a third brightness image includes: Determining a brightness compensation coefficient of the pixel in the second brightness image based on the pixel value of the pixel in the second brightness image, wherein the brightness compensation coefficient and the pixel value of the pixel in the second brightness image satisfy a monotonically decreasing relationship; Based on the brightness compensation coefficient of the pixel in the second brightness image, performing brightness compensation on the pixel value of the pixel in the second brightness image to obtain a compensated pixel value of the pixel in the second brightness image; The third brightness image is obtained based on the compensated pixel values of the pixels in the second brightness image.
6. The method according to claim 4, characterized in that The step of performing contrast adjustment on the third brightness image to obtain a fourth brightness image includes: dividing the third brightness image into a plurality of brightness sub-images; Performing contrast adjustment on the multiple brightness sub-images respectively to obtain multiple brightness sub-images after adjustment; The fourth brightness image is formed by the adjusted multiple brightness sub-images.
7. The method according to claim 1, characterized in that The method further comprises: Performing brightness enhancement processing on the defogging image to obtain a defogging image after brightness enhancement processing; The defogging image after the brightness enhancement processing and the defogging image before the brightness enhancement processing are fused to obtain a fused defogging image.
8. The method according to claim 1, characterized in that The original image and the dehazed image are both images in RAW format.
9. An electronic device, characterized in that: include: a processor and a memory for storing instructions executable by the processor; The processor is configured to execute the instructions so that the electronic device performs the method as described in any one of claims 1-8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.