An image processing method, apparatus, device, and storage medium

Abstract

This application relates to an image processing method, apparatus, device, and storage medium. The method includes: acquiring an original image and determining the three-channel values ​​of each pixel in the original image; converting the original image into a grayscale image and determining the variance of the grayscale values ​​in the grayscale image; adjusting the atmospheric light value and the transmittance of each pixel in the image based on the variance of the grayscale values; inputting the three-channel values ​​of each pixel in the original image, the adjusted atmospheric light value, and the adjusted target transmittance into a preset atmospheric light scattering model to generate a restored image corresponding to the original image. This application can restore the original degraded image without causing image distortion.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device and storage medium. Background Technology

[0002] During image acquisition, the acquired images are affected by the surrounding environment. The presence of fog, water vapor, dust, or other factors that may affect image acquisition results in a decline in image quality, leading to degraded images. For example, when acquiring images through image acquisition equipment in foggy conditions, the light reflected from objects is scattered, refracted, and reflected by numerous fine particles in the air before reaching the acquisition device. This degrades the image quality, reducing contrast and sharpness, and resulting in the loss of significant details compared to the real scene. This greatly diminishes the application value of the acquired images and significantly impacts industrial production and daily life. For instance, degraded images obtained by urban traffic monitoring systems in foggy weather conditions make it difficult to determine vehicle information and monitor traffic conditions. Therefore, defogging of foggy images is of great importance.

[0003] When using existing image dehazing methods to process hazy images, image distortion occurs after dehazing. Therefore, it is necessary to propose an effective image processing method. Summary of the Invention

[0004] The technical problem to be solved by this application is to provide an image processing method, apparatus, device and storage medium that can restore the original degraded image without causing image distortion.

[0005] To address the aforementioned technical problems, this application provides an image processing method, the method comprising:

[0006] Acquire the original image and determine the three-channel values ​​of each pixel in the original image;

[0007] Based on the values ​​of the three channels of each pixel in the original image, the first atmospheric light value of the original image is determined;

[0008] The original image is converted into a grayscale image, and the variance of the grayscale values ​​in the grayscale image is determined.

[0009] The first atmospheric light value is adjusted based on the variance of the gray values ​​in the grayscale image to obtain the second atmospheric light value;

[0010] Based on the values ​​of the three channels of each pixel in the original image and the second atmospheric light value, the first transmittance of each pixel in the original image is obtained;

[0011] The first transmittance of each pixel in the original image is adjusted based on the variance of the gray values ​​in the grayscale image to obtain the target transmittance of each pixel.

[0012] The values ​​of the three channels of each pixel in the original image, the second atmospheric light value, and the target transmittance are input into a preset atmospheric light scattering model to generate a restored image corresponding to the original image.

[0013] On the other hand, this application provides an image processing apparatus, the apparatus comprising:

[0014] The original image acquisition module is used to acquire the original image and determine the three-channel values ​​of each pixel in the original image.

[0015] The first atmospheric light value determination module is used to determine the first atmospheric light value of the original image based on the three-channel values ​​of each pixel in the original image.

[0016] The variance determination module is used to convert the original image into a grayscale image and determine the variance of the grayscale values ​​in the grayscale image;

[0017] An atmospheric light value adjustment module is used to adjust the first atmospheric light value based on the variance of the gray values ​​in the grayscale image to obtain a second atmospheric light value;

[0018] The first transmittance determination module is used to obtain the first transmittance of each pixel in the original image based on the three-channel values ​​of each pixel in the original image and the second atmospheric light value.

[0019] The transmittance adjustment module is used to adjust the first transmittance of each pixel in the original image based on the variance of the gray values ​​in the grayscale image, so as to obtain the target transmittance of each pixel.

[0020] The restored image generation module is used to input the values ​​of the three channels of each pixel in the original image, the second atmospheric light value, and the target transmittance into a preset atmospheric light scattering model to generate a restored image corresponding to the original image.

[0021] On the other hand, this application provides an apparatus comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described above.

[0022] On the other hand, this application provides a computer storage medium storing at least one instruction or at least one program, wherein the at least one instruction or the at least one program is loaded by a processor and executed as described above in the image processing method.

[0023] Implementing the embodiments of this application has the following beneficial effects:

[0024] This application converts the original image into a grayscale image and determines the variance of the grayscale values ​​in the grayscale image. Based on the variance of the grayscale values ​​in the grayscale image, it adjusts the first atmospheric light value and the first transmittance of each pixel in the original image. The three-channel values ​​of each pixel in the original image, the adjusted atmospheric light value, and the adjusted transmittance are input into a preset atmospheric light scattering model to generate a restored image corresponding to the original image. This application adjusts the atmospheric light value and transmittance of the image based on the variance of the grayscale values ​​in the grayscale image, improving the accuracy of the estimation of atmospheric light value and transmittance. This results in a restored image calculated based on the adjusted atmospheric light value and transmittance that better matches the real scene, restoring the detailed information of the original scene. This achieves the technical effect of restoring the original degraded image without causing image distortion. Attached Figure Description

[0025] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the implementation environment provided in the embodiments of this application;

[0027] Figure 2 This is a flowchart of an image processing method provided in an embodiment of this application;

[0028] Figure 3 This is a flowchart of the method for calculating the variance of grayscale values ​​in a grayscale image provided in the embodiments of this application;

[0029] Figure 4 This is a flowchart of an atmospheric light value adjustment method provided in an embodiment of this application;

[0030] Figure 5 This is a flowchart of a method for determining the adjustment coefficient of target atmospheric light value provided in an embodiment of this application;

[0031] Figure 6 This is a schematic diagram illustrating the relationship between the atmospheric light value adjustment coefficient and variance provided in the embodiments of this application;

[0032] Figure 7 This is a flowchart of a pixel transmittance adjustment method provided in an embodiment of this application;

[0033] Figure 8This is a flowchart of a method for determining the target transmittance adjustment coefficient provided in an embodiment of this application;

[0034] Figure 9 This is a schematic diagram illustrating the relationship between the transmittance adjustment coefficient and variance provided in the embodiments of this application;

[0035] Figure 10 This is a flowchart of a transmittance refinement method provided in an embodiment of this application;

[0036] Figure 11 This is a schematic diagram of the image processing effect provided in the embodiments of this application;

[0037] Figure 12 This is a schematic diagram of an image processing device provided in an embodiment of this application;

[0038] Figure 13 This is a schematic diagram of a device structure provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0041] The image processing method provided in this application relates to computer vision technology in artificial intelligence. Computer vision is a science that studies how to enable machines to "see." More specifically, it refers to machine vision that uses cameras and computers to replace human eyes for target recognition, tracking, and measurement, and then performs further image processing to make the computer-processed images more suitable for human observation or transmission to instruments for detection. Computer vision technology attempts to establish an artificial intelligence system capable of acquiring information from images or multidimensional data, enabling computers to simulate human visual processes and possess the ability to perceive the environment and human visual functions. It is a synthesis of image processing, artificial intelligence, and pattern recognition technologies.

[0042] Please see Figure 1 The illustration shows an implementation environment provided in the embodiments of this application. The implementation environment may include at least one first terminal 110 and a second terminal 120, which can communicate with each other via a network.

[0043] Specifically, the first terminal 110 sends the original image to be restored to the second terminal 120. In this embodiment, the original image to be restored can be an image acquired in real time by the image acquisition device of the first terminal 110, or it can be an image stored in the first terminal 110. The image here can include video images or individual image frames. The second terminal 120 receives the original image to be restored sent by the first terminal 110 and processes the original image to be restored based on the image processing model to obtain the restored image.

[0044] The first terminal 110 can communicate with the second terminal 120 based on a browser / server (B / S) or client / server (C / S) model. The first terminal 110 may include physical devices such as smartphones, tablets, laptops, digital assistants, smart wearable devices, in-vehicle terminals, and servers, and may also include software running on the physical device, such as applications. The operating system running on the first terminal 110 in this embodiment may include, but is not limited to, Android, iOS, Linux, and Windows.

[0045] The second terminal 120 and the first terminal 110 can establish a communication connection via wired or wireless means. The second terminal 120 may include an independently operating server, a distributed server, or a server cluster consisting of multiple servers, wherein the server may be a cloud server.

[0046] To address the image distortion problem that may occur during dehazing of foggy images in existing technologies, this application provides an image processing method. The execution entity of this method can be either the first terminal or the second terminal described above; that is, the method can run on a client terminal or a server. For details, please refer to [link to relevant documentation]. Figure 2 The method may include:

[0047] S210. Obtain the original image and determine the three-channel values ​​of each pixel in the original image.

[0048] The original image in this embodiment can be a degraded image obtained due to environmental factors during image acquisition. A degraded image refers to an image that has lost some scene details compared to the real scene, such as a foggy image, a rainy image, or an image containing dust. Specifically, the original image in this embodiment can be a foggy image acquired in a foggy scene. The specific foggy image can be determined from a foggy video image or directly from an independent foggy image frame. Specifically, when the original image is determined from a foggy video image, the foggy video image can be divided into image frames, and each of the divided video image frames can be used as the original image for image processing. When the original image is determined from an independent foggy image frame, the independent foggy image frame, specifically a foggy picture, can be directly used as the original image for image processing. Therefore, the image processing method provided in this embodiment is applicable to both video dehazing and image dehazing scenarios.

[0049] Furthermore, the original image in the embodiments of this application can also be an old image, such as an old photograph, so the image processing method of the embodiments of this application can also be applied to the scenario of old film restoration.

[0050] The three-channel value of each pixel refers to the values ​​of the three RGB color channels of that pixel, namely the R channel value, the G channel value, and the B channel value.

[0051] S220. Based on the values ​​of the three channels of each pixel in the original image, determine the first atmospheric light value of the original image.

[0052] The determination of the first atmospheric light value of the original image can be based on the three-channel values ​​of each pixel in the original image to generate a dark channel image corresponding to the original image; and the first atmospheric light value of the original image can be calculated based on the single-channel values ​​of each pixel in the dark channel image.

[0053] Specifically, based on the dark channel prior theory, the minimum value of the RGB three channels of each pixel in the original image is taken to obtain the gray value of the pixel in the corresponding dark channel image, thereby generating the dark channel image; then, the top 0.1% of pixels with the largest gray value in the dark channel image are found, and the corresponding pixels in the original image are determined respectively. The mean values ​​of these pixels in the R, G, and B channels are calculated respectively, and the mean values ​​of the three channels are used as the first atmospheric light value A0 of the corresponding RGB channel.

[0054] The atmospheric light value estimated by this method is a rough estimate and is inaccurate, especially when there is a large bright area in the sky. The atmospheric light value estimate is usually too large. Therefore, in the embodiments of this application, the first atmospheric light value needs to be adjusted in the future.

[0055] S230. Convert the original image into a grayscale image and determine the variance of the grayscale values ​​in the grayscale image.

[0056] For the specific method of calculating the variance of grayscale values ​​in a grayscale image, the original image must first be converted into a grayscale image, and then the corresponding variance calculation is performed. For details, please refer to [link to relevant documentation]. Figure 3 The method may include:

[0057] S310. For each pixel in the original image, assign corresponding weights to the values ​​of the three channels of the pixel, calculate the weighted sum of the values ​​of the three channels of the pixel, and determine the weighted sum of the values ​​of the three channels of the pixel as the gray value corresponding to the pixel.

[0058] S320. Generate the grayscale image based on the grayscale value corresponding to each pixel.

[0059] S330. Calculate the average gray value of each pixel in the grayscale image.

[0060] S340. Calculate the variance of the gray values ​​in the gray image based on the gray values ​​of each pixel and the average value of the gray values.

[0061] Specifically, the formula for converting a color RGB three-channel image to a grayscale image is as follows:

[0062] Gray(x)=R(x)·w1+G(x)·w2+B(x)·w3 (1)

[0063] Where w1+w2+w3=1, and R(x), G(x) and B(x) are the values ​​of the three channels of the corresponding pixel.

[0064] Specifically, in the embodiments of this application, as an example, w1 = 0.299, w2 = 0.587, and w3 = 0.114 can be taken.

[0065] The average gray value of each pixel in a grayscale image is obtained by averaging the gray values ​​of all pixels. After obtaining the grayscale value and average value of each pixel in the grayscale image, the variance S of the grayscale values ​​can be calculated using the following formula. 2 That is, to find the variance S of Gray(x). 2 :

[0066]

[0067] Where w is the width of the grayscale image and h is the height of the grayscale image, that is, the grayscale image is regarded as a pixel matrix, w is the number of rows of the pixel matrix and h is the number of columns of the pixel matrix; Let Gray(x) be the average grayscale value of each pixel in the grayscale image. i ) represents the grayscale value of the i-th pixel in the grayscale image.

[0068] In this embodiment, calculating the variance of grayscale values ​​in the grayscale image can be used to subsequently adjust atmospheric light values ​​and transmittance. When the variance of the grayscale image is small, it indicates that the image content changes relatively little. This could be in a foggy scene or under a fixed camera, where objects are far from the camera and the depth of field changes little, thus requiring a greater degree of defogging. Conversely, a larger variance indicates greater changes in the image content. In hazy weather, this is more likely due to low haze concentration or a large change in depth of field under a moving camera, thus requiring a smaller degree of defogging.

[0069] S240. Adjust the first atmospheric light value based on the variance of the gray values ​​in the grayscale image to obtain the second atmospheric light value.

[0070] For details on adjusting the first atmospheric light value, please refer to [link / reference needed]. Figure 4 The method may include:

[0071] S410. Based on the relationship between the atmospheric light value adjustment coefficient and the variance, determine the target atmospheric light value adjustment coefficient corresponding to the variance of the gray values ​​in the grayscale image.

[0072] In this embodiment, the relationship between the atmospheric light value adjustment coefficient and the variance can be determined based on a preset mapping relationship, so that the target atmospheric light value adjustment coefficient can be obtained according to the variance of the calculated gray value.

[0073] S420. The product of the target atmospheric light value adjustment coefficient and the first atmospheric light value is determined as the second atmospheric light value.

[0074] Please see Figure 5 This application illustrates a method for determining a target atmospheric light value adjustment coefficient, which may include:

[0075] S510. Determine whether the variance of the grayscale values ​​in the grayscale image is less than a first threshold.

[0076] S520. When the variance of grayscale values ​​in the grayscale image is less than a first threshold, the first adjustment coefficient is determined as the target atmospheric light value adjustment coefficient.

[0077] S530. When the variance of grayscale values ​​in the grayscale image is not less than a first threshold, determine whether the variance of grayscale values ​​in the grayscale image is greater than a second threshold.

[0078] S540. When the variance of the grayscale values ​​in the grayscale image is greater than the second threshold, the second adjustment coefficient is determined as the target atmospheric light value adjustment coefficient.

[0079] S550. When the variance of the grayscale values ​​in the grayscale image is not greater than the second threshold, a third adjustment coefficient is obtained based on the first preset function, and the third adjustment coefficient is determined as the target atmospheric light value adjustment coefficient.

[0080] That is, when the variance of the grayscale values ​​in the grayscale image is greater than or equal to the first threshold and less than or equal to the second threshold, a third adjustment coefficient is obtained based on the first preset function, and the third adjustment coefficient is determined as the target atmospheric light value adjustment coefficient.

[0081] Wherein, the first threshold is less than the second threshold.

[0082] As an example, the relationship between the atmospheric light value adjustment factor and the variance can be achieved using the following function:

[0083]

[0084] The corresponding relationship diagram can be found in [reference]. Figure 6 Where α is the atmospheric light value adjustment coefficient, the minimum boundary of the variance is set as k1, and the maximum boundary is set as k2. When the variance is less than the minimum boundary k1, the adjustment coefficient is always the set minimum value α. min The first adjustment coefficient is used when the variance is greater than the maximum boundary k2. When the variance is greater than the maximum boundary k2, the adjustment coefficient is the maximum value of 1, which is the second adjustment coefficient. In other words, the final atmospheric light value is the estimated atmospheric light value A0. When the variance is greater than or equal to the minimum boundary k1 and less than or equal to the maximum boundary k2, the third adjustment coefficient can be determined by the second part of the formula (3).

[0085]

[0086] That is, the first preset function in step S550 above can be formula (4).

[0087] In the specific implementation process, k1 can be set to 20 and k2 to 60. The values ​​of k1 and k2 can be dynamically changed and adjusted according to the characteristics of the video image.

[0088] α min To set a minimum adjustment coefficient, it is necessary to determine that the adjustment coefficient cannot be less than or equal to 0. If the adjustment coefficient is 0, the image will be judged as having no lighting, resulting in severe distortion. Conversely, the adjustment coefficient cannot be too small, as this will cause the image to be too dark and inconsistent with the real scene. In this embodiment, a minimum adjustment coefficient α can be set. min The value is 0.6. This value is for reference only and can be dynamically adjusted according to the characteristics of the video image.

[0089] In step S420, the product of the target atmospheric light value adjustment coefficient and the first atmospheric light value is determined as the second atmospheric light value. Specifically, the estimated first atmospheric light value A0 is multiplied by the atmospheric light value adjustment coefficient α to obtain the final atmospheric light value A, and the corresponding formula is as follows:

[0090] A=α×A0 (5)

[0091] S250. Based on the values ​​of the three channels of each pixel in the original image and the second atmospheric light value, the first transmittance of each pixel in the original image is obtained.

[0092] Based on the dark channel prior theory, the dark channel value of a hazy image approaches 0. After deduction and calculation according to this theory, the estimated formula for transmittance can be obtained as follows:

[0093]

[0094] Among them, I min (x) is the minimum value among the three channels of the pixel, and A' is the atmospheric light value corresponding to the minimum channel. Based on the above formula (6), the first transmittance of each pixel in the image can be obtained.

[0095] S260. Based on the variance of the gray values ​​in the grayscale image, the first transmittance of each pixel in the original image is adjusted to obtain the target transmittance of each pixel.

[0096] Please see Figure 7 This illustrates a method for adjusting the transmittance of pixels according to an embodiment of this application, the method including:

[0097] S710. Based on the relationship between the transmittance adjustment coefficient and the variance, determine the target transmittance adjustment coefficient corresponding to the variance of the grayscale values ​​in the grayscale image.

[0098] In this embodiment, the relationship between the transmittance adjustment coefficient and the variance can be determined based on a preset mapping relationship, so that the corresponding transmittance adjustment coefficient can be obtained based on the variance of the calculated gray value.

[0099] S720. Calculate the sum of the first transmittance of each pixel in the original image and the target transmittance adjustment coefficient to obtain the second transmittance of each pixel.

[0100] S730. The second transmittance of each pixel is compared with a preset value, and the smaller of the second transmittance of each pixel and the preset value is determined as the optimized transmittance of each pixel.

[0101] S740. The optimized transmittance of each pixel is determined as the target transmittance of the pixel.

[0102] Please see Figure 8 It illustrates a method for determining a target transmittance adjustment coefficient, the method comprising:

[0103] S810. Determine whether the variance of the grayscale values ​​in the grayscale image is less than a third threshold.

[0104] S820. When the variance of the grayscale values ​​in the grayscale image is less than the third threshold, the fourth adjustment coefficient is determined as the target transmittance adjustment coefficient.

[0105] S830. When the variance of grayscale values ​​in the grayscale image is not less than the third threshold, determine whether the variance of grayscale values ​​in the grayscale image is greater than the fourth threshold.

[0106] S840. When the variance of grayscale values ​​in the grayscale image is greater than the fourth threshold, the fifth adjustment coefficient is determined as the target transmittance adjustment coefficient.

[0107] S850. When the variance of grayscale values ​​in the grayscale image is not greater than the fourth threshold, a sixth adjustment coefficient is obtained based on the second preset function, and the sixth adjustment coefficient is determined as the target transmittance adjustment coefficient.

[0108] That is, when the variance of the grayscale values ​​in the grayscale image is greater than or equal to the third threshold and less than or equal to the fourth threshold, a sixth adjustment coefficient is obtained based on the second preset function, and the sixth adjustment coefficient is determined as the target transmittance adjustment coefficient.

[0109] The third threshold is less than the fourth threshold.

[0110] As an example, the relationship between the transmittance adjustment factor and the variance can be achieved using the following function:

[0111]

[0112] The corresponding relationship diagram can be found in [reference]. Figure 9 Where β is the transmittance adjustment coefficient, and the maximum boundary k4 and minimum boundary k3 of the variance are set. When the variance is less than the minimum boundary k3, the adjustment coefficient is 0, which is the fourth adjustment coefficient; when the variance is greater than the maximum boundary k4, the adjustment coefficient is the maximum value β. max The fifth adjustment factor; when the variance is greater than or equal to the minimum boundary k3 and less than or equal to the maximum boundary k4, the sixth adjustment factor can be determined by the second part of formula (7), i.e.

[0113]

[0114] That is, the second preset function in step S850 above can be formula (8).

[0115] In practical implementation, this method can be modified by setting k3 to 20, k4 to 50, and adjusting the maximum value of the coefficient β. max It can be set to 0.3, and this value can be changed dynamically according to the characteristics of the video image.

[0116] After obtaining the target transmittance adjustment coefficient, according to the methods in steps S720 and S730, the target transmittance adjustment coefficient t0 is added to the target transmittance adjustment coefficient β, and the sum is compared with the preset value θ to obtain the optimized transmittance t0':

[0117] t0'=min(t0+β,θ) (9)

[0118] Therefore, the optimized transmittance t0' mentioned above can be used as the target transmittance.

[0119] Furthermore, the optimized transmittance t0' can be further refined; for specific refinement methods, please refer to [reference needed]. Figure 10 The method includes:

[0120] S1010. Based on the values ​​of the three channels of each pixel in the original image, guided filtering is used to refine the optimized transmittance of each pixel to obtain the third transmittance corresponding to each pixel.

[0121] S1020. Determine the third transmittance corresponding to each pixel as the target transmittance of each pixel.

[0122] Guided filtering is used to refine the transmittance t0', with the hazy image I(x) serving as the guide image. Let ω be the window centered on pixel x. x y represents the number of pixels within the window, and the refined transmittance is expressed as:

[0123]

[0124] I(y) is a three-channel image of size w×h, t is a single-channel w×h matrix, and the final result of t is obtained by processing and summing the three channels of I(y).

[0125] Similarly, a x It is a three-channel w×h matrix, b x It is a single-channel w×h matrix, where

[0126]

[0127]

[0128] Among them, a x The molecular part is processed separately for each of the three channels to obtain a w×h matrix for the three channels; μ x and These are the image I(x) in window ω. x The mean and t0 in the window ω x The mean of μ, therefore x A three-channel, w×h matrix. A single-channel, w×h matrix.

[0129] in For t0' in window ω x The variance in ω, |ω| is the window size ω x The number of pixels in μ x The image I(x) in window ω x The average value, Is t0' in window ω x The average value is used to obtain the refined transmittance t. In this embodiment, the denominator is... Specifically, it can be a 3x3 matrix, where ε is a normalization parameter that can be set to 10. -6 , which is a three-dimensional vector, and the window size can be set to 16.

[0130] S270. Input the values ​​of the three channels of each pixel in the original image, the second atmospheric light value, and the target transmittance into a preset atmospheric light scattering model to generate a restored image corresponding to the original image.

[0131] The atmospheric light scattering model is as follows:

[0132] I(x)=J(x)·t(x)+A·(1-t(x)) (13)

[0133] Where I(x) is the foggy image, i.e. the observed image, J(x) is the fog-free image, i.e. the image to be recovered, t(x) is the transmittance, A is the atmospheric light value, i.e. the color vector of atmospheric light, and x is the pixel index.

[0134] The atmospheric light value A and transmittance t(x) were obtained through the above calculations. Given I(x), the defogging image J(x) can be calculated using formula (13), i.e.

[0135]

[0136] A schematic diagram showing the image after dehazing of an image acquired in foggy weather using the image processing method of this application can be found in the embodiment of the present application. Figure 11 ,from Figure 11 As can be seen from the image processing method of this application, after processing the foggy image, the defogging effect is obvious and the defogging image retains the information of the actual scene.

[0137] In practical implementation, the scenarios requiring image dehazing processing mostly fall into two categories: one is images captured by fixed cameras, such as traffic monitoring; the other is images captured by mobile cameras, such as short videos shot for everyday entertainment. These two scenarios have different characteristics; traffic monitoring images typically have a greater depth of field, meaning the distance to the lens is greater, and the depth of field of each pixel is similar; mobile cameras usually contain close-up objects, resulting in significant differences in depth of field across different parts of the image. Existing image dehazing methods are limited by their application scenarios and cannot be fully applied to the aforementioned two shooting scenarios. However, the image processing method provided in this application has no scenario limitations; images captured in any scenario can be processed using the image processing method of this application. Therefore, the image processing method of this application can be flexibly applied to different scenarios, adaptively adjusting the degree of dehazing, and achieving a significant and distortion-free dehazing effect.

[0138] Existing image dehazing methods often overlook inaccurate estimations of atmospheric light values, resulting in darker dehazed images that require further processing to improve brightness. The image processing method in this application adjusts the estimated atmospheric light values ​​based on the variance of grayscale images, thus avoiding a dark dehazed image. Furthermore, this application adjusts the transmittance of each pixel based on the variance of grayscale images, thereby improving the accuracy of atmospheric light value and transmittance estimation by combining these adjustments.

[0139] This application converts the original image into a grayscale image and determines the variance of the grayscale values ​​in the grayscale image. Based on the variance of the grayscale values ​​in the grayscale image, it adjusts the first atmospheric light value and the first transmittance of each pixel in the original image. The values ​​of the three channels of each pixel in the original image, the adjusted atmospheric light value, and the adjusted transmittance are input into a preset atmospheric light scattering model to generate a restored image corresponding to the original image. This application adjusts the atmospheric light value and transmittance of the image based on the variance of the grayscale values ​​in the grayscale image, improving the accuracy of the estimation of atmospheric light value and transmittance. This results in a restored image calculated based on the adjusted atmospheric light value and transmittance that better matches the real scene, restoring the detailed information of the original scene. This achieves the technical effect of restoring the original foggy image without causing image distortion.

[0140] Please see Figure 12 This application also provides an image processing apparatus, the apparatus comprising:

[0141] The original image acquisition module 1210 is used to acquire an original image and determine the three-channel values ​​of each pixel in the original image.

[0142] The first atmospheric light value determination module 1220 is used to determine the first atmospheric light value of the original image based on the three-channel values ​​of each pixel in the original image;

[0143] The variance determination module 1230 is used to convert the original image into a grayscale image and determine the variance of the grayscale values ​​in the grayscale image;

[0144] Atmospheric light value adjustment module 1240 is used to adjust the first atmospheric light value based on the variance of gray values ​​in the grayscale image to obtain a second atmospheric light value;

[0145] The first transmittance determination module 1250 is used to obtain the first transmittance of each pixel in the original image based on the values ​​of the three channels of each pixel in the original image and the second atmospheric light value.

[0146] Transmittance adjustment module 1260 is used to adjust the first transmittance of each pixel in the original image based on the variance of gray values ​​in the grayscale image, so as to obtain the target transmittance of each pixel.

[0147] The restored image generation module 1270 is used to input the values ​​of the three channels of each pixel in the original image, the second atmospheric light value, and the target transmittance into a preset atmospheric light scattering model to generate a restored image corresponding to the original image.

[0148] Furthermore, the variance determination module 1230 includes:

[0149] The grayscale value calculation module is used to assign corresponding weights to the values ​​of the three channels of each pixel in the original image, calculate the weighted sum of the values ​​of the three channels of the pixel, and determine the weighted sum of the values ​​of the three channels of the pixel as the grayscale value corresponding to the pixel.

[0150] A grayscale image generation module is used to generate the grayscale image based on the grayscale value corresponding to each pixel.

[0151] The grayscale value average calculation module is used to calculate the average grayscale value of each pixel in the grayscale image;

[0152] The grayscale value variance calculation module is used to calculate the variance of grayscale values ​​in the grayscale image based on the grayscale value of each pixel in the grayscale image and the average value of the grayscale values.

[0153] Furthermore, the atmospheric light value adjustment module 1240 includes:

[0154] The target atmospheric light value adjustment coefficient determination module is used to determine the target atmospheric light value adjustment coefficient corresponding to the variance of the gray values ​​in the grayscale image based on the relationship between the atmospheric light value adjustment coefficient and the variance.

[0155] The second atmospheric light value determination module is used to determine the second atmospheric light value by multiplying the target atmospheric light value adjustment coefficient and the first atmospheric light value.

[0156] Furthermore, the target atmospheric light value adjustment coefficient determination module includes:

[0157] The first determining module is used to determine the first adjustment coefficient as the target atmospheric light value adjustment coefficient when the variance of the gray values ​​in the grayscale image is less than the first threshold.

[0158] The second determining module is used to determine the second adjustment coefficient as the target atmospheric light value adjustment coefficient when the variance of the gray values ​​in the grayscale image is greater than the second threshold.

[0159] The third determining module is used to determine the third adjustment coefficient as the target atmospheric light value adjustment coefficient when the variance of the grayscale value in the grayscale image is greater than or equal to the first threshold and less than or equal to the second threshold.

[0160] Wherein, the first threshold is less than the second threshold.

[0161] Furthermore, the transmittance adjustment module 1260 includes:

[0162] The target transmittance adjustment coefficient determination module is used to determine the target transmittance adjustment coefficient corresponding to the variance of the grayscale values ​​in the grayscale image based on the relationship between the transmittance adjustment coefficient and the variance.

[0163] The second transmittance calculation module is used to calculate the sum of the first transmittance of each pixel in the original image and the target transmittance adjustment coefficient to obtain the second transmittance of each pixel.

[0164] The optimized transmittance determination module is used to compare the second transmittance of each pixel with a preset value, and determine the smaller value between the second transmittance of each pixel and the preset value as the optimized transmittance of each pixel.

[0165] The first target transmittance determination module is used to determine the optimized transmittance of each pixel as the target transmittance of the pixel.

[0166] Furthermore, the target transmittance adjustment coefficient determination module includes:

[0167] The fourth determining module is used to determine the fourth adjustment coefficient as the target transmittance adjustment coefficient when the variance of the grayscale values ​​in the grayscale image is less than the third threshold.

[0168] The fifth determining module is used to determine the fifth adjustment coefficient as the target transmittance adjustment coefficient when the variance of the grayscale values ​​in the grayscale image is greater than the fourth threshold.

[0169] The sixth determining module is used to determine the target transmittance adjustment coefficient by obtaining a sixth adjustment coefficient based on a second preset function when the variance of grayscale values ​​in the grayscale image is greater than or equal to the third threshold and less than or equal to the fourth threshold.

[0170] The third threshold is less than the fourth threshold.

[0171] Furthermore, the device may also include:

[0172] The transmittance refinement module is used to refine the optimized transmittance of each pixel based on the three-channel values ​​of each pixel in the original image by using guided filtering to obtain the third transmittance corresponding to each pixel.

[0173] The second target transmittance determination module is used to determine the third transmittance corresponding to each pixel as the target transmittance of each pixel.

[0174] The apparatus provided in the above embodiments can execute the method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the above embodiments can be found in the method provided in any embodiment of the present invention.

[0175] This embodiment also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded by a processor and executed as any of the methods described in this embodiment.

[0176] This embodiment also provides a device, the structural diagram of which can be found in the following figure. Figure 13 The device 1300 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1322 (e.g., one or more processors) and memory 1332, and one or more storage media 1330 (e.g., one or more mass storage devices) for storing application programs 1342 or data 1344. The memory 1332 and storage media 1330 may be temporary or persistent storage. The program stored in the storage media 1330 may include one or more modules (not shown), each module including a series of instruction operations on the device. Furthermore, the CPU 1322 may be configured to communicate with the storage media 1330 and execute the series of instruction operations in the storage media 1330 on the device 1300. The device 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input / output interfaces 1358, and / or one or more operating systems 1341, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Etc. Any of the methods described above in this embodiment can be based on... Figure 13 The equipment shown is used for implementation.

[0177] This specification provides the operational steps of the methods described in the embodiments or flowcharts, but more or fewer operational steps may be included based on conventional or non-inventive labor. The steps and order listed in the embodiments are merely one possible execution order among many steps and do not represent the only execution order. In actual system or interrupt product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0178] The structure shown in this embodiment is only a partial structure related to the solution of this application and does not constitute a limitation on the device to which the solution of this application is applied. Specific devices may include more or fewer components than shown, or combinations of certain components, or arrangements of different components. It should be understood that the methods, apparatuses, etc., disclosed in this embodiment can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or unit modules through some interfaces.

[0179] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0180] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, include: Acquire the original image and determine the three-channel values ​​of each pixel in the original image; Based on the values ​​of the three channels of each pixel in the original image, the first atmospheric light value of the original image is determined; The original image is converted into a grayscale image, and the variance of the grayscale values ​​in the grayscale image is determined. The first atmospheric light value is adjusted based on the variance of the gray values ​​in the grayscale image to obtain the second atmospheric light value; Based on the values ​​of the three channels of each pixel in the original image and the second atmospheric light value, the first transmittance of each pixel in the original image is obtained; The first transmittance of each pixel in the original image is adjusted based on the variance of the gray values ​​in the grayscale image to obtain the target transmittance of each pixel. The values ​​of the three channels of each pixel in the original image, the second atmospheric light value, and the target transmittance are input into a preset atmospheric light scattering model to generate a restored image corresponding to the original image.

2. The image processing method according to claim 1, characterized in that, The step of adjusting the first atmospheric light value based on the variance of the grayscale values ​​in the grayscale image to obtain the second atmospheric light value includes: Based on the relationship between the atmospheric light value adjustment coefficient and the variance, the target atmospheric light value adjustment coefficient corresponding to the variance of the gray values ​​in the grayscale image is determined. The product of the target atmospheric light value adjustment coefficient and the first atmospheric light value is determined as the second atmospheric light value.

3. The image processing method according to claim 2, characterized in that, The determination of the target atmospheric light value adjustment coefficient corresponding to the variance of the grayscale values ​​in the grayscale image, based on the relationship between the atmospheric light value adjustment coefficient and the variance, includes: When the variance of grayscale values ​​in the grayscale image is less than a first threshold, the first adjustment coefficient is determined as the target atmospheric light value adjustment coefficient. When the variance of grayscale values ​​in the grayscale image is greater than the second threshold, the second adjustment coefficient is determined as the target atmospheric light value adjustment coefficient. When the variance of grayscale values ​​in the grayscale image is greater than or equal to the first threshold and less than or equal to the second threshold, a third adjustment coefficient is obtained based on the first preset function, and the third adjustment coefficient is determined as the target atmospheric light value adjustment coefficient. Wherein, the first threshold is less than the second threshold.

4. The image processing method according to claim 1, characterized in that, The adjustment of the first transmittance of each pixel in the original image based on the variance of the grayscale values ​​in the grayscale image to obtain the target transmittance of each pixel includes: Based on the relationship between the transmittance adjustment coefficient and the variance, the target transmittance adjustment coefficient corresponding to the variance of the grayscale values ​​in the grayscale image is determined. The first transmittance of each pixel in the original image is calculated by summing the first transmittance of the target transmittance adjustment coefficient to obtain the second transmittance of each pixel; The second transmittance of each pixel is compared with a preset value, and the smaller value between the second transmittance of each pixel and the preset value is determined as the optimized transmittance of each pixel. The optimized transmittance of each pixel is determined as the target transmittance of the pixel.

5. The image processing method according to claim 4, characterized in that, The determination of the target transmittance adjustment coefficient corresponding to the variance of the grayscale values ​​in the grayscale image, based on the relationship between the transmittance adjustment coefficient and the variance, includes: When the variance of grayscale values ​​in the grayscale image is less than the third threshold, the fourth adjustment coefficient is determined as the target transmittance adjustment coefficient. When the variance of grayscale values ​​in the grayscale image is greater than the fourth threshold, the fifth adjustment coefficient is determined as the target transmittance adjustment coefficient. When the variance of grayscale values ​​in the grayscale image is greater than or equal to the third threshold and less than or equal to the fourth threshold, a sixth adjustment coefficient is obtained based on the second preset function, and the sixth adjustment coefficient is determined as the target transmittance adjustment coefficient. The third threshold is less than the fourth threshold.

6. The image processing method according to claim 4, characterized in that, After determining the smaller of the second transmittance of each pixel and the preset value as the optimized transmittance of each pixel, the process further includes: Based on the three-channel values ​​of each pixel in the original image, guided filtering is used to refine the optimized transmittance of each pixel to obtain the third transmittance corresponding to each pixel. The third transmittance corresponding to each pixel is determined as the target transmittance of each pixel.

7. The image processing method according to claim 1, characterized in that, The step of converting the original image to a grayscale image and determining the variance of the grayscale values ​​in the grayscale image includes: For each pixel in the original image, assign corresponding weights to the values ​​of the three channels of the pixel, calculate the weighted sum of the values ​​of the three channels of the pixel, and determine the gray value corresponding to the pixel by the weighted sum of the values ​​of the three channels of the pixel. The grayscale image is generated based on the grayscale value corresponding to each pixel. Calculate the average grayscale value of each pixel in the grayscale image; The variance of the gray values ​​in the grayscale image is calculated based on the grayscale value of each pixel and the average value of the grayscale values.

8. An image processing apparatus, characterized in that, include: The original image acquisition module is used to acquire the original image and determine the three-channel values ​​of each pixel in the original image. The first atmospheric light value determination module is used to determine the first atmospheric light value of the original image based on the three-channel values ​​of each pixel in the original image. The variance determination module is used to convert the original image into a grayscale image and determine the variance of the grayscale values ​​in the grayscale image; An atmospheric light value adjustment module is used to adjust the first atmospheric light value based on the variance of the gray values ​​in the grayscale image to obtain a second atmospheric light value; The first transmittance determination module is used to obtain the first transmittance of each pixel in the original image based on the three-channel values ​​of each pixel in the original image and the second atmospheric light value. The transmittance adjustment module is used to adjust the first transmittance of each pixel in the original image based on the variance of the gray values ​​in the grayscale image, so as to obtain the target transmittance of each pixel. The restored image generation module is used to input the values ​​of the three channels of each pixel in the original image, the second atmospheric light value, and the target transmittance into a preset atmospheric light scattering model to generate a restored image corresponding to the original image.

9. A device, characterized in that, The device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the image processing method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The storage medium stores at least one instruction or at least one program segment, which is loaded by a processor and executed by the image processing method as described in any one of claims 1 to 7.

Images