An image defogging method and system of improving color attenuation prior and combining AOD-Net
By improving the combination of color attenuation prior and AOD-Net, the problems of poor dehazing effect and high computational cost in single-image dehazing technology are solved, enabling rapid deployment and efficient image dehazing on hardware devices.
Patent Information
- Application Number
- CN202310426586.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-19
- Publication Date
- 2026-01-13
- Estimated Expiration
- 2043-04-19
AI Technical Summary
Existing single-image dehazing techniques suffer from poor dehazing results and high computational costs for deep learning network models, making them difficult to deploy quickly in practical applications.
By combining an improved color attenuation prior and the AOD-Net method, image quality is improved through image preprocessing and depth map constraints, using an improved loss supervision function, and hardware porting is performed using the RK3399 hardware device.
While ensuring the dehazing effect, the engineering cycle of the algorithm is shortened, the realism of the image and the dehazing effect are improved, making it suitable for practical applications.
Smart Images

Figure CN116563143B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image dehazing, and more specifically to an improved image dehazing method and system that combines color attenuation prior with AOD-Net. Background Technology
[0002] Images are the most intuitive and rapid way for people to obtain information. To ensure the effectiveness of this information, images must be complete and detailed. However, with industrial and economic development, the rapid increase in factories and motor vehicles has led to severe air pollution. When light travels, it undergoes refraction and reflection with numerous suspended particles in the air, causing some information in the captured image to be obscured, or even resulting in image distortion and loss of detail. This affects the effective information contained in the image, making subsequent high-level tasks such as feature point matching, image fusion, and super-resolution reconstruction difficult. In real life, insufficient image clarity can negatively impact intelligent transportation systems, behavior recognition, and vehicle tracking. For example, traffic accidents caused by reduced road visibility and unclear vehicle images are commonplace. Therefore, to obtain more reliable information from images, many researchers have conducted research on image enhancement, processing images with reduced clarity through dehazing, deraining, and deblurring to improve image quality and obtain more effective content, thus ensuring the normal operation of systems that require outdoor image acquisition.
[0003] There have been many research results on single-image dehazing, from basic physical modeling based on atmospheric scattering models to image quality restoration using the powerful learning capabilities of computer technology, image dehazing technology has become increasingly mature. However, in specific applications, there are still the following shortcomings: (1) Traditional methods have poor dehazing effects and are difficult to meet the standards for use. (2) Deep learning network models with better effects are large, have many parameters, and require a large amount of computation, which makes them difficult to implement in practice and takes a long time. Summary of the Invention
[0004] This invention proposes an image dehazing method and system based on an improved color attenuation prior combined with AOD-Net. The aim is to address the shortcomings of current single-image dehazing techniques by designing an image dehazing method that combines an improved color attenuation prior with AOD-Net, enabling hardware portability while achieving good dehazing results. The method first improves the traditional color attenuation prior method, using it as the image dehazing process. Then, a perceptual loss is added as the loss supervision function of the neural network to improve the quality and realism of single images. Finally, the improved dehazing method is implemented on the RK3399 hardware device under the Android system.
[0005] The objective of this invention is achieved through the following technical solution:
[0006] An image dehazing method that improves color attenuation prior and combines it with AOD-Net includes the following steps:
[0007] S1, using an improved color attenuation prior to preprocess the image, remove image artifacts, and obtain a better dehazing effect image;
[0008] The improved color attenuation prior method is obtained by changing the value of atmospheric light A and constraining the depth map.
[0009] S2, the processed image is used as input to the AOD-Net dehazing network for image quality optimization and realism enhancement. The loss supervision function used by the AOD-Net network is improved by using a loss supervision function that combines L1 and perceptual loss.
[0010] S3. Input the test image processed in step S1 into the trained AOD-Net dehazing network to achieve image dehazing.
[0011] Furthermore, the formula for calculating perceptual loss is as follows;
[0012]
[0013] x is the real image, y is the generated image, and F... i (x) and F i (y) represents the feature representation of the i-th layer in the neural network, i.e., vgg16 or vgg19, respectively, and N is the number of feature layers.
[0014] Furthermore, the improved color attenuation prior method in step S1 is as follows:
[0015] (1) Improve atmospheric light value A
[0016] In the original method, the obtained depth map is sorted by minimum pixel value filtering, and the top P% of the maximum pixel values are selected and mapped to the corresponding position in the foggy image. The maximum value of the three channels at this position is used as the atmospheric light value A of the algorithm. Since errors will occur in the process of calculating the depth map, continuing to use the depth map with errors will cause error accumulation. Therefore, the minimum filtering sort is performed on the three channels of the foggy image respectively, and then the maximum pixel value in the three channels is selected as the estimated value of atmospheric light value A.
[0017] (2) Improved depth map
[0018] By applying a contextual regularization constraint to the obtained depth map, artifacts caused by pixel edge jumps are eliminated, thereby enhancing the dehazing effect of the image.
[0019] Furthermore, the specific implementation of the improved depth map is as follows;
[0020] The depth map d(x) is used to reflect the depth information of each pixel. Different pixel values represent different depth values. Therefore, in estimating the transmittance map, instead of simply limiting the range of the image transmittance map t(x) to 0.1-0.9 using the original method, we adopt context regularization of the depth map based on the characteristic that the depth map can reflect the depth of the pixel. This is done by using formula (2) for constraint:
[0021] W(x,y)|d(x)-d(y)|(2)
[0022] Where x and y are two adjacent pixels in the depth map, and W(x,y) is used as the weight function, and the calculation formula is shown in (3):
[0023]
[0024] Where d(x) and d(y) are the neighboring depth map pixel values, and α is a specified parameter. Integrating the constraints into the entire image domain will form contextual regularization constraints on the image depth map:
[0025]
[0026] Where Ω represents the image domain, W x It is the weighted matrix of pixel x, that is, the weight matrix composed of the weight values between pixel x and its surrounding neighboring pixels;
[0027] Since the pixel values are discretely distributed, equation (4) is further discretized into (5):
[0028]
[0029] Where I is the index set of image pixels, corresponding to Ω in equation (3), i, j represent the coordinates of image pixels, W i Let w be the weighted matrix for pixel i. ij It is the discrete version of w(x,y), d i and d j These are the discrete versions of d(x) and d(y), respectively; calculating the difference between the pixel values of the image is equivalent to performing differential calculation; directly introducing the Prewitt filter operator to perform weighted processing on the influence of the pixel position of the depth map is equivalent to performing convolution operation, which reduces edge ambiguity while realizing edge extraction, and accelerates the algorithm through a fast convolution function. At this time, the calculation formula is (6):
[0030]
[0031] Where represents the element-wise multiplication operator. Represents the convolution operator, D j It is a first-order differential operator, w j It is a weighted matrix; in order to make the obtained depth map more accurate, the problem is transformed into finding the optimal solution of formula (7), that is, finding the minimum value of formula (7):
[0032]
[0033] Where d is the depth map obtained from the color attenuation prior. For patch-wise transport graph The first part of the calculated patch-wise depth map is used to measure the patch-wise depth map derived from the mapping of d(x) to the boundary constraints. The fidelity of d(x) is first determined by the first part, and the second part is to model the context constraints of d(x), where λ is the regularization parameter that balances the two terms.
[0034] To find the minimum value of (7), a semi-quadratic splitting method is used to introduce an auxiliary variable u. j And the formula (7) is transformed into a new cost function as shown in (8):
[0035]
[0036] Where β is a weight, u is obtained through a semi-quadratic splitting method. j As shown in (9):
[0037]
[0038] Where sign() is the sign function, which sets u j The optimal d is found by minimizing the following function:
[0039]
[0040] Equations (3)-(14) are quadratic functions of d, therefore satisfying (11):
[0041]
[0042] in This indicates that by aligning D around the center pixel j The filter obtained by mirroring is then used to apply a two-dimensional FFT to the above equation, assuming circular boundary conditions. The resulting optimized depth map is as follows:
[0043]
[0044] Where F represents the FFT operation, This indicates taking the average value.
[0045] Furthermore, patch-wise transport graph Where I C A C These are the color channel values for I and A. and These are the upper and lower limits of the clear image J(x), respectively, and the maximum and minimum possible values of the image pixels are directly adopted as [0,0,0] and [255,255,255].
[0046] This invention also provides an improved color attenuation prior combined with AOD-Net image dehazing system, comprising the following modules:
[0047] The image dehazing module uses an improved color attenuation prior to preprocess the image, remove image artifacts, and obtain a higher quality dehazing image.
[0048] The improved color attenuation prior method is obtained by changing the value of atmospheric light A and constraining the depth map.
[0049] The image optimization module takes the processed image as input to the AOD-Net dehazing network to perform image quality optimization and realism enhancement operations. It improves the loss supervision function used by the AOD-Net network and uses a loss supervision function that combines L1 and perceptual loss.
[0050] The testing module inputs the test image processed by the image dehazing module into the trained AOD-Net dehazing network to achieve image dehazing.
[0051] Furthermore, the formula for calculating perceptual loss is as follows;
[0052]
[0053] x is the real image, y is the generated image, and F... i (x) and F i (y) represents the feature representation of the i-th layer in the neural network, i.e., vgg16 or vgg19, respectively, and N is the number of feature layers.
[0054] Furthermore, the improved color attenuation prior method in the image dehazing module is as follows:
[0055] (1) Improve atmospheric light value A
[0056] In the original method, the obtained depth map is sorted by minimum pixel value filtering, and the top P% of the maximum pixel values are selected and mapped to the corresponding position in the foggy image. The maximum value of the three channels at this position is used as the atmospheric light value A of the algorithm. Since errors will occur in the process of calculating the depth map, continuing to use the depth map with errors will cause error accumulation. Therefore, the minimum filtering sort is performed on the three channels of the foggy image respectively, and then the maximum pixel value in the three channels is selected as the estimated value of atmospheric light value A.
[0057] (2) Improved depth map
[0058] By applying a contextual regularization constraint to the obtained depth map, artifacts caused by pixel edge jumps are eliminated, thereby enhancing the dehazing effect of the image.
[0059] Furthermore, the specific implementation of the improved depth map is as follows;
[0060] The depth map d(x) is used to reflect the depth information of each pixel. Different pixel values represent different depth values. Therefore, in estimating the transmittance map, instead of simply limiting the range of the image transmittance map t(x) to 0.1-0.9 using the original method, we adopt context regularization of the depth map based on the characteristic that the depth map can reflect the depth of the pixel. This is done by using formula (2) for constraint:
[0061] W(x,y)|d(x)-d(y)|(2)
[0062] Where x and y are two adjacent pixels in the depth map, and W(x,y) is used as the weight function, and the calculation formula is shown in (3):
[0063]
[0064] Where d(x) and d(y) are the neighboring depth map pixel values, and α is a specified parameter. Integrating the constraints into the entire image domain will form contextual regularization constraints on the image depth map:
[0065]
[0066] Where Ω represents the image domain, W x It is the weighted matrix of pixel x, that is, the weight matrix composed of the weight values between pixel x and its surrounding neighboring pixels;
[0067] Since the pixel values are discretely distributed, equation (4) is further discretized into (5):
[0068]
[0069] Where I is the index set of image pixels, corresponding to Ω in equation (3), i, j represent the coordinates of image pixels, W i Let w be the weighted matrix for pixel i. ij It is the discrete version of w(x,y), d i and d j These are the discrete versions of d(x) and d(y), respectively; calculating the difference between the pixel values of the image is equivalent to performing differential calculation; directly introducing the Prewitt filter operator to perform weighted processing on the influence of the pixel position of the depth map is equivalent to performing convolution operation, which reduces edge ambiguity while realizing edge extraction, and accelerates the algorithm through a fast convolution function. At this time, the calculation formula is (6):
[0070]
[0071] Where represents the element-wise multiplication operator. Represents the convolution operator, D j It is a first-order differential operator, w j It is a weighted matrix; in order to make the obtained depth map more accurate, the problem is transformed into finding the optimal solution of formula (7), that is, finding the minimum value of formula (7):
[0072]
[0073] Where d is the depth map obtained from the color attenuation prior. For patch-wise transport graph The first part of the calculated patch-wise depth map is used to measure the patch-wise depth map derived from the mapping of d(x) to the boundary constraints. The fidelity of d(x) is first determined by the first part, and the second part is to model the context constraints of d(x), where λ is the regularization parameter that balances the two terms.
[0074] To find the minimum value of (7), a semi-quadratic splitting method is used to introduce an auxiliary variable u. j And the formula (7) is transformed into a new cost function as shown in (8):
[0075]
[0076] Where β is a weight, u is obtained through a semi-quadratic splitting method. j As shown in (9):
[0077]
[0078] Where sign() is the sign function, which sets u j The optimal d is found by minimizing the following function:
[0079]
[0080] Equations (3)-(14) are quadratic functions of d, therefore satisfying (11):
[0081]
[0082] in This indicates that by aligning D around the center pixel j The filter obtained by mirroring is then used to apply a two-dimensional FFT to the above equation, assuming circular boundary conditions. The resulting optimized depth map is as follows:
[0083]
[0084] Where F represents the FFT operation, This indicates taking the average value.
[0085] Furthermore, patch-wise transport graph Where I C A C These are the color channel values for I and A. and These are the upper and lower limits of the clear image J(x), respectively, and the maximum and minimum possible values of the image pixels are directly adopted as [0,0,0] and [255,255,255].
[0086] The present invention has the following beneficial effects:
[0087] (1) The single-image dehazing method included in this invention combines traditional methods with deep learning. Unlike many current deep learning methods that rely on atmospheric scattering models for image dehazing, this method uses an improved traditional method as the image dehazing part and the pre-processed image as the input to the network, which can improve the dehazing effect and the realism of the image.
[0088] (2) This invention uses an improved traditional method as an image dehazing method and a neural network AOD-Net as an optimization method to improve image quality. First, the improved traditional image dehazing method is used to dehaze the foggy image so that the image has a certain dehazing effect. Then, the processed image is used as the input of the neural network AOD-Net. The learning ability of the neural network is used to adjust the quality effect of the image.
[0089] (3) The image dehazing method designed in this invention based on improved color attenuation prior and combined with AOD-Net can prove that after using the traditional method with improved effect to preprocess the image, the image can be processed by the neural network layer to achieve better dehazing effect. In specific applications, the better neural network can be selected as the image optimization network according to the subsequent DSP performance, so as to ensure better dehazing effect while making the algorithm's engineering cycle short and its practicality high. Attached Figure Description
[0090] Figure 1 This is a flowchart of the image dehazing method based on improved color attenuation prior and combined with AOD-Net according to the present invention.
[0091] Figure 2 This is a schematic diagram of the Prewitt operator in an embodiment of the present invention. Detailed Implementation
[0092] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0093] Figure 1 This is a flowchart of the image dehazing method based on improved color attenuation prior and combined with AOD-Net according to the present invention. The method of this embodiment includes the following steps:
[0094] S1: To address the image artifact problem caused by color attenuation prior, we first improve the traditional method to obtain images with better dehazing effects;
[0095] S2: The processed image is used as input to the AOD-Net dehazing network for image quality optimization and realism enhancement.
[0096] S3: Combining the advantages and disadvantages of traditional methods and deep learning, we improve the loss supervision function used by the AOD-Net network by using a loss supervision function that combines L1 and perceptual loss.
[0097] S4: The improved image dehazing model is ported to hardware, and the algorithm is implemented on the DSP.
[0098] In this embodiment, the method for dehazing the image in S1 is an improved version of the traditional color attenuation prior method. The color attenuation prior method, as a traditional image dehazing method, was developed by Zhu et al. by observing the relationship between fog density and brightness / saturation among foreground, background, and midground in multiple images. They concluded that fog density is positively correlated with the difference in brightness and saturation, and proposed a linear fitting equation: d(x) = θ0 + θ1v(x) + θ2s(x) + ε(x), where d(x) is the image depth map, v(x) and s(x) are the image brightness and saturation respectively, and ε(x) represents an expected value of 0. After applying varying degrees of fogging to 5000 images using a neural network, the fogged images were compared and fitted with clear images. The resulting formula for calculating the depth map is d(x) = 0.121779 + 0.959710 * v(x) - 0.780245 * s(x) + ε(x). Based on the relationship between the depth map and the transmittance map, t(x) = e -βd(x) The transmittance map is obtained, where t(x) is the image transmittance map, β is the atmospheric scattering coefficient, d(x) is the image depth map obtained earlier, θ0, θ1, and θ2 are unknown coefficients, and e is the natural constant. To obtain the atmospheric light value A, after applying a minimum value filter to the depth map, the first 0.1% of pixels in the image are selected. The corresponding position in the foggy image is then calculated, and the pixels at that position are sorted again. The maximum value is selected as the atmospheric light value A. Finally, the clear image is derived by inverse operation based on the atmospheric scattering model I(x) = J(x)t(x) + A(1-t(x)). Where I(x) represents the generated foggy image, J(x) represents the original clear image, A represents the atmospheric light value, and t(x) represents the medium transmission.
[0099] Specifically, the improved color attenuation prior method is as follows:
[0100] (1) Improved atmospheric light value A
[0101] In the original method, the obtained depth map is sorted by minimum pixel value filtering, and the top 0.1% of the maximum pixel values are assigned to the corresponding positions in the foggy image. The maximum value of the three channels at this position is used as the atmospheric light value A of the algorithm. Since errors will occur in the process of calculating the depth map, continuing to use the depth map with errors will cause error accumulation. Therefore, the minimum filtering sort is performed on the three channels of the foggy image respectively, and then the maximum pixel value in the three channels is selected as the estimated value of atmospheric light value A.
[0102] (2) Improved depth map
[0103] The depth map d(x) reflects the depth information of each pixel, with different pixel values representing different depth values. Therefore, in estimating the transmittance map, instead of simply limiting the range of t(x) to (0.1-0.9) using the original method, we adopt context regularization based on the characteristic that the depth map can reflect the depth of pixels, and constrain it using formula (2):
[0104] W(x,y)|d(x)-d(y)| (2)
[0105] Where x and y are two adjacent pixels in the depth map, and W(x,y) is the weight function. When the difference between d(x) and d(y) is too large, the weight function makes d(x)-d(y) tend to 0. That is, when the difference between two adjacent pixels x and y in the depth map is large, the corresponding weight function W(x,y) will definitely be small. Therefore, W(x,y) is as shown in (3):
[0106] W(x,y)=e -||d(x) -d(y)|| 2 / 2α 2 (3)
[0107] Where d(x) and d(y) are the pixel values of adjacent depth maps, and α is a specified parameter, which is taken as 0.5 in this paper. Integrating the constraints into the entire image domain will form a contextual regularization constraint on the image depth map d(x):
[0108]
[0109] Where Ω represents the image domain, W x Let be the weighted matrix of pixel x, that is, the weight matrix composed of the weight values of pixel x and its surrounding neighboring pixels. In this paper, the neighborhood range is selected as 3x3. Since the pixel values are discretely distributed, equation (4) is further discretized into (5):
[0110]
[0111] Where I is the index set of image pixels, corresponding to Ω in equation (3), i, j are image pixels, W i Let w be the weighted matrix for pixel i. ij It is the discrete version of w(x,y), d i and d jThese are the discrete versions of d(x) and d(y), respectively. Since calculating the difference between image pixel values is equivalent to performing a differential calculation, the Prewitt filter operator is directly introduced to weight the influence of pixel positions in the depth map, i.e., to perform a convolution operation. This reduces edge blurring while achieving edge extraction, providing flexibility for using context constraints. The algorithm is further accelerated by a fast convolution function, using a first-order differential Prewitt operator such as... Figure 2 As shown.
[0112] Furthermore, since the L1 norm sums the absolute values of each parameter, while the L2 norm sums the squares of each parameter, the effect of the L1 norm is less affected when abnormal pixels such as excessively bright or dark pixels appear in the image. Therefore, this paper uses the integral of the L1 norm for regularization. The calculation formula is (6):
[0113]
[0114] Where ω is the index set, and represents the element-wise multiplication operator. Represents the convolution operator. D j It is a first-order differential operator, w j (j∈ω) is a weighted matrix.
[0115] To make the obtained d(x) more accurate, the problem can be transformed into finding the optimal solution of formula (7), that is, finding the minimum value of formula (7):
[0116]
[0117] Where d is the depth map obtained from the color attenuation prior. For patch-wise transport graph The calculated patch-wise depth map. Patch-wise transport map. Where I C A C These are the color channel values for I and A. and These are the upper and lower limits of the clear image J(x), respectively. In this paper, we directly use the maximum and minimum possible values of image pixels [0,0,0] and [255,255,255]. The first part of formula (7) is the data item, which is used to measure the patch-level depth map derived by d(x) on the boundary constraint mapping. The fidelity is then determined. The second part models the contextual constraints of d(x), where λ is the regularization parameter balancing these two terms, and its value in this paper is 1.
[0118] To find the minimum value of (7), a semi-quadratic splitting method is used to introduce an auxiliary variable u. jAnd the formula (7) is transformed into a new cost function as shown in (8):
[0119]
[0120] Where β is a weight, chosen as 0.05 in the experiment, and u is obtained using the semi-quadratic splitting method. j As shown in (9):
[0121]
[0122] Where sign() is the sign function, which sets u j The optimal d is found by minimizing the following function:
[0123]
[0124] Equation (10) is a quadratic function of d, therefore it satisfies (11):
[0125]
[0126] in This indicates that by aligning D around the center pixel j The filter obtained by mirroring is then used to apply a two-dimensional FFT to the above equation, assuming circular boundary conditions. The resulting optimized depth map is as follows:
[0127]
[0128] Where F represents the FFT operation, F represents the average value. -1 It is the reciprocal of F.
[0129] Specifically, S3 uses AOD-Net as the main image optimization network. Its network structure is well known to those skilled in the art. Since this neural network learns parameters by comparing the input image with the ground-true image (the original clear image), the generated image can be closer to the real image in terms of detail perception. Using the dehazed J(x) as the network input can achieve better image realism while obtaining a good dehazing effect.
[0130] Specifically, S3 uses L1 and perceptual loss together as the network's loss supervision function. Perceptual loss takes the features extracted by the convolutional neural network as part of the objective function. By comparing the feature values of the image to be generated after passing through the CNN with the feature values of the target image after passing through the CNN, the image to be generated and the target image are made more semantically similar (compared to pixel-level loss functions). This loss is more in line with the visual perception of the human eye.
[0131] Combining the advantages and disadvantages of traditional methods and deep learning, this paper improves the loss supervision function used in the AOD-Net network by employing a loss supervision function that combines L1 and perceptual loss. Perceptual loss compares the features obtained from convolution of real images (usually extracted using VGG16 or VGG19) with the features obtained from convolution of generated images (usually using the MSE loss function), making the high-level information (content and global structure) closer, which is the meaning of perception. This loss function is more in line with the visual characteristics of the human eye, resulting in images that are subjectively superior to human perception. The calculation formula is shown below:
[0132]
[0133] x is the real image, y is the generated image, Fi(x) and Fi(y) represent their feature representations in the i-th layer of the neural network, i.e., vgg16 or vgg19, respectively, and N is the number of feature layers.
[0134] Specifically, the DSP in S4 uses the iTop-RK3399 development platform, which is based on the ARM architecture and consists of two high-performance Cortex-A72 cores with a main frequency of up to 1.8GHz and four low-power Cortex-A53 cores with a main frequency of 1.4GHz. It is also equipped with an ARM Mali-T860MP4 graphics processor.
[0135] This invention also provides an improved color attenuation prior combined with AOD-Net image dehazing system, comprising the following modules:
[0136] The image dehazing module uses an improved color attenuation prior to preprocess the image, remove image artifacts, and obtain a higher quality dehazing image.
[0137] The improved color attenuation prior method is obtained by changing the value of atmospheric light A and constraining the depth map.
[0138] The image optimization module takes the processed image as input to the AOD-Net dehazing network to perform image quality optimization and realism enhancement operations. It improves the loss supervision function used by the AOD-Net network and uses a loss supervision function that combines L1 and perceptual loss.
[0139] The testing module inputs the test image processed by the image dehazing module into the trained AOD-Net dehazing network to achieve image dehazing.
[0140] The specific implementation methods and corresponding steps of each module are not described in this invention.
[0141] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications made within the technical scope of the present invention that achieve the technical effects of the present invention are covered within the protection scope of the present invention.
Claims
1. An image defogging method of improving color attenuation prior and combining AOD-Net, characterized in that, Comprise the following steps: S1, using improved color attenuation prior method for image preprocessing, removing image artifacts, obtaining better quality fog treatment effect image; The improved color attenuation prior method is obtained by changing the value of atmospheric light A and constraining the depth map; The improved color attenuation prior method in step S1 is as follows: (1) improve the atmospheric light value A In the original method, by performing pixel value minimum filtering sorting on the obtained depth map, the maximum value of the first P% of pixels is selected to correspond to the corresponding position of the foggy image, and the maximum value of the three channels at this position is taken as the atmospheric light value A of the algorithm. Since errors will occur in the process of calculating the depth map, using the depth map with errors to continue calculating will cause error accumulation, so the minimum filtering sorting is directly performed on the three channels of the foggy image, and then the maximum pixel value in the three channels is selected as the estimated value of the atmospheric light value A; (2) improved depth map By constraining the obtained depth map with a context regularization, the artifact problem caused by pixel edge jumping is eliminated, thereby enhancing the image dehazing processing effect; S2, the processed image is used as the input of AOD-Net dehazing network for image quality optimization and reality improvement operation, and the loss supervision function used by AOD-Net network is improved, and the loss supervision function combining L1 and perceptual loss is used; The calculation formula of perceptual loss is as follows; (1) x is the real picture, y is the generated picture, F i (x) and F i (y) represent the feature representation of the i-th layer in the neural network, namely vgg16 or vgg19, respectively, and N is the number of feature layers. S3, the test image processed in step S1 is input into the trained AOD-Net dehazing network to realize image dehazing.
2. The image defogging method of claim 1, wherein the color attenuation prior is improved and combined with the AOD-Net. The specific implementation of the improved depth map is as follows; The depth map d(x) is used to reflect the depth information of each pixel point, and different pixel values represent different depth values, so in the estimation of the transmittance map, instead of simply limiting the range of the image transmittance map t(x) to 0.1-0.9 by the original method, the context regularization is adopted according to the characteristics of the depth map that can reflect the depth of the pixel points, and the constraint is performed by using formula (2): (2) Wherein, x, y are two adjacent pixel points in the depth map, W(x, y) is a weight function, and the calculation formula is shown in formula (3): (3) where d(x), d(y) are adjacent depth image pixel values, ɑ To define parameters, the constraint set is integrated into the entire image domain, which forms a contextual regularization constraint on the image depth map: (4) where Ω is the image domain, W x is the weight matrix of pixel x, i.e., the weight values between pixel x and its surrounding neighborhood pixels. Since the pixel value is a discrete distribution, formula (4) is further discretized to formula (5): (5) where I is the index set of image pixel points, corresponding to Ω in equation (3), i, j represent the coordinates of image pixel points, W i is the weighting matrix of pixel point i, w ij is the discrete version of w(x, y), d i and d j are the discrete versions of d(x) and d(y), respectively; the difference calculation of image pixel values is equivalent to the differential calculation, and the influence of the Prewitt filter operator on the pixel position of the depth image is directly introduced for weighted processing, that is, convolution operation, which reduces the edge blur degree while realizing edge extraction, and the algorithm is accelerated through the fast convolution function, and the calculation formula is (6): (6) wherein represents an element-wise multiplication operator, represents a convolution operator, D j is a first-order differential operator, w j is a weighting matrix; in order to make the obtained depth map more accurate, the problem is converted into finding the optimal solution of formula (7), i.e. finding the minimum value of formula (7): (7) Where d is the depth map obtained from the color attenuation prior. For patch-wise transport graph The first part of the calculated patch-wise depth map is used to measure the patch-wise depth map derived from the mapping of d(x) to the boundary constraints. The first part focuses on fidelity, and the second part involves modeling the contextual constraints of d(x). It is the regularization parameter that balances these two terms; To solve the minimum value of (7), the method of semi-quadratic splitting is used to introduce auxiliary variables u j and formula (7) is converted into a new cost function as shown in (8): (8) where β is a weight, and u is obtained by a semi-quadratic splitting method j is shown in (9). (9) where sign() is the sign function, which sets u j The optimum d is found by minimizing the following function: (10) Formula (3)-(14) is a quadratic function of d, so it satisfies (11): (11) wherein represents the filter obtained by mirroring the center pixel pair D j around itself, applying a two-dimensional FFT and assuming circular boundary conditions, the resulting optimized depth map is: (12) where F denotes an FFT operation, denotes an average.
3. The image defogging method of claim 2, wherein the color attenuation prior is improved and combined with the AOD-Net. patch-wise level transmission graph where I C ,A C are the color channel values of I and A, and are the upper and lower bounds of the sharp map J(x), directly taking the possible maximum and minimum values of the image pixel points [0, 0, 0] and [255, 255, 255], respectively.
4. An image defogging system for improving color attenuation prior and combining AOD-Net, characterized in that, Comprise the following modules: Image dehazing processing module, using improved color attenuation prior for image preprocessing, removing image artifacts, obtaining better quality fog treatment effect image; The improved color attenuation prior method is obtained by changing the value of atmospheric light A and constraining the depth map; The improved color attenuation prior method in the image dehazing processing module is as follows: (1) improve the atmospheric light value A In the original method, by pixel value minimum filtering sorting on the obtained depth map, the first P% pixel maximum value is selected to correspond to the corresponding position of the fog image, and the maximum value of the three channels of the position is taken as the atmospheric light value A of the algorithm. Since errors will occur in the process of calculating the depth map, using the depth map with errors to continue calculating will cause error accumulation, therefore, the three channels of the fog image are directly subjected to minimum filtering sorting, and then the maximum pixel value in the three channels is selected as the estimated value of the atmospheric light value A; (2) Improved depth map By performing a context regularization constraint on the obtained depth map, the artifact problem caused by pixel edge jumping is eliminated, thereby enhancing the image dehazing processing effect; Image optimization module, the processed image is taken as the input of AOD-Net dehazing network for image quality optimization and reality improvement operation, the loss supervision function used by AOD-Net network is improved, and the loss supervision function combining L1 and perceptual loss is used; The calculation formula of perceptual loss is as follows; (1) x is the real picture, y is the generated picture, F i (x) and F i (y) represent the feature representation of the i-th layer in the neural network, namely vgg16 or vgg19, respectively, and N is the number of feature layers. Test module, the test image processed by the image dehazing processing module is input into the trained AOD-Net dehazing network to realize image dehazing.
5. The image defogging system of claim 4, wherein the color attenuation prior is improved and combined with the AOD-Net. The specific implementation mode of the improved depth map is as follows; The depth map d(x) is used to reflect the depth information of each pixel point, and different pixel values represent different depth values, therefore, in the estimation of the transmittance map, instead of simply limiting the range of the image transmittance map t(x) to 0.1-0.9 by the original method, the context regularization method is adopted according to the characteristics of the depth map that can reflect the depth of the pixel point, and the constraint is performed by using formula (2): (2) Wherein, x, y are two adjacent pixel points in the depth map, W(x, y) is a weight function, and the calculation formula is as shown in formula (3): (3) where d(x), d(y) are the adjacent depth image pixel values, ɑ To define parameters, the constraint set is integrated into the entire image domain, which forms a contextual regularization constraint on the image depth map: (4) where Ω is the image domain, W x is the weight matrix of pixel x, i.e., the weight values between pixel x and its surrounding neighborhood pixels. Since the pixel value is a discrete distribution, formula (4) is further discretized as formula (5): (5) where I is the index set of image pixels, corresponding to Ω in equation (3), i, j represent the coordinates of image pixels, W i is the weighting matrix of pixel i, w ij is the discrete version of w(x, y), d i and d j are the discrete versions of d(x) and d(y), respectively; the difference calculation of image pixel values is equivalent to the differential calculation, and the influence of the Prewitt filter operator on the pixel position of the depth image is directly introduced for weighted processing, that is, convolution operation, which reduces the edge blur degree while realizing edge extraction, and the algorithm is accelerated through the fast convolution function, and the calculation formula is (6): (6) wherein represents an element-wise multiplication operator, represents a convolution operator, D j is a first-order differential operator, w j is a weighting matrix; in order to make the obtained depth map more accurate, the problem is converted into finding the optimal solution of formula (7), i.e. finding the minimum value of formula (7): (7) where d is the depth map obtained from the color attenuation prior, patch-wise level transmission map The computed patch-wise level depth map, the first part of the data item, is used to measure the fidelity of d(x) to the boundary-constrained map-derived patch-level depth map The second part is to model the contextual constraint on d(x), is the regularization parameter balancing the two terms; To solve the minimum of (7), the method of semi-quadratic splitting is used to introduce auxiliary variables u j and transform (7) into a new cost function as shown in (8): (8) where β is a weight, and u is obtained by a semi-quadratic splitting method j is shown in (9). (9) where sign() is the sign function, which sets u j The optimum d is found by minimizing the following function: (10) Formula (3)-(14) is a quadratic function of d, therefore, formula (11) is satisfied: (11) wherein represents the filter obtained by mirroring the center pixel pair D j around itself, applying a two-dimensional FFT and assuming circular boundary conditions, the resulting optimized depth map is: (12) where F denotes an FFT operation, denotes an average.
6. The image defogging system of claim 5, wherein the color attenuation prior is improved and combined with the AOD-Net. patch-wise level transmission graph where I C ,A C are the color channel values of I and A, and are the upper and lower bounds of the sharp map J(x), directly taking the possible maximum and minimum values of the image pixel points [0, 0, 0] and [255, 255, 255], respectively.
Citation Information
Patent Citations
Aerial image defogging method based on boundary constraint and color correction
CN113313702A
A single stage multi-spectral image and video dehazing system and its method thereof
IN201911024268A