A method and system for constructing an image defogging model based on a generative adversarial network

Abstract

The present application relates to a kind of construction system and method of image defogging model based on generative adversarial network, the system at least includes second defogging module and generative adversarial network training module, second defogging module is configured to: prior defogging is carried out to foggy image to obtain verification defogging image and send to the generative adversarial network training module;The generative adversarial network training module is configured to: based on generative adversarial mapping network to the way of learning the RGB histogram of no fog image based on the RGB histogram of prior defogging image, the correlation distribution law of the characteristic value of the prior defogging image and the no fog image is obtained, to convert the distribution law into prior verification defogging model.Relative to the phenomenon of color shift in prior art, the present application uses the RGB information data after DCP defogging, and does the adversarial training with the RGB information data of no fog image, effectively solves the bright part defogging distortion problem that DCP cannot avoid.

Description

[0001] This application is a divisional application of patent application number 201911422944.7, filed on December 31, 2019, entitled "A Method and System for Dehazing and Enhancing a Single Image Based on Generative Adversarial Networks", and is classified as an invention patent. Technical Field

[0002] This invention relates to the field of image processing technology, and in particular to a method and system for constructing an image dehazing model based on generative adversarial networks. Background Technology

[0003] Fog is a natural phenomenon that blurs scenes, reduces visibility, and alters colors. Various factors in real life affect the image quality of visual systems, causing a certain degree of degradation in images acquired by imaging devices. Of particular concern is the frequent occurrence of smog across China in recent years, which has severely impacted normal social activities and industrial production. This poses significant challenges to the security of widely distributed surveillance video for criminal tracking, outdoor monitoring using computer vision, remote sensing systems, and flight navigation systems. In particular, smog poses a major safety hazard to autonomous driving technology. Image blurring leads to varying degrees of attenuation of scene features, significantly reducing the robustness and reliability of related electronic equipment systems operating in this environment. Therefore, the research and improvement of high-quality dehazing algorithms for images degraded by fog, along with the development of scientific evaluation systems, remain key areas and research hotspots in image processing and computer vision recognition. Image dehazing technology mainly refers to removing fog and haze interference from images through certain methods, restoring image color, contrast, and scene detail information to obtain high-quality images with satisfactory visual effects, while also acquiring more effective image information and providing theoretical scientific evaluation. It has significant engineering application value in reducing the impact of severe weather conditions on outdoor imaging equipment such as transportation, video surveillance, and navigation systems, and in improving the reliability and stability of related systems.

[0004] Traditional dehazing algorithms are mainly divided into two categories: dehazing algorithms based on atmospheric light scattering models for image restoration and dehazing algorithms based on image enhancement theory. Currently, the mainstream dehazing algorithms are based on atmospheric light scattering models, with the most widely used being the dark channel prior dehazing algorithm. This dark channel prior dehazing algorithm has met the requirements in many fields, but due to the lack of effective prior knowledge about foggy images, it cannot achieve optimal transparency, resulting in color shifts during image restoration. To meet the high image quality requirements in fields such as surveillance video tracking and criminal detection, computer vision-based outdoor monitoring, remote sensing systems, flight navigation, and autonomous driving technologies, further improving the quality of dehazed images is a key technical problem that needs to be solved.

[0005] Chinese Patent Publication No. CN106127702B discloses a deep learning-based image dehazing method for removing fog interference from foggy images and reducing the impact of fog on image quality. It includes: acquiring a sample set and a test sample set; performing HSL spatial transformation on the foggy images in the sample set to extract local low-brightness features, and scaling and normalizing all feature components; identifying the discriminant perspective, enabling adversarial training of the deep discriminative neural network; training the aforementioned feature components using a deep generative adversarial neural network to learn and establish a mapping network between foggy images and perspective; and performing dehazing tests on the test sample set using a deep generative neural network. This method addresses the technical problem of insufficient prior information in previous dehazing algorithms.

[0006] However, this patent cannot perceive the features weakened by fog in the dehazed image, which leads to the loss of certain details and color shifts in the image, making the fog image have the characteristic of weakening scene information and making it difficult to extract the original scene information.

[0007] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the inventors studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a single-image dehazing enhancement system based on generative adversarial networks (GANs), comprising a sample data acquisition module, a GAN training module, and a first dehazing module. The sample data acquisition module acquires several images as sample data suitable for constructing a dehazing model in the GAN training module through a publicly available image library and / or web crawling technology. The GAN training module extracts feature values ​​from the images and learns to transform these feature values ​​into a dehazing model that can be used by the first dehazing module to restore the image to be dehazed into a valid dehazed image. The sample data acquisition module acquires at least several foggy images, which have been pre-verified by the second dehazing module, and several fog-free images. This allows the generative adversarial network training module to learn the correlation distribution of feature values ​​between the prior foggy images and the fog-free images by using the generative adversarial mapping network to learn the histogram of feature values ​​of the prior foggy images. This distribution is then transformed into a pre-verified dehazing model, which the first dehazing module embeds into the image to be dehazed to restore it as a valid dehazed image. For example, the feature values ​​can be RGB.

[0009] According to a preferred embodiment, the second dehazing module is configured to perform prior dehazing on the foggy image to obtain the verification dehazed image in the following manner: In computer vision and computer graphics, a mapping model is configured to be used to a priori transform the foggy image into the verification dehazed image. The mapping model includes at least a transmittance value and an atmospheric light value to describe the mapping relationship between the foggy image and the verification dehazed image. The atmospheric light value is obtained by selecting the minimum channel image in the RGB three channels of the foggy image to obtain the dark channel image, and obtaining the atmospheric light value based on the dark channel image. The transmittance value is obtained by obtaining the transmittance value based on the dark channel prior theory. The foggy image is then subjected to prior dehazing based on the mapping model, the atmospheric light value, and the transmittance value to obtain the verification dehazed image.

[0010] According to a preferred embodiment, the generative adversarial network training module compares the effective dehazed image recovered by the dehazed model previously verified in the first dehazed module and the verified dehazed image previously verified by the second dehazed module with the image to be dehazed. This is used to obtain the correlation distribution law of the feature values ​​200 of the effective dehazed image and the verified dehazed image, so that the generative adversarial network training module can adaptively correct the dehazed model based on the previously verified effective dehazed image and the previously verified verified dehazed image.

[0011] According to a preferred embodiment, the generative adversarial network training module includes a generator and a discriminator. The generator constructs a generative network, which is used to mimic the haze-free image from the prior dehazed image to generate an intermediate dehazed image. The discriminator constructs a discriminator network, which calculates a cost function based on the intermediate dehazed image. If the calculated cost function is less than a preset dehazed threshold, the generative network is used as the dehazed model. Otherwise, the intermediate dehazed image is used as the input image of the generator for further learning until the calculated cost function is less than the preset dehazed threshold, thereby updating the generative network.

[0012] According to a preferred embodiment, the generative adversarial network training module includes a generator and a discriminator. The generator constructs a generative network, which is used to mimic the haze-free image from the prior dehazed image to generate an intermediate dehazed image. The discriminator constructs a discriminator network, which calculates a cost function based on the intermediate dehazed image. If the calculated cost function is less than a preset dehazed threshold, the generative network is used as the dehazed model. Otherwise, the intermediate dehazed image is used as the input image of the generator for further learning until the calculated cost function is less than the preset dehazed threshold, thereby updating the generative network.

[0013] According to a preferred embodiment, the first dehazing module divides the image to be dehazed into several small image blocks, and statistically analyzes the pixel value distribution of each small image block to obtain the histogram statistical result of each small image block. The histogram statistical result is input into the generator network to output the histogram corresponding to the dehazing result of each small image block. The histograms corresponding to each small image block are accumulated and stitched together to form a large image block. The large image block is filtered by a guided filter to remove unnatural transitions at the edges and corners to obtain the effective dehazed image.

[0014] According to a preferred embodiment, the generative network learns to establish a mapping network from a foggy image to a perspective view to obtain a generated perspective view, and a discriminative network generates a perspective view and an optimal perspective view; the generative network and the discriminative network meet the following requirements:

[0015]

[0016] Wherein, dist is the optimal viewpoint obtained from the training samples, serving as the discriminative viewpoint in the discrimination process. Dist is the discriminative viewpoint obtained from the samples, satisfying the distribution data p; t is the generating viewpoint of the generating network G; F is the input feature, i.e., the training feature extracted from the fog image samples; where, for the discriminative network D, it is used to distinguish between the viewpoint and the generating viewpoint; while for the generating network G, it prevents the discriminative network D from correctly distinguishing the generating viewpoint, thus enabling adversarial training against the discriminative network D.

[0017] According to a preferred embodiment, the dehazing enhancement system includes an evaluation module configured as follows: in response to receiving an effective dehazed image output by a first dehazing module, the evaluation module sends an evaluation signal to a second dehazing module, causing the second dehazing module to input a verification dehazed image to the evaluation module; the evaluation module assesses the relative quality improvement of the effective dehazed image relative to the verification dehazed image based on at least one preset evaluation metric; if the relative quality improvement is greater than or equal to a preset relative quality value, the first dehazing module outputs the effective dehazed image, and the generative adversarial network training module adaptively updates the dehazing model based on the previously verified effective dehazed image and the previously verified verification dehazed image; if the relative quality improvement is less than the preset relative quality value, the generative adversarial network training module corrects the dehazing model using at least one Gaussian regression model.

[0018] According to a preferred embodiment, the present invention also discloses a single-image dehazing enhancement method based on generative adversarial networks for use in the aforementioned system.

[0019] Compared to existing technologies, this invention utilizes the RGB information data after DCP dehazing and performs adversarial training against the RGB information data of a hazy-free image, effectively solving the problem of brightness distortion in dehazing areas that DCP cannot avoid. To address the brightness distortion problem caused by DCP, a generative adversarial network (GAN) is employed, while a discriminant network evaluates the output to ensure the output resembles a realistic image. The generator network first generates a target image. The target image is the most crucial part of this network, as it guides the generator network's focus on the hazy region. The target image is generated by a recurrent network. Then, the generator network uses a well-designed autoencoder, taking the input image and the target image as input. To obtain broader contextual information, multi-scale losses are employed on the decoder side of the autoencoder. Each loss compares the difference between the output of a convolutional layer and the corresponding ground truth. The input of the convolutional layer contains features from the decoder layer. In addition to these losses, a perceptual loss is used for the final output of the autoencoder to obtain a more comprehensive similarity to the ground truth. The final output is also the output of the generator network. After obtaining the generated image output, the discriminant network checks its realism. In fact, the target hazy region is not given during the testing phase. Therefore, there is no information that the discriminative network can focus on in local regions. To address this issue, the discriminative network is guided to target local areas. In summary, incorporating the target image into both the generative and discriminative networks is a novel approach that can effectively achieve image dehazing.

[0020] This invention also provides a system for constructing an image dehazing model based on a generative adversarial network (GAN). The system includes at least a second dehazing module and a GAN training module. The second dehazing module is configured to: perform prior dehazing on a hazy image to obtain a verification dehazed image and send it to the GAN training module. The GAN training module is configured to: obtain the correlation distribution law of the feature values ​​of the prior dehazed image and the hazy image by learning the RGB histogram of the dehazed image based on the RGB histogram of the prior dehazed image using a generative adversarial mapping network, thereby transforming the distribution law into a prior verification dehazing model.

[0021] Preferably, the generative adversarial network training module includes a generator and a discriminator. The generator is used to construct a generative network, which is used to mimic the haze-free image from the prior dehazed image to generate an intermediate dehazed image. The discriminator is used to construct a discriminator network, which calculates a cost function based on the intermediate dehazed image. If the calculated cost function is less than a preset dehazed threshold, the generative network is used as the dehazed model. Otherwise, the intermediate dehazed image is used as the input image of the generator to continue learning until the calculated cost function is less than the preset dehazed threshold, so as to update the generative network.

[0022] Preferably, it further includes a first dehazing module, which is embedded in the dehazing model to restore the image to be dehazed to an effective dehazed image; wherein, the first dehazing module inputs the histogram statistics into the generator network to output the histogram corresponding to the dehazing result of each small image block, and accumulates and stitches the histograms corresponding to each small image block into a large image block, and uses a guided filter to filter the large image block in a way that removes unnatural transitions at the edges and corners to obtain the effective dehazed image.

[0023] Preferably, the generative adversarial network training module is further configured to: compare the previously validated effective dehazed image with the previously validated verified dehazed image, obtain the correlation distribution law of the feature values ​​of the effective dehazed image and the verified dehazed image, and adaptively correct the dehazed model based on the previously validated effective dehazed image and the previously validated verified dehazed image.

[0024] Preferably, the second dehazing module is configured to perform prior dehazing on the foggy image to obtain the verification dehazed image in the following manner: In computer vision and computer graphics, a mapping model capable of prior-converting the foggy image into the verification dehazed image is configured. The mapping model includes at least a transmittance value and an atmospheric light value to describe the mapping relationship between the foggy image and the verification dehazed image. The atmospheric light value is obtained by selecting the minimum channel image from the RGB three channels of the foggy image to obtain the dark channel image, and obtaining the atmospheric light value based on the dark channel image. The transmittance value is obtained by determining the transmittance value based on the dark channel prior theory. The foggy image F is subjected to prior dehazing based on the mapping model, the atmospheric light value, and the transmittance value to obtain the verification dehazed image. The foggy image is subjected to prior dehazing based on the mapping model, the atmospheric light value, and the transmittance value to obtain the verification dehazed image.

[0025] Preferably, the system further includes an evaluation module, which, in response to receiving an effective dehazed image output by the first dehazed module, sends an evaluation signal to the second dehazed module, causing the second dehazed module to input a verification dehazed image to the evaluation module; evaluates the relative quality improvement of the effective dehazed image relative to the verification dehazed image based on at least one preset evaluation index; if the relative quality improvement is greater than or equal to a preset relative quality value, the first dehazed module outputs the effective dehazed image, and the generative adversarial network training module can adaptively update the dehazed model based on the previously verified effective dehazed image and the previously verified verification dehazed image; if the relative quality improvement is less than the preset relative quality value, the generative adversarial network training module uses at least one Gaussian regression model to correct the dehazed model.

[0026] The present invention also provides a method for constructing an image dehazing model based on a generative adversarial network. The method includes at least: performing prior dehazing on a foggy image to obtain a verification dehazed image; obtaining the correlation distribution law of feature values ​​of the prior dehazed image and the fog-free image by learning the RGB histogram of the prior dehazed image based on the generative adversarial mapping network, thereby transforming the distribution law into a prior verification dehazing model.

[0027] Preferably, the method further includes: constructing a generator network and a discriminator network; the generator network is used to mimic the haze-free image from the prior dehazed image to generate an intermediate dehazed image; the discriminator network calculates a cost function based on the intermediate dehazed image; if the cost function calculation result is less than a preset dehazed threshold, the generator network is used as the dehazed model; otherwise, the intermediate dehazed image is used as the input image of the generator for further learning until the cost function calculation result is less than the preset dehazed threshold, so as to update the generator network.

[0028] Preferably, the method further includes: inputting the histogram statistics into the generator network to output the histogram corresponding to the dehazing result of each small image block, accumulating and stitching the histograms corresponding to each small image block into a large image block, and using a guided filter to filter the large image block in a way that removes unnatural transitions at the edges and corners to obtain the effective dehazed image.

[0029] Preferably, the method further includes: comparing the previously validated effective dehazed image and the previously validated verified dehazed image, obtaining the correlation distribution law of the feature values ​​of the effective dehazed image and the verified dehazed image, and adaptively correcting the dehazed model based on the previously validated effective dehazed image and the previously validated verified dehazed image. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a single-image dehazing enhancement system based on generative adversarial networks provided by the present invention.

[0031] List of reference numerals

[0032] F: Foggy image 100: Sample acquisition module

[0033] DF1: Prior dehazed image; 200: Generative Adversarial Network module

[0034] NF: Haze-free image 300: First dehazing module

[0035] DF2: Effective dehazing image; 400: Second dehazing module

[0036] WF: Image to be dehazed; 500: Evaluation module

[0037] DF3: Verify Dehazed Image Detailed Implementation

[0038] The following is in conjunction with the appendix Figure 1 Please provide a detailed explanation.

[0039] To facilitate understanding of this invention, the technical terms used in this invention will be explained as follows.

[0040] Image defogging: A method for restoring images containing haze to restore the original visibility and color of the scene to the greatest extent possible. Based on a mapping model, atmospheric light values, and transmittance values, a priori defogging of the hazy image F can be performed to obtain a validated defogging image.

[0041] Generative Adversarial Networks (GANs) consist of both generative networks and adversarial networks.

[0042] Fog images: These are images taken under foggy or hazy conditions. Due to the absorption and scattering of ambient light by atmospheric pollutants such as water droplets, smoke, fog, and dust particles, the intensity of transmitted light is attenuated. This alters the light intensity received by the optical sensor, resulting in reduced image contrast, narrowed dynamic range, blurred images, loss of detail, obscured features, and decreased color fidelity, ultimately failing to achieve satisfactory visual effects.

[0043] The Dark Channel Prior algorithm is based on statistically significant observations. For an observed image J, the dark channel prior observations are as follows:

[0044] J dark(x) =miny∈Ω(x)(minc∈(r,g,b)J c (y));

[0045] If J dark(x) →0, then J is a fog-free outdoor image.

[0046] In the formula, Ω(x) is a local window centered at x. c It is a color channel of J.

[0047] Generative Adversarial Networks (GANs) are a type of deep learning model that represents one of the most promising methods for unsupervised learning on complex distributions in recent years.

[0048] RGB: Color refers to the three primary colors, where R represents Red, G represents Green, and B represents Blue.

[0049] In this invention, a "module" is a processor consisting of hardware, software, or a combination of hardware and software with corresponding functions. The main equipment configuration used for training in this invention is: an Intel(R) Core(TM) i5-7300HQ CPU @ 2.5GHz and an NVIDIA GeForce GTX 1050Ti.

[0050] Example 1

[0051] This embodiment discloses a single-image dehazing enhancement system based on generative adversarial networks, including a sample data acquisition module 100, a generative adversarial network training module 200, and a first dehazing module 300.

[0052] The sample data acquisition module 100 is used to acquire sample data. It primarily acquires several images through publicly available image libraries and / or through web crawling techniques. These images include both foggy and fog-free images. These images are suitable for the generative adversarial network training module 200 to construct a dehazing model. In this embodiment, the sample data acquisition module 100 can acquire several foggy images F and several fog-free images NF, and then perform preliminary verification on the several foggy images F via the second dehazing module 400 to obtain the corresponding verification dehazing image DF1.

[0053] A generative adversarial network (GAN) training module 200 extracts feature values ​​from several images and can learn and transform these feature values ​​into a dehazing model through a generative adversarial mapping (GAP) network. This dehazing model can be used by the first dehazing module 300 to restore the image WF to be dehazed into a valid dehazed image DF2. The GAN training module 200 can obtain the correlation distribution of feature values ​​between the prior dehazed image DF1 and the non-hazed image NF by learning the RGB histogram of the prior dehazed image DF1 from the RGB histogram of the prior dehazed image DF1. This distribution pattern is then transformed into a pre-validated dehazing model. The first dehazing module 300 embeds the pre-validated dehazing model into the image WF to be dehazed, restoring it into a valid dehazed image DF2. Using the RGB information data after DCP dehazing, adversarial training is performed against the RGB information data of the non-hazed image, effectively solving the problem of brightness distortion in dehazing that DCP cannot avoid. To address the brightness distortion problem caused by DCP, a generative adversarial network is used, and an output discriminant network is generated for evaluation to ensure that the output looks like a realistic image. During this process, the generative network first generates a target image. The target image is the most crucial part of this network because it guides the generative network to focus on the fogged regions. The target image is generated by a recurrent network. The generative network then utilizes a well-designed autoencoder, taking both the input image and the target image as input. To obtain broader contextual information, a multi-scale loss is employed on the decoder side of the autoencoder. Each loss compares the difference between the output of a convolutional layer and the corresponding ground truth. The input to the convolutional layers contains features from the decoder layers. In addition to these losses, a perceptual loss is used for the final output of the autoencoder to obtain a more comprehensive similarity to the ground truth. The final output is also the output of the generative network. After obtaining the generated image output, the discriminative network checks its realism. However, the target fogged region is not provided during the testing phase. Therefore, there is no information for the discriminative network to focus on in local areas. To address this issue, the target image is used to guide the discriminative network towards the local target region. In summary, incorporating the target image into both the generative and discriminative networks effectively achieves image dehazing.

[0054] Preferably, the generative adversarial network training module 200 includes a generator and a discriminator. The generator constructs a generative network, which uses the prior dehazed image DF1 to mimic the haze-free image NF to generate an intermediate dehazed image. The discriminator constructs a discriminator network, which calculates a cost function based on the intermediate dehazed image. If the cost function calculation result is less than a preset dehazing threshold, the generative network is used as the dehazing model; otherwise, the intermediate dehazed image is used as the input image of the generator for further learning until the cost function calculation result is less than the preset dehazing threshold, at which point the generative network is updated. Preferably, the first dehazing module 300 is configured to: divide the image into a series of small blocks, such that the fog concentration within each small block is not significantly different; statistically analyze the distribution of pixel values ​​within each small block, and statistically analyze the gray-level histograms for the R, G, and B channels. Assuming the horizontal axis of the gray-level histogram is spaced at 4-pixel intervals, there are a total of 256 / 4 = 64 intervals. There are three channels, and the statistical data is represented as a 64×1×3 matrix. Dividing the statistical matrix by the maximum value ensures that all input data ranges from 0 to 1, facilitating model learning. The histogram statistics of each small image patch are input into the trained network. Through a series of one-dimensional convolutions, the histogram of the corresponding block in the predicted dehazed result is finally output. Both the output and input histograms are divided by the sum of their data, making the cumulative sum equal to 1. This sum is then converted into a cumulative distribution histogram for histogram matching. After histogram matching, the dehazed result for each patch is obtained and stitched together into a larger image. Due to block-based prediction, there may be unnatural transitions between blocks; a guided filter removes these transitions. The original hazy image serves as the guide image. Preferably, the small image patch size is 8×8 pixels. Preferably, the network structure consists of three parts: a feature extraction layer, a mapping layer, and a compression layer. The feature extraction layer consists of three layers: the first layer comprises 64 3×1×3 convolutional kernels, the second layer comprises 64 3×1×64 convolutional kernels, and the third layer comprises 32 3×1×64 convolutional kernels. All convolutional kernels perform one-dimensional convolutions. Each layer performs zero-padding on the input features during convolution to ensure that all output features maintain a size of 64 in the first dimension. The mapping layer has 16 layers, each containing 32 3×1×32 convolutional kernels. The mapping layer employs a one-dimensional residual network structure. The output of each layer in the mapping layer is the sum of the input and the convolution result. The compression layer compresses the output of the mapping layer to a size of 64×1×3 for subsequent histogram matching of the three channels.

[0055] Preferably, the generative network learns to establish a mapping network from the fogged image to the perspective view, obtaining the generated perspective view, and the discriminative network generates the perspective view and the optimal perspective view. The generative network and the discriminative network meet the following requirements:

[0056]

[0057] Where T(D, G) is the objective function (loss function / value function) of the adversarial training between the discriminator network D and the generator network G, dist is the optimal perspective obtained from the training samples, serving as the discriminator perspective in the discrimination process. dist is the discriminator perspective obtained from the samples, satisfying the distribution data p. t is the generator perspective of the generator network G. F is the input feature, i.e., the training features extracted from the fog image samples. P data (t dist () is the discriminative perspective t dist It follows the true data distribution; data is essentially a symbolic mapping of the real training data, used to distinguish the real data distribution from the distribution generated by the generative network; It is a generative perspective The network follows a generative distribution, where g is a symbolic representation of the generative network G, used to specify the origin of the distribution.

[0058] In this context, the discriminant network D is used to distinguish between the viewpoint and the generated viewpoint. The generator network G, on the other hand, prevents the discriminant network D from correctly distinguishing the generated viewpoint, thus enabling adversarial training against the discriminant network D.

[0059] The discriminative network primarily classifies the input perspective to obtain the correct discrimination probability. Since perspective is correlated with image depth information, and depth information is a gradual change in a real scene, perspective changes follow certain patterns in real-world scenes. However, the generated perspective view may suffer from dehazing biases, resulting in a result that doesn't accurately reflect the changing patterns in a foggy scene or the light transmission state. Therefore, using a convolutional neural network to extract perspective features allows us to obtain features representing the perspective change patterns, which are then used as the criterion for judging perspective, thus determining the discrimination probability between the two perspective views. In the training of the convolutional layers, the selection of initial parameters directly affects the training accuracy. Parameter updates are related to the gradient changes of each input node, and training can easily result in a saturated nonlinear model. To reduce the impact of input and initialization on parameter updates and improve the training learning rate, a batch normalization layer, including normalization and the re-LU activation function, is introduced into the discriminative network. The convolutional network perceives the input information, extracts fog-related features from the perspective layer by layer, and distinguishes the two perspective views through a fully connected approach. The discriminative network primarily classifies the input perspective to obtain the correct discrimination probability. Since perspective is related to the depth information of the image, and depth information is a type of gradient information in the real scene, the change in perspective in a real scene follows certain patterns. However, due to structural biases, the generated perspective rate cannot satisfy the changing patterns in the real scene, nor can it accurately reflect the light transmission state in a foggy scene. Therefore, a convolutional neural network is used to extract perspective features, obtaining the feature variation patterns of the perspective. Using these features as a standard for perspective, the probabilities of two perspectives can be distinguished. In the training of the convolutional layers, the initial parameters have a significant impact on the training accuracy. The new model is related to the gradient changes of the input nodes in each layer, facilitating the obtaining of a saturated nonlinear model after training. To reduce the impact of input and initialization on parameter updates and improve the training learning rate, a batch normalization layer is introduced in the discriminative network. The perspective in a foggy scene is related to the saturation, color difference, vector contrast, black channel, and halo of the fog image. Therefore, a convolutional network is used to extract these features layer by layer. Since the extracted highly abstract features represent perspective and contain perspective information, the desired perspective can be obtained through mapping and reconstruction. Abstract features are decomposed using a step convolutional network, and a perspective view of the scene is obtained by mapping the sigmoid activation function. To address the issue of insufficient initial gradient and loss function descent, a batch normalization layer is added to the network. To make the perspective information more reliably reflect the actual scene, a spatial pooling layer is added to the generated network to extract coarse perspective information and perform sparsification, ensuring that the generated network has the best perspective effect.

[0060] Preferably, the generative adversarial network training module 200 compares the effective dehazed image DF2 recovered by the dehazed model previously verified in the first dehazed module 100 and the verified dehazed image DF3 previously verified by the second dehazed module 200 with the image WF to be dehazed, to obtain the correlation distribution of the feature values ​​of the effective dehazed image DF2 and the verified dehazed image DF3. The generative adversarial network training module 200 can adaptively correct the dehazed model based on the previously verified effective dehazed image DF2 and the previously verified verified dehazed image DF3. Preferably, the dehazing enhancement system includes an evaluation module 500. This embodiment corrects the dehazed model from two perspectives: model accuracy and training set search. A two-layer joint Gaussian process regression model is proposed to reduce model error; from the perspective of dataset selection, the data-driven model is optimized, and a specific design is made for the data-driven model used by the dehazing algorithm. The preset evaluation index can be:

[0061]

[0062] In the formula, PSNR reflects the ratio of the maximum possible power of the signal to the destructive noise power that affects its representation accuracy. The evaluation module 500 is configured as follows: in response to receiving the effective dehazed image DF2 output by the first dehazed module 300, it sends an evaluation signal to the second dehazed module 400, causing the second dehazed module 400 to input a verification dehazed image DF3 to the evaluation module 500. The relative improvement in quality between the effective dehazed image DF2 and the verification dehazed image DF3 is calculated using PSNR. The relative improvement in quality can be expressed as PSNR. DF2 With PSNR DF3 The difference is calculated. If the relative improvement in quality is greater than or equal to a set relative quality value, the first dehazing module 300 outputs an effective dehazed image DF2, and the generative adversarial network training module 200 adaptively updates the dehazing model based on the previously validated effective dehazed image DF2 and the previously validated validation dehazed image DF3. If the relative improvement in quality is less than the set relative quality value, the generative adversarial network training module 200 uses at least one Gaussian regression model to correct the dehazing model.

[0063] Example 2

[0064] This embodiment discloses a priori dehazing system based on dark channels. This embodiment may be a further improvement and / or supplement to Embodiment 1, and repeated content will not be described again. Where there is no conflict or contradiction, the whole and / or part of the preferred embodiments of other embodiments may be used as supplements to this embodiment.

[0065] The second dehazing module 400 is configured to perform prior dehazing on a hazy image F to obtain a verification dehazed image DF1 as follows: In computer vision and computer graphics, a mapping model is configured to convert the prior of the hazy image F into the verification dehazed image DF1. The mapping model includes at least transmittance and atmospheric light values ​​to describe the mapping relationship between the hazy image F and the verification dehazed image DF1. Atmospheric light values ​​are obtained: The minimum channel image used to obtain the dark channel image is selected from the RGB three channels of the hazy image F, and the atmospheric light value is obtained based on the dark channel image. Transmittance values ​​are obtained: The transmittance value is obtained based on the dark channel prior theory. Based on the mapping model, atmospheric light values, and transmittance values, the hazy image F is dehazed prior to obtain the verification dehazed image DF1.

[0066] Preferably, the second dehazing module 200 is a dark channel prior dehazing module, which performs prior dehazing according to at least the following steps:

[0067] S1: Find the minimum values ​​of the R, G, and B channels for each pixel in the hazy image, and then perform minimum value filtering to obtain the dark channel image of the hazy image. The specific expression is as follows:

[0068]

[0069] In the formula, J represents the foggy image, i.e., F; J c J represents one of the three color channels: R, G, and B; J c (y) represents the value of a certain color channel for each pixel in the image; Ω(x) is a filtering region centered at pixel x; J dark This is the dark channel diagram of J.

[0070] Based on the dark channel prior theory, the transparency value is determined as follows:

[0071]

[0072] Therefore, we can conclude that:

[0073]

[0074] Based on the dark channel prior theory:

[0075]

[0076] Therefore, we can conclude that:

[0077]

[0078] This is the transmittance value;

[0079] Introduce a defogging factor ω, which ranges from 0 to 1.

[0080]

[0081] S2: Calculate the position of the top 0.1% of pixels in the dark channel image, and then calculate the average value of the corresponding pixel values ​​in the foggy image as the atmospheric light value A of the foggy image.

[0082] S3: Based on the atmospheric scattering model – mapping model:

[0083] I(x)=J(x)t(x)+A(1-t(x))

[0084] Substitute the atmospheric light value and transmittance value into the mapping model to obtain the prior dehazed image DF1-J(x).

[0085] The prior dehazed image obtained based on the dark channel prior has at least the following drawbacks: due to the inconsistency between certain transmittance and depth information, visual defects such as halos and block effects exist. For this unreliable transmission estimation, filtering-based methods such as soft extinction and guided filtering are typically used to refine the computation, making the edge information of the transmitted image closer to the original image. However, the precision computation based on filtering has some drawbacks, such as the transmittance not conforming to the variation law of depth information, leading to estimation bias and weakening the contrast of the non-hazed image.

[0086] Example 3

[0087] This embodiment may be a further improvement and / or supplement to Embodiment 1, and repeated content will not be described again. Where there is no conflict or contradiction, the whole and / or part of the preferred embodiments of other embodiments may be used as supplements to this embodiment.

[0088] This embodiment discloses a single-image dehazing and enhancement method based on generative adversarial networks. It includes:

[0089] The sample data acquisition module 100 acquires several images as sample data suitable for the generative adversarial network training module 200 to construct a dehazing model by using publicly available image libraries and / or by web crawling technology;

[0090] The generative adversarial network training module 200 extracts feature values ​​from several images and transforms the feature values ​​into a dehazing model that can be used by the first dehazing module 300 to restore the image to be dehazed WF into an effective dehazed image DF2.

[0091] Thus, the generative adversarial network training module 200 can learn the correlation distribution law of the feature values ​​of the prior dehazed image DF1 and the unhazed image NF by learning the RGB histogram of the prior dehazed image DF1 from the generative adversarial mapping network. The distribution law is then transformed into a pre-validated dehazing model, so that the first dehazing module 300 can embed the pre-validated dehazing model into the image to be dehazed WF to restore it into an effective dehazed image DF2.

[0092] Preferably, the second dehazing module 400 is configured to perform prior dehazing on the foggy image F to obtain a verification dehazing image DF1 in the following manner:

[0093] In computer vision and computer graphics, a mapping model is configured to transform a prior hazy image F into a validation dehazed image DF1. This mapping model includes at least transmittance and atmospheric light values ​​to describe the mapping relationship between the hazy image F and the validation dehazed image DF1.

[0094] To obtain atmospheric light values: Select the minimum channel image from the RGB three channels of the foggy image F that can be used to obtain the dark channel image, and obtain the atmospheric light values ​​based on the dark channel image;

[0095] Transmittance value determination: The transmittance value is determined based on the dark channel prior theory;

[0096] Based on the mapping model, atmospheric light value, and transmittance value, the foggy image F is dehazed a priori to obtain the verified dehazed image DF1.

[0097] Preferably, the generative adversarial network training module 200 compares the effective dehazed image DF2 recovered by the dehazed model previously verified in the first dehazed module 100 and the verified dehazed image DF3 previously verified by the second dehazed module 200 with the image WF to be dehazed, in order to obtain the correlation distribution of the feature values ​​of the effective dehazed image DF2 and the verified dehazed image DF3, so that the generative adversarial network training module 200 can adaptively correct the dehazed model based on the previously verified effective dehazed image DF2 and the previously verified verified dehazed image DF3.

[0098] It should be noted that the specific embodiments described above are exemplary, and those skilled in the art can devise various solutions inspired by the disclosure of this invention. These solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents.

Claims

1. A system for constructing an image defogging model based on a generative adversarial network, characterized in that, The system at least comprises a first defogging module, a second defogging module and a generative adversarial network training module, The first defogging module embeds a defogging model so that the to-be-defogged image is restored to an effective defogging image; wherein the first defogging module divides the to-be-defogged image into a plurality of small block images, and counts the pixel value distribution of each small block image to obtain a histogram statistical result of each small block image, inputs the histogram statistical result into a generation network to output a histogram corresponding to the defogging result of each small block image, and splices the histogram corresponding to each small block image into a large block image, and filters the large block image by using a guided filter to remove unnatural transitions at corners to obtain the effective defogging image; The second defogging module is configured to perform prior defogging on the foggy image to obtain a verification defogging image and send it to the generative adversarial network training module; The second defogging module is configured to perform prior defogging on the foggy image to obtain the verification defogging image in the following manner: In computer vision and computer graphics, a mapping model capable of being used to convert the foggy image into the verification defogging image is configured, and the mapping model at least includes a transmittance value and an atmospheric light value used to describe the mapping relationship between the foggy image and the verification defogging image, The atmospheric light value is calculated: a minimum value channel graph capable of being used to obtain a dark channel graph is selected in the RGB three channels of the foggy image, and the atmospheric light value is obtained based on the dark channel graph; The transmittance value is calculated: the transmittance value is determined based on the dark channel prior theory; The foggy image is prior defogged based on the mapping model, the atmospheric light value and the transmittance value to obtain the verification defogging image; The generative adversarial network training module is configured to obtain the correlation distribution rule of the feature values of the prior defogging image and the haze-free image in the manner that the RGB histogram of the prior defogging image is learned based on a generative adversarial mapping network, so as to convert the distribution rule into a prior-verified defogging model; the correlation distribution rule of the feature values of the effective defogging image and the verification defogging image is obtained by comparing the prior-verified effective defogging image and the prior-verified verification defogging image, and the defogging model is adaptively corrected based on the prior-verified effective defogging image and the prior-verified verification defogging image.

2. The system according to claim 1, wherein, The generative adversarial network training module comprises a generator and a discriminator, The generator is used to construct a generation network, and the generation network is used for the prior defogging image to imitate the haze-free image to generate an intermediate defogging image; The discriminator is used to construct a discrimination network, and the discrimination network calculates a cost function based on the intermediate defogging image; if the cost function calculation result is less than a pre-set defogging threshold, the generation network is taken as a defogging model; otherwise, the intermediate defogging image is taken as the input image of the generator to continue learning until the cost function calculation result is less than the pre-set defogging threshold, so as to update the generation network.

3. The system according to claim 2, wherein the system is configured to: The system further comprises an evaluation module, sending an evaluation signal to the second dehazing module in response to receiving the effective dehazing image output by the first dehazing module, so that the second dehazing module inputs a verification dehazing image to the evaluation module; evaluating the quality improvement relative value of the effective dehazing image relative to the verification dehazing image based on at least one preset evaluation index, in the case that the quality improvement relative value is greater than or equal to a set quality relative value, the first dehazing module outputs the effective dehazing image, and the generative adversarial network training module can adaptively update the dehazing model based on the previously verified effective dehazing image and the previously verified verification dehazing image; in the case that the quality improvement relative value is less than the set quality relative value, the generative adversarial network training module corrects the dehazing model by using at least one Gaussian regression model.

4. A method for constructing an image defogging model based on a generative adversarial network, characterized in that, The method at least comprises: restoring the image to be dehazed into an effective dehazing image; wherein the image to be dehazed is divided into a plurality of small block images, and the pixel value distribution of each small block image is counted to obtain a histogram statistical result of each small block image, the histogram statistical result is input into a generative network to output a histogram corresponding to the dehazing result of each small block image, and the histogram corresponding to each small block image is accumulated and spliced into a large block image, and the large block image is filtered by using a guided filter to remove unnatural transitions at corners to obtain the effective dehazing image; performing prior dehazing on the foggy image to obtain a verification dehazing image; wherein in computer vision and computer graphics, a mapping model capable of converting a foggy image into a verification dehazing image is configured, and the mapping model at least includes a transmittance value and an atmospheric light value used to describe the mapping relationship between the foggy image and the verification dehazing image; obtaining the atmospheric light value: selecting a minimum value channel graph capable of obtaining a dark channel graph in the RGB three channels of the foggy image, and obtaining the atmospheric light value based on the dark channel graph; obtaining the transmittance value: determining the transmittance value based on the dark channel prior theory; performing prior dehazing on the foggy image based on the mapping model, the atmospheric light value and the transmittance value to obtain the verification dehazing image; learning the correlation distribution rule of the feature values of the prior dehazing image and the haze-free image based on the RGB histogram of the prior dehazing image in the generative adversarial mapping network, so as to convert the distribution rule into a previously verified dehazing model; comparing the previously verified effective dehazing image and the previously verified verification dehazing image to obtain the correlation distribution rule of the feature values of the effective dehazing image and the verification dehazing image, and adaptively correcting the dehazing model based on the previously verified effective dehazing image and the previously verified verification dehazing image.

5. The method of claim 4, wherein the method further comprises: The method further comprises: building a generative network and a discriminative network; the generative network is used to imitate the haze-free image by the prior dehazing image to generate an intermediate dehazing image; The discriminant network calculates a cost function based on the intermediate defogging image; if the result of the cost function calculation is less than a pre-set defogging threshold, the generation network is taken as a defogging model; otherwise, the intermediate defogging image is taken as an input image of the generator to continue learning until the result of the cost function calculation is less than the pre-set defogging threshold, so as to update the generation network.

Images