Image defogging recognition method and system based on improved generative adversarial network

Abstract

The application discloses an image defogging recognition method and system based on an improved generative adversarial network, which is used for feature extraction of an image to output feature maps of different scales, generates a mapping from a fixed input to an ambient light for estimating a pseudo ambient light, simultaneously generates a mapping to the ambient light after scattering using the same input, splices and combines the generated mapping and an input haze-free image into a new pseudo haze image, judges whether the pseudo haze image is a real image, reversely calculates the transmittance and ambient light value of an atmospheric scattering model, obtains parameters in an actual scene, and obtains a defogged image using the calculated parameters. The application can make the network extract more image features and better estimate the sound transmission law and ambient light value, and the color loss module can avoid the phenomenon of color change of the generated image, solve the problems of image information distortion and color change in the image recognition in the image defogging link, and improve the precision of image defogging recognition.

Description

Technical Field

[0001] This invention relates to the field of adversarial network model technology, and in particular to an image dehazing and recognition method and system based on improved generative adversarial networks. Background Technology

[0002] With the rapid changes in environmental factors, many regions are experiencing an increased frequency of fog and haze. These weather phenomena significantly impact satellite remote sensing monitoring, target recognition and tracking, and traffic monitoring. Therefore, image dehazing has crucial practical application value in these areas. Furthermore, the presence of fog and haze reduces color saturation and contrast in images, causing the loss of many image details. Thus, image dehazing also holds significant research value.

[0003] Traditional image dehazing methods often use convolutional networks to estimate transmittance and ambient light values ​​to restore a hazy image. However, these methods tend to ignore contextual information during the estimation process and focus solely on feature processing, neglecting the underlying mechanisms that cause hazy images. Consequently, traditional methods often result in poor dehazing performance and low accuracy in image recognition. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] To address this, this invention proposes an image dehazing recognition method based on an improved generative adversarial network. This method can directly learn the mapping relationship between hazy and foggy images. Furthermore, the deep residual connections added to the network model enable the network to extract more image features, thus better estimating sound transmission law and ambient light values. Simultaneously, color loss avoids the phenomenon of color changes in the generated image, solving the problems of image information distortion and color changes in the image dehazing process and improving the accuracy of image dehazing recognition.

[0006] Another objective of this invention is to propose an image dehazing method based on an improved generative adversarial network.

[0007] To achieve the above objectives, this invention proposes an image dehazing and recognition method based on an improved generative adversarial network, comprising:

[0008] A training dataset containing foggy image samples and corresponding fog-free image samples is obtained, and a dehazing recognition network model is constructed to train the dehazing recognition network model using the training dataset; wherein, the dehazing recognition network model includes a generator network model and a discriminator network model, the generator network model includes a feature extraction network and a feature enhancement network, and the feature enhancement network includes a residual connection network and a multi-head attention network;

[0009] The training dataset is input into the generative network model to perform a first feature classification operation on the features of the foggy image samples and the corresponding fog-free image samples using the feature extraction network to obtain a multi-scale feature map. The multi-scale feature map is then input into the feature enhancement network to perform a second feature classification operation on the multi-scale feature map using the residual connection network and the multi-head attention network to obtain a fused feature map. Based on the fused feature map, a mapping between the atmospheric ambient light image and the scattered atmospheric ambient light image is generated. The transmittance and ambient light value of the atmospheric scattering model are calculated according to the feature parameters of the mapping to generate clear, fog-free image samples.

[0010] The clear, fog-free image samples are input into the discrimination network model. A third feature classification operation is performed on the mapping and the clear, fog-free image samples using a preset stitching method to obtain pseudo-foggy images. The loss function is used to calculate the loss between the pseudo-foggy images and the foggy image samples. The model parameters of the defoggy recognition network model are optimized using the optimization objective function and the loss calculation results to obtain a trained defoggy recognition network model.

[0011] The new foggy image to be identified is input into the trained dehazing recognition network model to perform image dehazing recognition, so as to obtain the dehazing recognition result of the clear, fog-free image corresponding to the new foggy image.

[0012] In addition, the image dehazing recognition method based on the improved generative adversarial network according to the above embodiments of the present invention may also have the following additional technical features:

[0013] Further, in one embodiment of the present invention, the feature enhancement network employs a cascaded network; the second feature classification operation, which inputs the multi-scale feature map into the feature enhancement network to fuse the multi-scale feature map using the residual connection network and the multi-head attention network, to obtain a fused feature map, includes:

[0014] The multi-scale feature map is input into the cascaded network to extract the initial feature information of the multi-scale feature map using 5*5 channel-wise convolution.

[0015] Multi-scale contextual information of the initial feature information is extracted by 7*7 channel-wise dilated convolution;

[0016] The multi-scale context information is convolved using 1*1 pointwise convolution to output the fused feature map.

[0017] Furthermore, in one embodiment of the present invention, the stitching method includes a pixel-to-pixel synthesis and addition method; the discriminative network model includes multiple convolutional layers, wherein each convolutional layer uses ReLU as an activation function; a color consistency loss is added to the loss function, wherein the loss function is:

[0018]

[0019] Where p represents a pixel, ANGLE is the angle calculation function, y is the foggy image, and G... B (G A (x)) and G A (G B (y) is the generated pseudo-hazy image.

[0020] Furthermore, in one embodiment of the present invention, the step of calculating the transmittance and ambient light value of the atmospheric scattering model based on the characteristic parameters of the mapping to generate clear, haze-free image samples includes:

[0021] Formula for calculating atmospheric light scattering:

[0022]

[0023] I(x)=J(x)t(x)+A(x)(1-t(x))

[0024] Where A0 is the ambient light, A1 is the diffused ambient light, M0 and M1 are the mapping relationship matrices generated by the generation module from the background image to the ambient light image and the diffused ambient light image, x refers to the coordinates of each pixel point in the image, J(x) is the clear, fog-free image, I(x) is the actual foggy image, A is the ambient light value of the image shooting location, and t(x) represents the ambient light transmittance.

[0025]

[0026] J(x) = M0[I(x) + b(x)]

[0027] in:

[0028] Further, in one embodiment of the present invention, the generated mapping includes a first mapping for estimating pseudo-ambient light values ​​and a second mapping for estimating transmittance, wherein the first mapping is a mapping from the fused feature map to the atmospheric ambient light image, comprising: constructing a mapping of the fused feature map to the input image N(x) to generate atmospheric ambient light, so as to estimate the atmospheric ambient light image A0(x):

[0029]

[0030] The second mapping is the mapping from the fused feature map to the scattered atmospheric ambient light image, including: mapping from the same input image N(x) based on the fused feature map to the scattered atmospheric ambient light image A1(x):

[0031]

[0032] Based on the estimation, A0, t(x), and I(x) are obtained, and J(x) is solved by inverse formula to obtain the dehazed image.

[0033] Furthermore, in one embodiment of the present invention, the optimization objective of the generative network model is:

[0034]

[0035] Where Div represents the difference between the two distributions, and z represents the input noise data, following the distribution P. z The real data x follows a distribution P data The discriminant network model D is used for binary classification of input data, that is, to determine whether the input data belongs to real data or generated data. The optimization objective of the discriminant network model is:

[0036]

[0037] In the formula, V(G,D) is defined as:

[0038]

[0039] Where E represents the expected data distribution, and the optimization objective function of the dehazing recognition network model is: based on the optimization objectives of the generative network model and the discriminative network model.

[0040]

[0041] To achieve the above objectives, another aspect of the present invention proposes an image dehazing and recognition system based on an improved generative adversarial network, comprising:

[0042] The recognition model building module is used to acquire a training dataset containing foggy image samples and corresponding fog-free image samples, and to build a dehazing recognition network model, so as to train the dehazing recognition network model using the training dataset; wherein, the dehazing recognition network model includes a generator network model and a discriminator network model, the generator network model includes a feature extraction network and a feature enhancement network, and the feature enhancement network includes a residual connection network and a multi-head attention network;

[0043] The first feature classification module is used to input the training dataset into the generative network model, and use the feature extraction network to perform a first feature classification operation on the features of the foggy image samples and the corresponding fog-free image samples to obtain a multi-scale feature map; input the multi-scale feature map into the feature enhancement network, and use the residual connection network and the multi-head attention network to perform a second feature classification operation on the multi-scale feature map to obtain a fused feature map; generate a mapping between the atmospheric ambient light image and the scattered atmospheric ambient light image based on the fused feature map; and calculate the transmittance and ambient light value of the atmospheric scattering model according to the feature parameters of the mapping to generate clear fog-free image samples;

[0044] The second feature classification module is used to input the clear, fog-free image sample into the discrimination network model, use a preset stitching method to perform a third feature classification operation on the mapping and the clear, fog-free image sample to obtain a pseudo-foggy image, use a loss function to calculate the loss between the pseudo-foggy image and the foggy image sample, and use the optimization objective function and the loss calculation result to optimize the model parameters of the defogging recognition network model to obtain a trained defogging recognition network model.

[0045] The image dehazing recognition module is used to input a new hazy image to be recognized into the trained dehazing recognition network model to perform image dehazing recognition, so as to obtain the dehazing recognition result of the clear, hazy image corresponding to the new hazy image.

[0046] The image dehazing recognition method and system based on improved generative adversarial networks in this invention can quickly obtain dehazed images, reduce network computation, and improve the network's dehazing speed and image recognition efficiency while ensuring the dehazing effect.

[0047] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0048] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0049] Figure 1 This is a flowchart of an image dehazing and recognition method based on an improved generative adversarial network according to an embodiment of the present invention;

[0050] Figure 2 This is an architecture diagram of an image dehazing method based on an improved generative adversarial network according to an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of the network model according to an embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of the feature enhancement network structure according to an embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of the discrimination network structure according to an embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of the structure of an image dehazing and recognition system based on an improved generative adversarial network according to an embodiment of the present invention. Detailed Implementation

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0057] The image dehazing recognition method and system based on an improved generative adversarial network, according to embodiments of the present invention, are described below with reference to the accompanying drawings.

[0058] Figure 1 This is a flowchart of an image dehazing and recognition method based on an improved generative adversarial network according to an embodiment of the present invention.

[0059] like Figure 1 As shown, this image dehazing and recognition method based on improved generative adversarial networks includes:

[0060] S1, Obtain a training dataset containing foggy image samples and corresponding fog-free image samples, and construct a dehazing recognition network model to train the dehazing recognition network model using the training dataset; wherein, the dehazing recognition network model includes a generator network model and a discriminator network model, the generator network model includes a feature extraction network and a feature enhancement network, and the feature enhancement network includes a residual connection network and a multi-head attention network.

[0061] S2, the training dataset is input into the generative network model to perform a first feature classification operation on the features of foggy image samples and corresponding fog-free image samples using the feature extraction network to obtain a multi-scale feature map; the multi-scale feature map is input into the feature enhancement network to perform a second feature classification operation on the multi-scale feature map using the residual connection network and the multi-head attention network to obtain a fused feature map; based on the fused feature map, a mapping of atmospheric ambient light image and scattering atmospheric ambient light image is generated, and the transmittance and ambient light value of the atmospheric scattering model are calculated according to the feature parameters of the mapping to generate clear fog-free image samples;

[0062] S3. Input clear, fog-free image samples into the discriminative network model. Use a preset stitching method to perform third feature classification on the mapping and clear, fog-free image samples to obtain pseudo-foggy images. Use a loss function to calculate the loss between pseudo-foggy images and foggy image samples. Use the optimization objective function and loss calculation results to optimize the model parameters of the defogging recognition network model to obtain a trained defogging recognition network model.

[0063] S4. Input the new foggy image to be identified into the trained dehazing recognition network model to perform image dehazing recognition, so as to obtain the dehazing recognition result of the clear, fog-free image corresponding to the new foggy image.

[0064] The image dehazing recognition method based on improved generative adversarial networks according to embodiments of the present invention improves the image dehazing recognition accuracy by enhancing the feature extraction capability of the image, increasing global image details, and using a mapping method to perform corresponding mapping between two different images, and directly generating a clear image based on the mapping parameters.

[0065] like Figure 2 and 3The diagram shows the architecture and network structure of the method of this invention, which effectively improves the speed of image dehazing while ensuring dehazing performance. Firstly, this invention is divided into a training phase and a testing phase. In the training phase, a training set is constructed and input into the dehazing network. The dehazing generation network includes feature extraction and feature enhancement structures, learning the feature mapping relationship between foggy and fog-free images. Residual connection structures and multi-head attention structures are added in the feature enhancement phase to perform feature stitching, mapping, convolution, and pooling operations on the entire image, estimating the refractive index of atmospheric light and ambient light values. Finally, the foggy image is restored to a clear image using an image mapping and restoration method, and the clear image is input into the discrimination module for judgment. In the testing phase, the network structure tends to be balanced; the foggy image is directly input into the network to obtain the dehazed image. The dehazing method proposed in this invention can avoid fog residue and loss of detail information during the image dehazing process and can be applied in various environments.

[0066] In one embodiment of the present invention, such as Figure 4 The feature enhancement network shown inputs multi-scale feature maps into a cascaded network to extract initial feature information from the multi-scale feature maps using 5*5 channel-wise convolution; it then extracts multi-scale context information from the initial feature information using 7*7 channel-wise dilated convolution; and finally outputs the fused feature map by performing a 1*1 pointwise convolution operation on the multi-scale context information.

[0067] Specifically, the kernel sizes of the three convolutional layers in the feature extraction network are 7x7x64, 3x3x128, and 3x3x512, with strides of 1, 2, and 2 respectively, and zero padding is selected. The 7x7 kernel expands the network's receptive field to the image, allowing it to better capture detailed information. The two subsequent 3x3 kernels effectively reduce the size of the extracted feature maps, which is beneficial for network training. Cascaded residual network blocks can be used to increase the depth of the generator network, thereby improving the feature extraction capability.

[0068] Understandably, cascaded residual blocks pass information to deeper layers of the network through skip links, enhancing the feature responses of the background and image details and avoiding the loss of image feature information. This allows for a reduction in the number of network parameters and computational load while maintaining the same receptive field, and also enables the network to be deeper, thus improving network efficiency to some extent.

[0069] In one embodiment of the present invention, the feature enhancement network uses 1x1 convolutions to reduce the number of channels in the stitched feature maps.

[0070] In one embodiment of the invention, the generation of the mapping relationship involves two parts: one is a feature extraction network mapping from an image to an ambient light image for estimating pseudo-ambient lighting; the other is constructing a mapping from a given input image to a scattered ambient light image for estimating transmittance. The feature maps are normalized, and the ambient light parameters and refractive index of each image are calculated.

[0071] Furthermore, this invention generates mapping relationships M0 and M1 from the background image to the atmospheric ambient light image and the scattered atmospheric ambient light image. The transmittance map is obtained by solving the relationship between A0 and A1, i.e.:

[0072]

[0073] We know that whether it's a mapping from a hazy image to a hazy image, or a mapping from a hazy image to a hazy image, it can be written as a structure based on solving the transformation matrix and the bias matrix, for example:

[0074] I(x)=J(x)t(x)+A(x)(1-t(x))

[0075]

[0076] J(x) = M0[I(x) + b(x)]

[0077] in:

[0078] To reconstruct a clear original image, we must obtain the atmospheric ambient light and transmittance at the location where the image was created. Therefore, we need to construct a mapping network that generates atmospheric ambient light from a known input image to estimate the generated atmospheric ambient light image A0(x), which can be written in the following form:

[0079]

[0080] To estimate transmittance, we construct a mapping from the same input image N(x) to the scattered ambient light image A1(x) based on the formula above, which has the following form:

[0081]

[0082] Considering the powerful feature extraction capabilities of convolutional neural networks, we implemented both mapping networks using CNN backbone networks. After estimating the ambient light and transmittance, we finally converted the foggy image into a fog-free image according to the formula.

[0083] I(x)=J(x)t(x)+A(1-t(x))

[0084] Where x refers to the coordinates of each pixel in the image, J(x) is a clear, fog-free image, I(x) is the actual foggy image, A is the ambient light value of the location where the image was taken, and t(x) represents the transmittance.

[0085] Furthermore, the generative network model and the discriminative network model learn adversarially, ultimately reaching a Nash equilibrium point where both can achieve optimal performance. The generative network model is primarily used to generate data that meets expectations and minimize the distribution difference between generated and real data. Therefore, the optimization objective of the generative network model is:

[0086]

[0087] Where Div represents the difference between the two distributions, and z represents the input noise data, following the distribution P. z The real data x follows a distribution P data The discriminant network model block D is used for binary classification of the input data, that is, to determine whether the input data belongs to real data or generated data. Therefore, the optimization objective of the discriminant network model is:

[0088]

[0089] In the formula, V(G,D) is defined as:

[0090]

[0091] Where E represents the expected value of the data distribution. Combining the optimization objectives of the generative network model and the discriminative network model, the final optimization objective function of the entire dehazing recognition network model is obtained:

[0092]

[0093] Furthermore, such as Figure 5 As shown, the discriminative network model of this invention mainly uses a binary network to distinguish between the synthesized pseudo-fog image and the real foggy image. The closer the image is to the real foggy image, the closer the estimated ambient light parameters and transmittance are to the true values. The output value of the discriminative network guides the training of the generator network, enabling the generator network to generate more realistic environmental parameters during the learning iteration process, thus improving the network's convergence. Simultaneously, the discriminative network continuously updates its parameters in each iteration, distinguishing obvious features between the generated image and the real foggy image to improve its own discrimination ability. This process iterates until the discriminative network can no longer distinguish the generated image, reaching a balance between the two. The structure of the discriminative network model is as follows: Figure 5As shown, the network consists of three convolutional modules, all of which use ReLU as the activation function. The first two convolutional layers have a kernel size of 3x3 and are used to extract features from the output image of the generative model. Finally, a convolutional layer with a kernel size of 1x1 and a stride of 1 is added as a one-dimensional output, and a Sigmoid activation function is used to determine whether the generated image conforms to the true sample distribution. A color consistency loss is added to the loss function.

[0094]

[0095] Where p represents a pixel, ANGLE is the angle calculation function, which calculates the angular difference between two colors; RGB can be viewed as a three-dimensional vector of color. y represents the hazy image. G B (G A (x)) and G A (G B (y) is the image obtained by stitching the atmospheric light values ​​generated by the generation module with the haze-free image. Calculate G. A (G B By summing the angle between the color vector of each pixel in (y) and the color vector of each pixel in image y, it is possible to effectively determine whether the parameters generated by the generation module are close to the true values.

[0096] Furthermore, the transmittance estimated from the atmospheric ambient light image and the scattered ambient light image generated by the generator through the trained network model is independent of the transmittance of the foggy image sample.

[0097] Furthermore, since the ambient light and transmittance of the atmosphere remain almost constant over a certain period of time, the transmittance and ambient light values ​​can be estimated by the inverse calculation module based on the mapping relationship. Given a foggy image, a clear, fog-free image can then be derived.

[0098] In summary, this invention views a foggy image as a nonlinear composite of a fog-free image and a foggy image. A lightweight generator network generates a mapping from a fixed input to ambient light to estimate the pseudo-ambient light. A generator then generates a mapping from the same input to the scattered ambient light. Finally, an atmospheric scattering model is used to inversely solve the problem and obtain the defogging image, satisfying scenarios requiring real-time defogging. Compared to traditional methods that directly generate defogging images using a generator network, this method can obtain a defogging image by estimating transmittance through simple mathematical relationships, reducing the computational load of the network and improving the defogging speed while maintaining the defogging effect.

[0099] The image dehazing and recognition method based on improved generative adversarial networks described in this invention can be applied to many computer vision systems, such as community monitoring, intelligent transportation, civil aviation assistance, disaster relief, remote sensing observation, and autonomous driving. For example, in community monitoring, the improved generative adversarial network model and image dehazing method described in this invention can effectively process images of people entering and leaving the community during foggy and hazy weather in autumn and winter, avoiding the problem of missing people due to severe weather, thereby effectively improving the monitoring system's ability to cope with severe weather. In intelligent transportation, the improved generative adversarial network model and image dehazing method described in this invention can accurately identify and detect vehicle driving conditions and violations in foggy weather, improving the effectiveness of traffic monitoring systems in foggy weather. In civil aviation assistance, inputting image or video information from aircraft flight into the improved generative adversarial network model and image dehazing method described in this invention can accurately process relevant images in clouds and fog, improving visibility of civil aircraft at high altitudes during landing, thereby enhancing the observation capabilities of civil aircraft in clouds and fog. In post-disaster relief, the improved generative adversarial network model and image dehazing method described in this invention can reconstruct image information from fires, transforming foggy fire images into clear ones, thus improving rescue efficiency. For remote sensing observation, the improved generative adversarial network model and image dehazing method described in this invention can also process fog images caused by atmospheric light refraction, improving the observation efficiency and environmental adaptability of remote sensing systems. For autonomous driving, the improved generative adversarial network model and image dehazing method described in this invention can process road condition information in real-time and accurately, solving the problem of low visibility affecting driver judgment and ensuring safe driving in foggy weather. Therefore, the improved generative adversarial network model and image dehazing method described in this invention have a wide range of applications. By inputting foggy or smoke-containing images into the model described in this invention, improvements can alleviate the problems of detail loss and color alteration during image dehazing, while improving the overall dehazing effect of the image without sacrificing detection speed.

[0100] The image dehazing method based on an improved generative adversarial network according to embodiments of the present invention fuses and concatenates feature maps of different layers, increasing the semantic information capability of the upsampled feature maps. This increases the network depth and enhances its ability to extract global features. Simultaneously, the dilated convolution method used has fewer parameters than ordinary convolution methods, reducing the computational load and increasing the network's running speed. This invention expands the receptive field and enhances the model's ability to extract global detail parameters by designing an enhanced feature fusion branch and incorporating dilated convolution and feature residual modulo operations, thereby improving the efficiency of image dehazing recognition and effectively enhancing the accuracy of image generation and dehazing.

[0101] To achieve the above embodiments, such as Figure 6 As shown, this embodiment also provides an image dehazing and recognition system 10 based on an improved generative adversarial network. The system 10 includes:

[0102] The recognition model construction module 100 is used to acquire a training dataset containing foggy image samples and corresponding fog-free image samples, and to construct a dehazing recognition network model, so as to train the dehazing recognition network model using the training dataset; wherein, the dehazing recognition network model includes a generator network model and a discriminator network model, the generator network model includes a feature extraction network and a feature enhancement network, and the feature enhancement network includes a residual connection network and a multi-head attention network;

[0103] The first feature classification module 200 is used to input the training dataset into the generative network model, so as to use the feature extraction network to perform a first feature classification operation on the features of the foggy image samples and the corresponding fog-free image samples to obtain a multi-scale feature map; input the multi-scale feature map into the feature enhancement network, so as to use the residual connection network and the multi-head attention network to perform a second feature classification operation to fuse the multi-scale feature map to obtain a fused feature map; generate a mapping of atmospheric ambient light image and scattering atmospheric ambient light image based on the fused feature map; and calculate the transmittance and ambient light value of the atmospheric scattering model according to the feature parameters of the mapping to generate clear fog-free image samples.

[0104] The second feature classification module 300 is used to input clear, fog-free image samples into the discrimination network model, use a preset stitching method to perform a third feature classification operation on the mapping and clear, fog-free image samples to obtain pseudo-foggy images, use a loss function to calculate the loss of the pseudo-foggy images and foggy image samples, and use the optimization objective function and loss calculation results to optimize the model parameters of the defogging recognition network model to obtain a trained defogging recognition network model.

[0105] The image dehazing recognition module 400 is used to input a new hazy image to be recognized into a trained dehazing recognition network model to perform image dehazing recognition, so as to obtain the dehazing recognition result of the clear, hazy image corresponding to the new hazy image.

[0106] Furthermore, the feature enhancement network employs a cascaded network; the aforementioned first feature classification module 200 is also used for:

[0107] The multi-scale feature map is input into the cascaded network to extract the initial feature information of the multi-scale feature map using 5*5 channel-wise convolution.

[0108] Multi-scale contextual information of initial feature information is extracted through 7*7 channel-wise dilated convolution;

[0109] A fused feature map is obtained by performing a 1*1 pointwise convolution operation on multi-scale contextual information.

[0110] Furthermore, the stitching method includes a pixel-to-pixel synthesis and addition method; the discriminant network model includes multiple convolutional layers, wherein each convolutional layer uses ReLU as the activation function; and a color consistency loss is added to the loss function, wherein the loss function is:

[0111]

[0112] Where p represents a pixel, ANGLE is the angle calculation function, y is the foggy image, and G... B (G A (x)) and G A (G B (y) is the generated pseudo-hazy image.

[0113] Furthermore, the second feature classification module 300 is also used for:

[0114] Formula for calculating atmospheric light scattering:

[0115]

[0116] I(x)=J(x)t(x)+A(x)(1-t(x))

[0117] Where A0 is the ambient light, A1 is the diffused ambient light, M0 and M1 are the mapping relationship matrices generated by the generation module from the background image to the ambient light image and the diffused ambient light image, x refers to the coordinates of each pixel point in the image, J(x) is the clear, fog-free image, I(x) is the actual foggy image, A is the ambient light value of the image shooting location, and t(x) represents the ambient light transmittance.

[0118]

[0119] J(x) = M0[I(x) + b(x)]

[0120] in:

[0121] The image dehazing recognition system based on an improved generative adversarial network according to an embodiment of the present invention fuses and concatenates feature maps of different layers, increasing the semantic information capability of the upsampled feature maps. This method increases the network depth on the one hand, and enhances the network's ability to extract global features on the other. Simultaneously, the dilated convolution method used has fewer parameters than ordinary convolution methods, reducing the computational load of the network and increasing its running speed. This invention expands the receptive field and enhances the model's ability to extract global detailed parameter features by designing and adding a feature fusion enhancement branch, and incorporating dilated convolution and feature residual modulo operations, thereby effectively improving the accuracy of image dehazing recognition.

[0122] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. An image defogging identification method based on an improved generative adversarial network, characterized in that, The method comprises the following steps: obtaining a training data set containing fog image samples and corresponding non-fog image samples, and constructing a dehazing recognition network model to train the dehazing recognition network model by using the training data set; wherein the dehazing recognition network model comprises a generation network model and a discrimination network model, the generation network model comprises a feature extraction network and a feature enhancement network, and the feature enhancement network comprises a residual connection network and a multi-head attention network; inputting the training data set into the generation network model to perform a first feature classification operation on the features of the fog image samples and the corresponding non-fog image samples by using the feature extraction network to obtain multi-scale feature maps; inputting the multi-scale feature maps into the feature enhancement network to perform a second feature classification operation on the multi-scale feature maps by using the residual connection network and the multi-head attention network to obtain fused feature maps, generating a mapping of an atmospheric environment light image and a scattered atmospheric environment light image based on the fused feature maps, and calculating the transmittance and the environmental light value of the atmospheric scattering model according to the feature parameters of the mapping to generate a clear non-fog image sample; inputting the clear non-fog image sample into the discrimination network model, performing a third feature classification operation on the mapping and the clear non-fog image sample by using a preset splicing method to obtain a pseudo-fog image, performing loss calculation on the pseudo-fog image and the fog image sample by using a loss function, and optimizing the model parameters of the dehazing recognition network model by using an optimization objective function and the loss calculation result to obtain a trained dehazing recognition network model; inputting a new fog image to be recognized into the trained dehazing recognition network model for image dehazing recognition to obtain a dehazing recognition result of a clear non-fog image corresponding to the new fog image; the feature enhancement network adopts a cascade network; the inputting of the multi-scale feature maps into the feature enhancement network to perform the second feature classification operation on the multi-scale feature maps by using the residual connection network and the multi-head attention network to obtain the fused feature maps comprises: inputting the multi-scale feature map into the cascade network to extract initial feature information of the multi-scale feature map by using 5 5 the channel-by-channel convolution By 7 the multi-scale context information of the initial feature information is extracted by the channel-wise hole convolution of 7; Utilizing 1 The point-by-point convolution of 1 convolves the multi-scale context information to output the fusion feature map; the splicing method comprises a pixel-to-pixel synthesis addition method; the discrimination network model comprises a plurality of convolution layers, wherein each convolution layer uses ReLu as an activation function; a color consistency loss is added in the loss function, and the loss function is: wherein, represents a pixel, is an angle computation function, is a foggy image, and is the generated pseudo-foggy image; the calculation of the transmittance and the environmental light value of the atmospheric scattering model according to the feature parameters of the mapping to generate the clear non-fog image sample comprises: the calculation formula of the atmospheric light scattering is: wherein A0 is the atmospheric environment light, A1 is the scattered atmospheric environment light, M0 and M1 are mapping relationship matrices generated by a background image to an atmospheric environment light image and a scattered atmospheric environment light image generated by a generation module, x refers to the coordinates of each pixel point of an image, J(x) is a clear non-fog image, I(x) is an actual obtained fog image, A is the environmental light value of an image shooting place, and t(x) represents the environmental light transmittance. wherein: ; The generated mapping includes a first mapping for estimating a pseudo ambient light value and a second mapping for estimating transmittance, the first mapping being a mapping of the fused feature map to the atmospheric ambient light image, comprising: constructing a mapping of the fused feature map to the input image N(x) to generate the atmospheric ambient light, to estimate the atmospheric ambient light image A0(x): The second mapping is a mapping of the fused feature map to the scattered atmospheric ambient light image, comprising: a mapping of the same input image N(x) based on the fused feature map to the scattered atmospheric ambient light image A1(x): According to the estimated A0 and t(x), I(x), the formula is solved to obtain J(x), and a defogging image is obtained.

2. The method of claim 1, wherein, The optimization target of the generation network model is: wherein, represents the difference between two distributions, represents the input noise data, following a distribution , real data following a distribution The discriminative network model D is used to binary classify the input data, i.e. to discriminate whether the input data belongs to real data or generated data, then the optimization objective of the discriminative network model is: In the formulae is defined as: Wherein E represents the data distribution expectation, the optimization target function of the defogging recognition network model based on the optimization target of the generation network model and the discrimination network model: 。 3. An image defogging recognition system based on improved generative adversarial network, characterized in that, Including: The recognition model construction module is used to obtain a training data set containing fog image samples and corresponding fog-free image samples, and to construct a defogging recognition network model, so as to use the training data set to train the defogging recognition network model; wherein the defogging recognition network model includes a generation network model and a discrimination network model, the generation network model includes a feature extraction network and a feature enhancement network, and the feature enhancement network includes a residual connection network and a multi-head attention network; The first feature classification module is used to input the training data set into the generation network model, to perform a first feature classification operation on the features of the fog image samples and the corresponding fog-free image samples by using the feature extraction network to obtain a multi-scale feature map; the multi-scale feature map is input into the feature enhancement network, and a second feature classification operation is performed on the multi-scale feature map by using the residual connection network and the multi-head attention network to obtain a fused feature map, a mapping of atmospheric ambient light images and scattered atmospheric ambient light images is generated based on the fused feature map, and the transmittance and ambient light value of the atmospheric scattering model are calculated according to the feature parameters of the mapping to generate clear fog-free image samples; The second feature classification module is used to input the clear fog-free image sample into the discrimination network model, perform a third feature classification operation on the mapping and the clear fog-free image sample by using a preset splicing method to obtain a pseudo fog image, perform loss calculation on the pseudo fog image and the fog image sample by using a loss function, and optimize the model parameters of the defogging recognition network model by using the optimization target function and the loss calculation result to obtain a trained defogging recognition network model; The image defogging recognition module is used to input a new fog image to be recognized into the trained defogging recognition network model for image defogging recognition, to obtain a defogging recognition result of a clear fog-free image corresponding to the new fog image. The feature enhancement network adopts a cascade network; the first feature classification module is also used for: inputting the multi-scale feature map into the cascade network to extract initial feature information of the multi-scale feature map by using 5 5 By 7 the multi-scale context information of the initial feature information is extracted by the channel-wise hole convolution of 7; Utilizing 1 The point-by-point convolution of 1 convolves the multi-scale context information to output the fusion feature map; The splicing method comprises a pixel-to-pixel synthesis addition method; the discrimination network model comprises a plurality of convolution layers, wherein each convolution layer uses ReLu as an activation function; a color consistency loss is added to the loss function, and the loss function is as follows: wherein, represents a pixel, is an angle computation function, is a foggy image, and is the generated pseudo-foggy image; The second feature classification module is further configured to: The calculation formula of atmospheric light scattering is: Wherein, A0 is the atmospheric environment light, A1 is the scattered atmospheric environment light, M0, M1 are the mapping relationship matrices generated by the background image to the atmospheric environment light image, the scattered atmospheric environment light image generated by the generation module, x refers to each pixel point coordinate of the image, J(x) is the clear and fog-free image, I(x) is the actual obtained foggy image, A is the ambient light value of the image shooting place, and t(x) represents the ambient light transmittance; wherein: .

Images