Video monitoring image quality enhancement method under bad weather condition

Abstract

The invention relates to a video monitoring image quality enhancement method under bad weather conditions, comprising the following steps: constructing a scatter rendering model based on a generative adversarial network, and then training and verifying the model; the scattered scene rendering model comprises a discriminator, a residual dense network, a feature fusion module and a generator; the method comprises the following steps: firstly, judging whether an initial image input into a bokeh rendering model is a real sample or a generated image by a discriminator, then inputting the initial image into a residual dense network, extracting feature information of the input image step by step, strengthening foreground target features to obtain a plurality of feature maps, fusing the plurality of feature maps to generate a feature fusion map, and finally, carrying out image fusion on the feature fusion map. The generator adds random noise into the feature fusion image and outputs a generated image; constructing an image denoising and restoring model based on a Unet model, and then training and verifying the model; and inputting the generated image into an image denoising restoration model, and outputting a clear restored image. The method has the advantages that foreground target information can be enhanced, and the authenticity and accuracy of image restoration are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence image processing, and in particular to a method for enhancing the quality of video surveillance images under adverse weather conditions. Background Art

[0002] Digital safety management and control of power operation sites mainly uses video surveillance equipment to collect on-site images, extract frames from the video images, and then use artificial intelligence algorithm models to perform recognition and analysis. Since there are many outdoor work sites in power construction, in adverse weather conditions such as rain, fog and snow, the video images collected by on-site monitoring equipment are of poor quality and blurry, which seriously affects the effect of video monitoring, causing the accuracy of identifying risky behaviors based on visual recognition technology to drop sharply, becoming an important obstacle to the viewing experience of video content and the accuracy of identifying targets and abnormal behaviors, and bringing great inconvenience to remote supervision and digital safety management and control. How to enhance the quality of video surveillance images in adverse weather and lighting conditions and improve the recognition of on-site video images is an issue that needs to be considered urgently.

[0003] The key difficulty of video surveillance image quality enhancement technology is how to solve the problem of maintaining video surveillance image quality under asymmetric encoding and decoding conditions. Asymmetric encoding and decoding of video images refers to the uneven distribution of processing power and time during compression and decompression. A lot of processing power and time are required for compression, while decompression can be played back in real time better, that is, compression and decompression are performed at different speeds. In this case, video data damage and imaging quality problems are likely to occur. Therefore, in the process of video compression and decompression, it is necessary to adopt different algorithm strategies or technical parameters to optimize encoding efficiency and decoding quality, solve the problem of video data damage and imaging quality during the encoding and decoding process, and improve the quality of video surveillance at power operation sites under bad weather conditions. Summary of the invention

[0004] In order to solve the above problems, the purpose of the present invention is to provide a method for enhancing the quality of video surveillance images under adverse weather conditions, which can enhance foreground target information, improve image quality, and ensure the authenticity and accuracy of image restoration.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] A method for enhancing the quality of video surveillance images under adverse weather conditions comprises the following steps: constructing a bokeh rendering model based on a generative adversarial network, and then training and verifying the model; the bokeh rendering model comprises a discriminator, a residual dense network, a feature fusion module and a generator; the discriminator first determines whether the initial image input into the bokeh rendering model is a real sample or a generated image output by the generator, the initial image is input into the residual dense network, feature information of the input image is extracted step by step, foreground target features are enhanced, and multiple feature maps are obtained, and then the feature fusion module fuses the multiple feature maps and inputs them into the generator; the generator takes random noise as input and outputs a generated image; constructing an image denoising and restoration model based on a Unet model, and then training and verifying the model; the generated image is input into the image denoising and restoration model, and a clear restored image is output.

[0007] Preferably, the discriminator adopts a downsampling method, and its network structure has five layers, including four convolutional layers and one fully connected layer, and the convolution results of each convolutional layer are normalized and input into the next layer. The fully connected layer serves as the output layer to output the discrimination result. The results of each convolutional layer are converged using the LeakyReLU() activation function, and the fully connected layer uses the sigmoid() activation function to output probability, indicating the authenticity of the initial image.

[0008] More preferably, the residual dense network includes multiple improved incomplete modules connected in series, each incomplete module includes multiple convolutional layers, each convolutional layer adopts a dense connection structure, and each convolutional layer is connected to all subsequent convolutional layers in a feedforward manner, so that the layers are closely connected and the features of the previous convolutional layer are fully utilized, the transmission of inter-layer features is enhanced, and multiple feature maps are output.

[0009] Preferably, the feature fusion module first performs global average pooling on the input feature maps of each layer, then learns the correlation between channels through the fully connected layer, and then obtains the weights of the corresponding channels of different layer features through splicing slices and softmax function. Finally, the feature weights of each layer are multiplied by the feature map of the corresponding layer of the input feature fusion module, and the results are superimposed and fused to obtain the feature fusion map.

[0010] Preferably, the generator generates feature images of different sizes through an upsampling method, inputs a feature fusion map and noise, sets a corresponding convolution kernel according to the required feature map size, extracts the feature information of the input image step by step, and then continuously optimizes the details of the image according to the depth of the network, and outputs the generated image.

[0011] Preferably, the image denoising and restoration model is to add a noise removal network module DNB to the Unet model. Specifically, the Unet model is a fully convolutional network FCN, which consists of an encoder and a decoder, which are connected by a jump connection in the middle; the input generated image is compressed using an encoder consisting of a convolutional layer and a pooling layer to form an implicit data representation, and then the image is enlarged using a decoder consisting of a convolutional layer and a transposed convolutional layer, and finally the restored image is output; in the encoder and decoder, multiple noise removal network modules DNB are introduced, and the noise removal network module DNB includes three parts, the first part is the first layer, which consists of a 3*3 convolutional layer and a Relu activation function, the second part is the 2nd to 16th layers, a total of 15 Unit block units, and the third part consists of a 3*3 convolutional layer.

[0012] The present invention has the following beneficial effects:

[0013] 1. The bokeh rendering technology based on generative adversarial network (BGGAN) of the present invention is implemented through an end-to-end convolutional neural network generator and a multi-receptive field discriminator. The generator is responsible for outputting the image after bokeh rendering, while the discriminator supervises the difference between the generated image and the real bokeh image. In this way, the foreground target information of the image taken by the control ball in the power grid application scenario is enhanced, and the image quality is improved. At the same time, during the training process, PatchGAN discriminators of different depths are used to form a multi-receptive field discriminator to focus on details of different sizes of the image, further improving the naturalness and realism of the bokeh effect.

[0014] 2. The image enhancement technology based on the denoising concept of the present invention adopts the improved UNet architecture for image restoration, aiming to restore high-quality images from low-quality input images. It is particularly suitable for images that are greatly disturbed by environmental factors. It extracts multi-scale features through the encoder-decoder structure, and reconstructs image details using jump connections, achieving efficient noise removal and image quality improvement. In addition, the introduction of the noise removal network block (DNB) and the attention mechanism further enhances the model's ability to identify and remove noise, ensuring the authenticity and accuracy of image restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of the image quality enhancement process of the present invention;

[0016] Figure 2 A schematic diagram of a generative adversarial network model for the present invention;

[0017] Figure 3 It is the code structure diagram of the generator of the present invention;

[0018] Figure 4 It is a schematic diagram of the dense connection structure of the present invention;

[0019] Figure 5 It is a schematic diagram of the feature fusion process of the present invention;

[0020] Figure 6 It is a schematic diagram of the architecture of the Unet model of the present invention;

[0021] Figure 7 It is a schematic diagram of the framework of the denoising algorithm of the present invention;

[0022] Figure 8 It is a Unitblock structure diagram of the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0024] A method for enhancing the quality of video surveillance images under adverse weather conditions comprises the following steps:

[0025] Step 10: Please refer to Figure 1 and Figure 2 , build a bokeh rendering model based on a generative adversarial network, and then train and validate the model.

[0026] The bokeh rendering model includes a discriminator, a residual dense network, a feature fusion module and a generator; first, the discriminator determines whether the initial image input into the bokeh rendering model is a real sample or a generated image output by the generator; secondly, the initial image is input into the residual dense network, and the feature information of the input image is extracted step by step to obtain multiple feature maps; then the feature fusion module fuses the multiple feature maps and inputs them into the generator; the generator takes random noise as input and outputs the generated image.

[0027] The discriminator adopts the downsampling method. The network structure has five layers. The entire network structure has no pooling layer, including four convolutional layers and one fully connected layer. The convolution results of each convolutional layer are normalized and input into the next layer. The fully connected layer serves as the output layer and outputs the discrimination result, ranging from 0 to 1, indicating whether the input image belongs to a real sample or an image generated by the generator. The results of each convolutional layer are converged using the LeakyReLU() activation function, and the fully connected layer uses the sigmoid() activation function to output the probability, indicating the authenticity of the initial image. A BN layer is also added to the discriminator, and each layer uses an activation function for nonlinear processing. The specific hierarchical structure is shown in Table 1 DCGAN discriminant model hierarchy:

[0028] Table 1:

[0029]

[0030]

[0031] For Table 1, the specific description is as follows:

[0032] Layer1: Input layer. First input the image, set the Stride to 1, and set the Padding to 0. Use the Conv2d() function to perform convolution, and then use the LeakyReLU() activation function to optimize the network convergence speed.

[0033] Layer2: Input image, set Stride to 2, set Padding to 1, add BN layer to batch normalize the data, and use LeakyReLU() activation function to accelerate convergence;

[0034] Layer3: Input image, set Stride to 2, set Padding to 1, add BN layer to batch normalize the data, and use LeakyReLU() activation function to accelerate convergence;

[0035] Layer4: Input image, set Stride to 2, set Padding to 1, add BN layer to batch normalize the data, and use LeakyReLU() activation function to accelerate convergence;

[0036] Layer5: Output layer, Stride is set to 3, Padding is set to 1, and the sigmoid() function is used to output the probability to indicate the authenticity of the image.

[0037] The generator generates feature images of different sizes through upsampling method. The network structure also has five layers in total, and the entire network structure of the generator has no pooling layer. Input feature fusion map and a noise of nz dimension that obeys uniform distribution, set the corresponding convolution kernel according to the required feature map size, the size of the convolution kernel affects the size of the learnable spatial feature value of the generated network, and then extract the feature information of the input image step by step, such as lines, style, etc., and then continuously optimize the details of the picture according to the depth of the network, and output the generated image.

[0038] In the code, the nn.Sequential() function can be used to add convolution, activation, etc. to the module in order according to the order passed in. After the network layer is built, the forward method is automatically called. In each layer except the output layer, batch normalization (BN) processing is added to alleviate the model collapse problem. According to the requirements of each layer of the network structure, the corresponding activation function is used, and finally, a 3-channel RGB image of a pixel is output. The generator code structure diagram is as follows Figure 3 As shown, the specific hierarchical structure is shown in Table 2 DCGAN generation model hierarchy:

[0039] Table 2:

[0040]

[0041] For Table 2, the specific description is as follows:

[0042] Layer1 is the input layer, which inputs random noise, sets Stride to 1, and Padding to 0. Next, the ConvTranspose2d() function is used to perform inverse convolution, and the data is batch normalized in the BN layer, and the ReLU activation function is used to accelerate convergence, and the corresponding feature map is output;

[0043] Layer2: Set Stride to 2, Padding to 1, add BN layer to batch normalize the data, use ReLU activation function to accelerate convergence, and output the corresponding feature map;

[0044] Layer3: Set Stride to 2, Padding to 1, add BN layer to batch normalize the data, use ReLU activation function to accelerate convergence, and output the corresponding feature map;

[0045] Layer4: Set Stride to 2, Padding to 1, add BN layer to batch normalize the data, use ReLU activation function to accelerate convergence, and output the corresponding feature map;

[0046] Layer5: Output layer, set Stride to 3, Padding to 1, use Tanh as the activation function, and output the corresponding image.

[0047] In summary, it can be seen that the settings of Padding, Stride, etc. of the two network models of the generator and the discriminator are almost the same. The overall network structure is symmetrical, but the activation functions used in each layer are different.

[0048] It should be noted that the generator and discriminator are trained and optimized alternately.

[0049] For the discriminator, it classifies real data and fake data generated by the generator. The loss function of the discriminator will punish the misjudgment made by the discriminator, such as judging real instances as fake, or judging fake instances as real; the discriminator back-propagates the loss calculated from the discriminator network loss function, and uses the error back-propagation mechanism to calculate the loss and update the weight parameters; when the discriminator is trained, the generator will not be trained, that is, when the generator generates sample data for the discriminator, the weight of the generator remains constant. In pytorch, the gradient of the generator is not calculated by using detach().

[0050] For the generator, random noise samples are used as input; the generator generates output from the sampled random noise samples, and the discriminator determines whether the above output is "true" or "false" as the output of the generator; the error loss is calculated from the classification output of the discriminator; the gradient is obtained through backpropagation of the discriminator and the generator; the gradient is used to update the weights of the generator.

[0051] Alternating training process: During the phase of discriminator training, keep the generator unchanged. Because the discriminator training tries to distinguish real data from fake data, the discriminator must learn how to recognize the generator's flaws. This is the difference between a fully trained generator and an untrained generator that can only produce random outputs. Similarly, during the phase of generator training, keep the discriminator unchanged. Otherwise, the generator may never converge, just like trying to hit a moving target.

[0052] In order to better retain and utilize the features extracted from each layer, dense connections and feature fusion are introduced on the basis of residual networks, and a residual dense network image super-resolution reconstruction algorithm is further proposed. Dense connections are used in the feature extraction part of the network to strengthen the feature transfer between layers. The proposed feature fusion module merges inter-layer features to reduce feature dimensions and achieve efficient and full utilization of feature information at each layer.

[0053] The residual dense network includes multiple serially connected improved residual modules, such as Figure 4 As shown in , each incomplete module includes multiple convolutional layers, each of which adopts a densely connected structure. Each convolutional layer is connected to all subsequent convolutional layers in a feedforward manner, so that the inter-layer connection is close and the features of the previous convolutional layer are fully utilized, the inter-layer feature transfer is enhanced, and multiple feature maps are output. Figure 4 As shown in the densely connected structure, i The input of the layer includes the feature maps F output by all previous layers 0 , F 1 , …, F i-1 , the feature map F output by this layer i It will also become the input of all subsequent layers. The layer contains a convolutional layer and a nonlinear activation function. i The feature map F of the layer output i It can be expressed as: fi = g(wi*[F 0 , F 1 , …, F i-1 ]+b i );Where wi and b i Represent the weight and bias term of the i-th layer respectively, [F 0 , F 1 , …, F i-1] represents the concatenation of feature maps, and g is the activation function. Dense connections connect the features of all convolutional layers in series, where the feature maps of each layer are passed to all subsequent layers, and the reconstruction performance is improved by retaining and utilizing the features of each layer without information loss. This not only improves the vanishing gradient of the model, but also strengthens the transmission of features between layers. Dense connections achieve feature reuse by concatenating features on the channel, which makes it have a better reconstruction effect than the residual network.

[0054] In the residual dense network, the information in the feature maps output by different convolutional layers complements each other to obtain richer and more effective inter-layer feature information. In addition, dense connections splice the feature maps of each layer together as the input of the subsequent upsampling part. If these feature maps are directly sent to the upsampling part, the amount of calculation will increase significantly. In order to more efficiently and fully utilize the feature information transmitted by dense connections, a feature fusion module is proposed.

[0055] See also Figure 5 ,First, the feature fusion module first performs global average pooling on the ,feature maps of each layer of the input, which reduces the feature dimension and ,enhances the receptive field, and removes redundant information to improve the ,computational efficiency of the network.

[0056] Next, the correlation between channels is learned through the fully connected layer. Assuming the number of feature maps of the input feature fusion module is N, the number of channels of the feature map is C, the output of the fully connected layer is expressed as follows:

[0057] Yi=a i X i ;

[0058] Where i = 1, 2, .., N, X i =[x i1 , x i2 , …, x iC ],Yi=[y i1 ,y i2 , …, y iC ], X i Represents the i-th feature map M of the input feature fusion module i The global feature obtained by global average pooling, a i Represents X i The corresponding fully connected layer weights.

[0059] After that, the weights of the corresponding channels of different layer features are obtained by splicing slices and softmax function. The weights of each channel of different layer features are expressed as follows:

[0060] W j =F S (Y j )

[0061] Where j = 1, 2, ..., C, Y j =[y 1j ,y 2j , …, y Nj ],W j =[w 1j , w 2j ,…,w Nj ], F S Represents the softmax function.

[0062] Finally, the feature weights of each layer are multiplied by the feature map of the corresponding layer of the input feature fusion module, and the results are superimposed and fused to obtain the feature fusion map. The output of the feature fusion module R = [r 1 , r 2 ,…,r c ] consists of the following:

[0063]

[0064] In the formula, m ij is the i-th feature map M of the input feature fusion module i The jth channel of .

[0065] The feature fusion module combines inter-layer features to reduce feature dimensions, efficiently and fully utilizes the feature information of each layer, and helps to improve network computing efficiency and super-resolution reconstruction effects.

[0066] Step 20: construct an image denoising and restoration model based on the Unet model, then train and verify the model, input the generated image into the image denoising and restoration model, and output a clear restored image.

[0067] The image denoising restoration model is to add a noise removal network module DNB to the Unet model. Figure 6 The Unet model is a fully convolutional network FCN, which consists of an encoder and a decoder, which are connected by a jump connection in the middle; the input generated image is compressed using an encoder composed of a convolution layer and a pooling layer to form an implicit data representation, and then the image is enlarged using a decoder composed of a convolution layer and a transposed convolution layer, and finally the restored image is output.

[0068] The main task of the encoder is to gradually compress the spatial resolution of the input image and extract higher-level features. This part contains a series of convolutional layers and max pooling layers, and each pooling operation reduces the spatial dimension of the image by half. The task of the decoder is to map the high-level features extracted by the encoder part back to the original image resolution by gradually restoring the spatial resolution of the image. The decoder contains deconvolution (upsampling) operations and combines the corresponding feature layers from the encoder to achieve fine boundary recovery. The output feature map of each encoder layer is spliced ​​with the feature map of the corresponding layer in the decoder to form a jump connection, which can fuse the local information in the encoder and the global information in the decoder, thereby improving the accuracy of the segmentation result.

[0069] In the encoder and decoder, multiple noise removal network modules DNB are introduced, which focus on noise identification and removal by including layer normalization, convolution operations of various sizes, attention mechanism and other components. Through fine-grained feature processing, DNB can effectively reduce image noise while retaining important image details.

[0070] Layer normalization: It normalizes all neuron activation values ​​in each layer of the neural network, converts the activation values ​​of each layer into a distribution with a mean of 0 and a standard deviation of 1, and then scales and offsets the result. Given an input vector Z of a layer in a neural network = (Z 1 , Z 2 , …, Z H ), where H is the number of neurons in this layer, and the calculation formula is as follows:

[0071]

[0072] Where μ is the mean of all neuron activation values ​​in this layer, and o is the standard deviation of all neuron activation values ​​in this layer.

[0073]

[0074] Where ∈ is a small positive number used to prevent division by zero errors. After normalization, each normalized activation value Zi is scaled and offset to maintain the expressiveness of the model. This process is accomplished by introducing learnable parameters γ and β:

[0075] Y i =γZ i +β;

[0076] In this way, the input of each layer can be ensured to remain stable during the training process, which helps to improve the training efficiency and stability of the model.

[0077] Convolutions of various sizes: Advanced denoising algorithm frameworks in deep learning such as Figure 7 As shown in the figure, it can be mainly divided into three parts. The first part is the first layer composed of a convolutional layer 3×3×64 and a Relu activation function; the second part is composed of 15 Unit blocks from the 2nd to the 16th layers. Figure 8 As shown in Figure 1, it consists of a convolutional layer of 3×3×64, a BN layer and an activation function Relu; the third part consists of a convolutional layer of 3×3. The entire network makes a global jump from the input to the output, forming a residual learning strategy. The entire network description can be expressed as:

[0078] G(x,y)=O(x,y)+b(x,y);

[0079] Where: G(x, y) is a two-dimensional noise image; O(x, y) is the original clean image without noise; b(x, y) represents the additional noise. The noise image is a degradation result of the original image under the superposition of additive noise. Therefore, the purpose of image denoising is to obtain a ^O(x, y), ^O(x, y) is an estimate of the original image. The smaller the difference between O(x, y) and ^O(x, y), the better the image denoising effect.

[0080] In order to further improve the quality of image restoration, this solution can also integrate attention mechanisms into DNB, such as split attention and jump attention modules. These attention modules can help the network focus more on the key areas of the image and optimize the information flow in the image reconstruction process, thereby retaining more details and texture information while denoising.

[0081] Step 1: preprocess the training data set, use Gaussian white noise to simulate real noise and add it to the original image to obtain the corresponding noise image x_noise_img;

[0082] Step 2, input the noise image xnoise_img into the denoising network, and the output is the noise distribution image xno1se;

[0083] Step 3, subtract the noise distribution map xnoise from the noise image x_noise_img to obtain the denoising result x_dn using the denoising network;

[0084] Step 4, input the noise image xnoise_img into the attention mechanism network, and the output is the noise weight image noise mask;

[0085] Step 5, combining the noise distribution image xnoise outputted in step 2 and the noise weight image noise_mask outputted in step 4, the output obtained is the noise residual image noise;

[0086] Step 6, subtract the noise residual image noise from the noise image xnoise_img to obtain the denoised clean image xmask dn;

[0087] Step 7: According to the xdn obtained in step 3 and the xmaskdn obtained in step 6, the mean square error MSE function is used as the loss function to continuously learn the mapping relationship between the noisy image and the clean image, and adjust the network model parameters until the network converges.

[0088] This invention innovatively proposes a bokeh rendering technology based on a generative adversarial network (BGGAN) to achieve image quality processing through an end-to-end convolutional neural network generator and a multi-receptive field discriminator, solving the problem of maintaining video surveillance quality under asymmetric encoding and decoding processing conditions. At the same time, the image enhancement technology based on the idea of ​​image denoising uses the UNet architecture and its improvements to perform image restoration methods, solving the problem of low efficiency in noise removal and image quality improvement.

[0089] The above description is only a specific implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structural transformation made by using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for enhancing the quality of video surveillance images under adverse weather conditions, characterized in that: The steps include: A bokeh rendering model based on a generative adversarial network is constructed, and then the model is trained and verified; the bokeh rendering model includes a discriminator, a residual dense network, a feature fusion module and a generator; the initial image input into the bokeh rendering model is first judged by the discriminator whether the initial image is a real sample or a generated image output by the generator, and then the initial image is input into the residual dense network, feature information of the input image is extracted step by step, foreground target features are enhanced, and multiple feature maps are obtained, and the feature fusion module fuses the multiple feature maps to generate a feature fusion map, which is then input into the generator, and the generator adds random noise to the feature fusion map and outputs the generated image; Construct an image denoising and restoration model based on the Unet model, then train and verify the model; input the generated image into the image denoising and restoration model, and output a clear restored image.

2. The method for enhancing the quality of video surveillance images under adverse weather conditions according to claim 1, characterized in that: The discriminator adopts a downsampling method, and its network structure has five layers, including four convolutional layers and one fully connected layer. The convolution results of each convolutional layer are normalized and input into the next layer. The fully connected layer serves as the output layer to output the discrimination results. The results of each convolutional layer are converged using the LeakyReLU() activation function, and the fully connected layer uses the sigmoid() activation function to output probability, indicating the authenticity of the initial image.

3. The method for enhancing the quality of video surveillance images under adverse weather conditions according to claim 1, characterized in that: The residual dense network includes multiple improved incomplete modules connected in series, each incomplete module includes multiple convolutional layers, each convolutional layer adopts a dense connection structure, and each convolutional layer is connected to all subsequent convolutional layers in a feedforward manner, so that the layers are closely connected and the features of the previous convolutional layers are fully utilized, the transmission of inter-layer features is enhanced, and multiple feature maps are output.

4. The method for enhancing the quality of video surveillance images under adverse weather conditions according to claim 1, characterized in that: The feature fusion module first performs global average pooling on the input feature maps of each layer, then learns the correlation between channels through the fully connected layer, and then obtains the weights of the corresponding channels of different layer features through splicing slices and softmax function. Finally, the feature weights of each layer are multiplied by the feature map of the corresponding layer of the input feature fusion module, and the results are superimposed and fused to obtain the feature fusion map.

5. According to the method for enhancing the quality of video surveillance images under adverse weather conditions as described in claim 4, it is characterized by: The generator generates feature images of different sizes through an upsampling method, inputs a feature fusion map and noise, sets the corresponding convolution kernel according to the required feature map size, extracts the feature information of the input image step by step, and then continuously optimizes the details of the image according to the depth of the network, and outputs the generated image.

6. The method for enhancing the quality of video surveillance images under adverse weather conditions according to claim 1, characterized in that: The image denoising and restoration model is to add a noise removal network module DNB to the Unet model. Specifically, the Unet model is a fully convolutional network FCN, which is composed of an encoder and a decoder, which are connected by a jump connection in the middle; the input generated image is compressed using an encoder composed of a convolution layer and a pooling layer to form an implicit data representation, and then the image is enlarged using a decoder composed of a convolution layer and a transposed convolution layer, and finally the restored image is output; in the encoder and decoder, multiple noise removal network modules DNB are introduced, and the noise removal network module DNB includes three parts, the first part is the first layer, which is composed of a 3*3 convolution layer and a Relu activation function, the second part is the 2nd to 16th layers, a total of 15 Unit block units, and the third part is composed of a 3*3 convolution layer.

Images