Image defogging method and system based on red channel guidance

Abstract

The present invention relates to an image defogging method and system based on red channel guidance, the method comprising: constructing a sample data set; constructing a defogging model; the defogging model comprises a red channel guiding module and a main body module; the red channel guiding module is used to obtain an R channel image of a to-be-processed RGB fog image, extract image features of the R channel image, and send the image features to the main body module; the main body module is used to perform a defogging operation on the to-be-processed RGB fog image according to the image features; the defogging model is trained using the sample data set to obtain a trained defogging model; and a real-shot fog image is defogged using the trained defogging model. The present invention guides the main body module to perform image defogging through the red channel guiding module, which can retain more image detail information, so that the defogging effect of the present invention is better.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image defogging method and system based on red channel guidance. Background Art

[0002] Since visibility is low in foggy weather, pictures taken in foggy weather are not clear. In order to obtain clear pictures in foggy weather, pictures taken in foggy weather need to be defogged.

[0003] At present, image defogging is performed based on the traditional atmospheric scattering model through optimization algorithms or convolutional neural networks. However, there is a certain difference between the fog generated by the traditional atmospheric scattering model and the real fog, which makes the convolutional neural network trained with the dataset synthesized by the traditional atmospheric scattering model have poor defogging effect on the real foggy images. Summary of the invention

[0004] In view of this, a method and system for image defogging based on red channel guidance are provided to solve the problem of poor defogging effect existing in the related art.

[0005] The present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides an image defogging method based on red channel guidance, comprising:

[0007] Construct a sample dataset;

[0008] Constructing a defogging model; the defogging model includes a red channel guiding module and a main body module; the red channel guiding module is used to obtain an R channel image of an RGB fog image to be processed, extract image features of the R channel image, and send the image features to the main body module; the main body module is used to perform a defogging operation on the RGB fog image to be processed according to the image features;

[0009] Using the sample data set to train the defogging model to obtain a trained defogging model;

[0010] The trained defogging model is used to defog the real foggy image.

[0011] Preferably, the red channel guiding module comprises 1 first input layer, 1 scale transformation convolution layer, 4 hole convolution layers, 2 convolution layers, 1 connection layer and 1 normalization layer;

[0012] The connection method of each layer is: first input layer → scale transformation convolution layer → dilated convolution layer 1 → dilated convolution layer 2 → dilated convolution layer 3 → dilated convolution layer 4 → connection layer → convolution layer 1 → convolution layer 2 → normalization layer;

[0013] After the first input layer obtains the R channel map, it extracts a first sub-feature map from the R channel map, and sends the first sub-feature map to the scale transformation convolution layer;

[0014] The scale transformation convolution layer performs scale transformation processing on the first sub-feature map to obtain first sub-feature maps at different scales, and sends the first sub-feature maps at different scales to the hole convolution layer;

[0015] The hole convolution layer obtains a second sub-feature map under different perception fields of view according to the first sub-feature map under different scales; the second sub-feature map is used to support the main module to perform a defogging operation on the RGB fog image to be processed.

[0016] Preferably, the scale transformation convolution layer includes an upsampling layer and a downsampling layer;

[0017] The upsampling layer is used to enlarge the scale of the first sub-feature map;

[0018] The downsampling layer is used to reduce the scale of the first sub-feature map.

[0019] Preferably, the main module includes: a sub-encoding module, a sub-decoding module and a feature reconstruction module;

[0020] The feature reconstruction module is connected to the sub-encoding module and the sub-decoding module respectively;

[0021] The sub-coding module is used to extract image features in the RGB fog image to be processed, and send the extracted image features in the RGB fog image to be processed to the feature reconstruction module;

[0022] The feature reconstruction module is used to integrate the image features in the to-be-processed RGB fog image output by the sub-encoding module, and send the integrated image features in the to-be-processed RGB fog image to the sub-decoding module;

[0023] The decoding module is used to reconstruct the image features of the integrated RGB fog image to be processed to obtain a target defogging image;

[0024] Preferably, the sub-encoding module comprises: a first guidance application unit, a first multi-scale dense fusion unit and an encoder;

[0025] The connection mode of each component is: second input layer → first guided application unit → first multi-scale dense fusion unit → encoder;

[0026] The second input layer is used to extract a third sub-feature map of the RGB fog image to be processed after acquiring the RGB fog image to be processed, and send the third sub-feature map to the first guidance application unit;

[0027] The first guiding application unit is used to obtain the image features of the R channel image, and is used to send the obtained image features of the R channel image to a preset position of the sub-encoding module, so that the sub-encoding module processes the RGB fog image to be processed according to the image features of the R channel image;

[0028] The first multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and send the fused feature map to the encoder;

[0029] The encoder is used to transform the scale of the fused feature map and to extract features from the fused feature map.

[0030] Preferably, the sub-decoding module comprises: a second guidance application unit, a second multi-scale dense fusion unit, a decoder and a skip feature fusion unit;

[0031] The connection mode of each component is: the third input layer → the second multi-scale dense fusion unit → the skip feature fusion unit → the second guided application unit → the decoder;

[0032] The third input layer is used to obtain image features in the to-be-processed RGB fog image sent by the encoder;

[0033] The second multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;

[0034] The skip feature fusion unit is used to receive a target feature map sent by the encoder of the sub-encoding module, the scale of which is the same as the current feature map, and fuse the target feature map with the current feature map;

[0035] The second guiding application unit is used to obtain the image features of the R channel image, and to send the obtained image features of the R channel image to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog image to be processed according to the image features of the R channel image;

[0036] The decoder is used to transform the scale of the features it receives, and to reconstruct the image of the features after the scale transformation to obtain the target defogging image.

[0037] Preferably, the sample data set includes: a training data set;

[0038] The training data set includes multiple pairs of images; each pair of images consists of two corresponding RGB fog images and clear images;

[0039] The step of using the sample data set to train the defogging model to obtain a trained defogging model includes:

[0040] The defogging model obtains a target RGB fog image;

[0041] The defogging model processes the RGB fog image to obtain a defogging image;

[0042] Calculating a loss function of the dehazed image according to the dehazed image and a clear image corresponding to the target RGB hazed image;

[0043] Updating model parameters of the defogging model according to the loss function;

[0044] Repeat the above steps until the loss function meets the preset loss value requirement.

[0045] Preferably, the sample data set also includes: a test data set;

[0046] After the sample data set is used to train the defogging model to obtain the trained defogging model, the method further includes:

[0047] The trained defogging model is tested using the test data set to obtain a test result.

[0048] Preferably, the test result includes a peak signal-to-noise ratio.

[0049] In a second aspect, the present invention further provides an image defogging system based on red channel guidance, which is applied to the method described in the first aspect of the present invention, and the system comprises:

[0050] A sample data building module is used to build a sample data set;

[0051] A module construction module is used to construct a defogging model; the defogging model includes a red channel guiding module and a main body module; the red channel guiding module is used to obtain an R channel image of an RGB fog image to be processed, extract image features of the R channel image, and send the image features to the main body module; the main body module is used to perform a defogging operation on the RGB fog image to be processed according to the image features;

[0052] A model training module, used for training the defogging model using the sample data set to obtain a trained defogging model;

[0053] The defogging module is used to defog the real foggy image using the trained defogging model.

[0054] The present invention adopts the above technical scheme, a method for image defogging based on red channel guidance, including: constructing a sample data set; constructing a defogging model; the defogging model includes a red channel guiding module and a main body module; the red channel guiding module is used to obtain the R channel image of the RGB fog image to be processed, and extract the image features of the R channel image, and send the image features to the main body module; the main body module is used to perform a defogging operation on the RGB fog image to be processed according to the image features; the defogging model is trained using the sample data set to obtain a trained defogging model; the actual fog image is defogged by the trained defogging model. Based on this, since the transmittance of the R channel is the highest among the R channel, the G channel and the B channel, light is most likely to pass through the R channel. Compared with the other two channels, the R channel has more picture detail information. The present invention guides the main body module to perform image defogging through the red channel guiding module, which can retain more picture detail information, making the defogging effect of the present invention better.

[0055] It should be noted that R, G and B represent the colors of the three channels of red, green and blue respectively. RGB images refer to images displayed in RGB color mode. Such images can only be viewed through media that use light to express colors, such as televisions and computers. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 1 is a flow chart of an image defogging method based on red channel guidance provided by an embodiment of the present invention;

[0058] Figure 2 It is a structural schematic diagram of an image defogging system based on red channel guidance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0060] Figure 1FIG. 1 is a flow chart of an image defogging method based on red channel guidance provided by an embodiment of the present invention. Figure 1 As shown, the image defogging method based on red channel guidance in this embodiment includes:

[0061] S101. Build a sample data set.

[0062] Specifically, the sample data set includes a training data set, a training data set, a foggy image and a clear image, a foggy image and a clear image. Figure 1 The fog image and the clear image are corresponding to each other and are pictures of the same shooting scene. The fog image is generated by adding synthetic fog based on the atmospheric scattering model to the corresponding clear image.

[0063] S102, constructing a defogging model; the defogging model includes a red channel guiding module and a main module; the red channel guiding module is used to obtain the R channel image of the RGB fog image to be processed, and extract the image features of the R channel image, and send the image features to the main module; the main module is used to perform a defogging operation on the RGB fog image to be processed according to the image features.

[0064] In detail, it is well known that the longer the wavelength of light, the weaker the attenuation of the light by fog. Therefore, among the three RGB channels, the red channel with the longest wavelength corresponds to the least attenuation, that is, the fog on the red channel image is lighter than the fog on the other two channel images. Therefore, the red channel image has more image details than the other two channel images. Therefore, this implementation uses the red channel guide module to guide the main module to perform defogging, so that this implementation can achieve better defogging effect.

[0065] More specifically, the red channel guiding module is used to obtain the R channel image of the RGB fog image to be processed, extract the image features of the R channel image, and send the image features to the main module. The main module obtains the picture detail information of the RGB fog image through the R channel image, and defogs the RGB fog image according to the detail information, so that the RGB fog image can retain the picture detail information of the RGB fog image obtained through the R channel image, so that the defogging effect of this embodiment is good.

[0066] S103: Use the sample data set to train the defogging model to obtain a trained defogging model.

[0067] In detail, first, the training parameters are set, and then the above defogging model is trained using python and the deep learning framework pytorch to converge to obtain the target defogging model (the trained defogging model).

[0068] In more detail, first, the training learning rate is set to 0.00005, and is reduced to 0.9 times of the original after every 10 iterations, and the total number of iterations is set to 100. Then, the foggy image in the training dataset is input into the defogging model, and the loss between the image output by the defogging model and the corresponding clear image is calculated. When the calculation results converge, the current defogging model is defined as the target defogging model.

[0069] S104: Defogging the actual foggy image using the trained defogging model.

[0070] Specifically, after the real foggy image is input into the trained defogging model, the trained defogging model performs a defogging operation on the real foggy image and outputs a clear image after defogging.

[0071] In summary, the present embodiment adopts the above technical scheme, a method for image defogging based on red channel guidance, including: constructing a sample data set; constructing a defogging model; the defogging model includes a red channel guiding module and a main body module; the red channel guiding module is used to obtain the R channel image of the RGB fog image to be processed, and extract the image features of the R channel image, and send the image features to the main body module; the main body module is used to perform a defogging operation on the RGB fog image to be processed according to the image features; the defogging model is trained using the sample data set to obtain the trained defogging model; the actual fog image is defogged by the trained defogging model. Based on this, since the transmittance of the R channel is the highest among the R channel, the G channel and the B channel, light is most likely to pass through the R channel. Compared with the other two channels, the R channel has more picture detail information. In this embodiment, the red channel guiding module guides the main body module to perform image defogging, which can retain more picture detail information, making the defogging effect of this embodiment better.

[0072] Preferably, the red channel guiding module comprises 1 first input layer, 1 scale transformation convolution layer, 4 hole convolution layers, 2 convolution layers, 1 connection layer and 1 normalization layer;

[0073] The connection method of each layer is: first input layer → scale transformation convolution layer → dilated convolution layer 1 → dilated convolution layer 2 → dilated convolution layer 3 → dilated convolution layer 4 → connection layer → convolution layer 1 → convolution layer 2 → normalization layer;

[0074] After the first input layer obtains the R channel map, it extracts a first sub-feature map from the R channel map, and sends the first sub-feature map to the scale transformation convolution layer;

[0075] The scale transformation convolution layer performs scale transformation processing on the first sub-feature map to obtain first sub-feature maps at different scales, and sends the first sub-feature maps at different scales to the hole convolution layer;

[0076] The hole convolution layer obtains a second sub-feature map under different perception fields of view according to the first sub-feature map under different scales; the second sub-feature map is used to support the main module to perform a defogging operation on the RGB fog image to be processed.

[0077] Specifically, the scale transformation convolution layer includes an upsampling layer and a downsampling layer; the upsampling layer is used to enlarge the scale of the first sub-feature map; the downsampling layer is used to reduce the scale of the first sub-feature map.

[0078] The construction method of the red channel guidance module is as follows:

[0079] 1. Connect each layer as follows:

[0080] First input layer → scale transformation convolution layer (upsampling layer) → dilated convolution layer 1 → dilated convolution layer 2 → dilated convolution layer 3 → dilated convolution layer 4 → connection layer → convolution layer 1 → convolution layer 2 → normalization layer;

[0081] 2. Repeat step 1 three times;

[0082] 3. The upsampling layer is replaced by the downsampling layer;

[0083] 4. Repeat step 1 three times to obtain the red channel guide module.

[0084] The first input layer receives the red channel in the input image and obtains the shallow features. After that, the scale transformation is performed to obtain feature maps of different scales. Then, the deep feature maps of different receptive fields are extracted through four atrous convolutional layers. The deep feature maps have more detailed information, which can provide supplement and guidance for subsequent dehazing.

[0085] The red channel guide module extracts the missing details from the red channel R, and sends this information to the corresponding position of the main module through linear mapping to guide the main module to perform defogging. The scale transformation convolution layer includes an upsampling layer and a downsampling layer, which can make the scale (i.e., size) of the feature map in the red channel guide module the same as the scale of the feature map in the current main module.

[0086] Among them, the linear mapping is: after the output of the red channel guide module enters the main module, two parallel weight parameters are calculated through two parallel convolutions (ReLU layers), and these two parameters are used to perform linear mapping on the feature map of the main module.

[0087] Preferably, the main module includes: a sub-encoding module, a sub-decoding module and a feature reconstruction module;

[0088] The feature reconstruction module is connected to the sub-encoding module and the sub-decoding module respectively;

[0089] The sub-coding module is used to extract image features in the RGB fog image to be processed, and send the extracted image features in the RGB fog image to be processed to the feature reconstruction module;

[0090] The feature reconstruction module is used to integrate the image features in the to-be-processed RGB fog image output by the sub-encoding module, and send the integrated image features in the to-be-processed RGB fog image to the sub-decoding module;

[0091] The decoding module is used to reconstruct the image features of the integrated RGB fog image to be processed to obtain a target defogging image;

[0092] Preferably, the sub-encoding module comprises: a first guidance application unit, a first multi-scale dense fusion unit and an encoder;

[0093] The connection mode of each component is: second input layer → first guided application unit → first multi-scale dense fusion unit → encoder;

[0094] The second input layer is used to extract a third sub-feature map of the RGB fog image to be processed after acquiring the RGB fog image to be processed, and send the third sub-feature map to the first guidance application unit;

[0095] The first guiding application unit is used to obtain the image features of the R channel image, and is used to send the obtained image features of the R channel image to a preset position of the sub-encoding module, so that the sub-encoding module processes the RGB fog image to be processed according to the image features of the R channel image;

[0096] The first multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and send the fused feature map to the encoder;

[0097] The encoder is used to transform the scale of the fused feature map and to extract features from the fused feature map.

[0098] Specifically, the construction method of the sub-coding module is as follows:

[0099] 1. Connect each layer as follows:

[0100] Second input layer → first guided application unit → first multi-scale dense fusion unit → encoder;

[0101] 2. Repeat step 1 three times.

[0102] Among them, the input of the second input layer is a three-channel foggy picture, and the second input layer extracts a shallow feature map of the three-channel foggy picture for subsequent deep feature extraction and processing.

[0103] More specifically, the first guided application unit includes: two first sub-convolutional layers and a linear calculation sub-unit, and the connection mode of each component is: first sub-convolutional layer 1→first sub-convolutional layer 2→linear calculation sub-unit; wherein the linear calculation formula adopted by the linear calculation sub-unit is as follows:

[0104] I l =conv1(I r )×I0+conv2(I r )

[0105] Among them, conv1 represents the first sub-convolutional layer 1, conv2 represents the first sub-convolutional layer 2, and I0 represents the input of the first sub-convolutional layer 1.

[0106] The first multi-scale dense fusion unit includes: a first sub-input layer, several first sub-connection layers, several first up-sampling convolution layers and several first down-sampling convolution layers, and the number of each component depends on the scale of the current input. Taking the third scale as an example, the connection mode of each component is: first sub-input layer → first up-sampling convolution layer 1 → first up-sampling convolution layer 2 → first up-sampling convolution layer 3 → first sub-connection layer 1 → first down-sampling convolution layer 1 → first down-sampling convolution layer 2 → first down-sampling convolution layer 3 → first sub-connection layer 2 → first up-sampling convolution layer 4 → first up-sampling convolution layer 5 → first sub-connection layer 3 → first down-sampling convolution layer 4 → first down-sampling convolution layer 5 → first sub-connection layer 4 → first up-sampling convolution layer 6 → first sub-connection layer 5 → first down-sampling convolution layer 6. Among them, the first sub-input layer inputs the output feature maps of the first three scales into the partial neural network, and fuses the feature maps of the first and second scales with the current feature map of the third scale through different numbers of up / first down-sampling convolution layers.

[0107] The encoder includes: a second sub-input layer, 1 second downsampling convolution layer, 11 second sub-convolution layers, 6 first activation layers, 7 second sub-connection layers, 2 first average pooling layers, and 1 maximum pooling layer. The connection method of each component is: second sub-input layer → second downsampling convolution layer → second sub-convolution layer 1 → first activation layer 1 → second sub-convolution layer 2 → second sub-connected layer 1 → second sub-convolution layer 3 → first activation layer 2 → second sub-convolution layer 4 → second sub-connected layer 2 → second sub-convolution layer 5 → first activation layer 3 → second sub-convolution layer 6 → second sub-connected layer 3 → second sub-convolution layer 7 → first activation layer 4 → second sub-convolution layer 8 → second sub-connected layer 4 → maximum pooling layer → first average pooling layer 1 → second sub-connected layer 5 → second sub-convolution layer 9 → first activation layer 5 → second sub-connected layer 6 → first average pooling layer 2 → second sub-convolution layer 10 → first activation layer 6 → second sub-convolution layer 11 → first activation layer 7 → second sub-connected layer 7.

[0108] Preferably, the sub-decoding module comprises: a second guidance application unit, a second multi-scale dense fusion unit, a decoder and a skip feature fusion unit;

[0109] The connection mode of each component is: the third input layer → the second multi-scale dense fusion unit → the skip feature fusion unit → the second guided application unit → the decoder;

[0110] The third input layer is used to obtain image features in the to-be-processed RGB fog image sent by the encoder;

[0111] The second multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;

[0112] The skip feature fusion unit is used to receive a target feature map sent by the encoder of the sub-encoding module, the scale of which is the same as the current feature map, and fuse the target feature map with the current feature map;

[0113] The second guiding application unit is used to obtain the image features of the R channel image, and to send the obtained image features of the R channel image to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog image to be processed according to the image features of the R channel image;

[0114] The decoder is used to transform the scale of the features it receives, and to reconstruct the image of the features after the scale transformation to obtain the target defogging image.

[0115] In detail, the second guided application unit and the second multi-scale dense fusion unit are respectively the same as the structure of the sub-coding module, the only difference is that the up / down sampling convolution layer is interchanged, which will not be repeated here. The decoder structure is the same as the encoder structure, the only difference is that the down sampling convolution layer is replaced by the up sampling convolution layer, which will not be repeated here. The connection method of each component is: the third input layer → the second multi-scale dense fusion unit → the skip feature fusion unit → the second guided application unit → the decoder; and repeated four times.

[0116] The skip feature fusion unit includes: the third sub-input layer, 2 third sub-connection layers, 2 third sub-convolutional layers, 1 second average pooling layer, and 1 second activation layer. The connection mode of each component is: third sub-input layer->third sub-connection layer 1->second average pooling layer->third sub-convolutional layer 1->third sub-convolutional layer 2->second average pooling layer->second activation layer->third sub-connection layer 2. The input of the second sub-input layer is the feature map of the current scale and the feature map of the corresponding scale of the encoding part, so as to obtain more detailed information in the feature map output by the encoder and make full use of the feature map output by the encoder.

[0117] Preferably, the sample data set includes: a training data set;

[0118] The training data set includes multiple pairs of images; each pair of images consists of two corresponding RGB fog images and clear images;

[0119] The step of using the sample data set to train the defogging model to obtain a trained defogging model includes:

[0120] The defogging model obtains a target RGB fog image;

[0121] The defogging model processes the RGB fog image to obtain a defogging image;

[0122] Calculating a loss function of the dehazed image according to the dehazed image and a clear image corresponding to the target RGB hazed image;

[0123] Updating model parameters of the defogging model according to the loss function;

[0124] Repeat the above steps until the loss function meets the preset loss value requirement.

[0125] Specifically, the target RGB fog image is input into the main module of the constructed defogging model, the R channel image of the target RGB fog image is input into the red channel guide module of the defogging model, the main module and the red channel guide module are fused for defogging, the defogging module outputs the defogging image, and the loss function of the defogging image is calculated with reference to the clear image corresponding to the target RGB fog image. Finally, the model parameters of the defogging model are updated according to the loss function. The loss function is as follows:

[0126]

[0127] Among them, J1(x) represents the output image after network dehazing; J2(x) represents the clear image in the training data.

[0128] Wherein, the Adam optimizer is used to update the model parameters of the defogging model.

[0129] Preferably, the sample data set also includes: a test data set;

[0130] After the sample data set is used to train the defogging model to obtain the trained defogging model, the method further includes:

[0131] The trained defogging model is tested using the test data set to obtain a test result.

[0132] Specifically, after the model training is completed, the fog image in the test data set is input into the trained dehazing model to obtain the output dehazed image, and the network dehazing effect is evaluated by calculating the peak signal-to-noise ratio and other parameters of the dehazed image.

[0133] Based on a general inventive concept, the present invention further provides an image defogging system based on red channel guidance, which is applied to the image defogging method based on red channel guidance in the above-mentioned embodiment of the present invention.

[0134] Figure 2 FIG. 1 is a schematic diagram of a structure of an image defogging system based on red channel guidance provided by an embodiment of the present invention. Figure 2 As shown, the image defogging system based on red channel guidance in this embodiment includes: a sample data construction module 21, a module construction module 22, a model training module 23 and a defogging module 24.

[0135] Among them, the sample data construction module 21 is used to construct a sample data set; the module construction module 22 is used to construct a defogging model; the defogging model includes a red channel guiding module and a main body module; the red channel guiding module is used to obtain the R channel image of the RGB fog image to be processed, and extract the image features of the R channel image, and send the image features to the main body module; the main body module is used to perform a defogging operation on the RGB fog image to be processed according to the image features; the model training module 23 is used to train the defogging model using the sample data set to obtain the trained defogging model; the defogging module 24 is used to defog the real-shot fog image through the trained defogging model.

[0136] Preferably, the red channel guiding module comprises 1 first input layer, 1 scale transformation convolution layer, 4 hole convolution layers, 2 convolution layers, 1 connection layer and 1 normalization layer;

[0137] The connection method of each layer is: first input layer → scale transformation convolution layer → dilated convolution layer 1 → dilated convolution layer 2 → dilated convolution layer 3 → dilated convolution layer 4 → connection layer → convolution layer 1 → convolution layer 2 → normalization layer;

[0138] After the first input layer obtains the R channel map, it extracts a first sub-feature map from the R channel map, and sends the first sub-feature map to the scale transformation convolution layer;

[0139] The scale transformation convolution layer performs scale transformation processing on the first sub-feature map to obtain first sub-feature maps at different scales, and sends the first sub-feature maps at different scales to the hole convolution layer;

[0140] The hole convolution layer obtains a second sub-feature map under different perception fields of view according to the first sub-feature map under different scales; the second sub-feature map is used to support the main module to perform a defogging operation on the RGB fog image to be processed.

[0141] Preferably, the scale transformation convolution layer includes an upsampling layer and a downsampling layer;

[0142] The upsampling layer is used to enlarge the scale of the first sub-feature map;

[0143] The downsampling layer is used to reduce the scale of the first sub-feature map.

[0144] Preferably, the main module includes: a sub-encoding module, a sub-decoding module and a feature reconstruction module;

[0145] The feature reconstruction module is connected to the sub-encoding module and the sub-decoding module respectively;

[0146] The sub-coding module is used to extract image features in the RGB fog image to be processed, and send the extracted image features in the RGB fog image to be processed to the feature reconstruction module;

[0147] The feature reconstruction module is used to integrate the image features in the to-be-processed RGB fog image output by the sub-encoding module, and send the integrated image features in the to-be-processed RGB fog image to the sub-decoding module;

[0148] The decoding module is used to reconstruct the image features of the integrated RGB fog image to be processed to obtain a target defogging image;

[0149] Preferably, the sub-encoding module comprises: a first guidance application unit, a first multi-scale dense fusion unit and an encoder;

[0150] The connection mode of each component is: second input layer → first guided application unit → first multi-scale dense fusion unit → encoder;

[0151] The second input layer is used to extract a third sub-feature map of the RGB fog image to be processed after acquiring the RGB fog image to be processed, and send the third sub-feature map to the first guidance application unit;

[0152] The first guiding application unit is used to obtain the image features of the R channel image, and is used to send the obtained image features of the R channel image to a preset position of the sub-encoding module, so that the sub-encoding module processes the RGB fog image to be processed according to the image features of the R channel image;

[0153] The first multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and send the fused feature map to the encoder;

[0154] The encoder is used to transform the scale of the fused feature map and to extract features from the fused feature map.

[0155] Preferably, the sub-decoding module comprises: a second guidance application unit, a second multi-scale dense fusion unit, a decoder and a skip feature fusion unit;

[0156] The connection mode of each component is: the third input layer → the second multi-scale dense fusion unit → the skip feature fusion unit → the second guided application unit → the decoder;

[0157] The third input layer is used to obtain image features in the to-be-processed RGB fog image sent by the encoder;

[0158] The second multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map;

[0159] The skip feature fusion unit is used to receive a target feature map sent by the encoder of the sub-encoding module, the scale of which is the same as the current feature map, and fuse the target feature map with the current feature map;

[0160] The second guiding application unit is used to obtain the image features of the R channel image, and to send the obtained image features of the R channel image to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog image to be processed according to the image features of the R channel image;

[0161] The decoder is used to transform the scale of the features it receives, and to reconstruct the image of the features after the scale transformation to obtain the target defogging image.

[0162] Preferably, the sample data set includes: a training data set;

[0163] The training data set includes multiple pairs of images; each pair of images consists of two corresponding RGB fog images and clear images;

[0164] The model training module 23 is specifically used to implement the following method:

[0165] The defogging model obtains a target RGB fog image;

[0166] The defogging model processes the RGB fog image to obtain a defogging image;

[0167] Calculating a loss function of the dehazed image according to the dehazed image and a clear image corresponding to the target RGB hazed image;

[0168] Updating model parameters of the defogging model according to the loss function;

[0169] Repeat the above steps until the loss function meets the preset loss value requirement.

[0170] Preferably, the sample data set also includes: a test data set;

[0171] The red channel guided image defogging system of this embodiment further includes: a testing module, which is used to test the trained defogging model using the test data set to obtain a test result.

[0172] Preferably, the test result includes a peak signal-to-noise ratio.

[0173] It should be noted that the image defogging system based on red channel guidance of this embodiment and the image defogging method based on red channel guidance of the above-mentioned embodiment are based on a general inventive concept, have the same or corresponding execution process and beneficial effects, and will not be repeated here.

[0174] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0175] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0176] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0177] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0178] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0179] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0180] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0181] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0182] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. An image defogging method based on red channel guidance, characterized in that: include: Construct a sample dataset; Build a dehazing model; The defogging model includes a red channel guiding module and a main module; The red channel guide module is used to obtain the R channel image of the RGB fog image to be processed, extract the image features of the R channel image, and send the image features to the main module; the main module is used to perform a defogging operation on the RGB fog image to be processed according to the image features; Using the sample data set to train the defogging model to obtain a trained defogging model; Defogging the real foggy image using the trained defogging model; The red channel guidance module includes 1 first input layer, 1 scale transformation convolution layer, 4 hole convolution layers, 2 convolution layers, 1 connection layer and 1 normalization layer; The connection method of each layer is as follows: the first input layer Scale-Transformation Convolutional Layer Dilated convolutional layer 1 Dilated convolutional layer 2 Dilated convolution layer 3 Dilated convolution layer 4 Connection Layer Convolutional layer 1 Convolutional Layer 2 Normalization layer; After the first input layer obtains the R channel map, it extracts a first sub-feature map from the R channel map, and sends the first sub-feature map to the scale transformation convolution layer; The scale transformation convolution layer performs scale transformation processing on the first sub-feature map to obtain first sub-feature maps at different scales, and sends the first sub-feature maps at different scales to the hole convolution layer; The atrous convolution layer obtains the second sub-feature maps under different perception fields according to the first sub-feature maps under different scales; The second sub-feature map is used to support the main module to perform a defogging operation on the RGB fog image to be processed; The main module includes: a sub-encoding module, a sub-decoding module and a feature reconstruction module; The feature reconstruction module is connected to the sub-encoding module and the sub-decoding module respectively; The sub-coding module is used to extract image features in the RGB fog image to be processed, and send the extracted image features in the RGB fog image to be processed to the feature reconstruction module; The feature reconstruction module is used to integrate the image features in the to-be-processed RGB fog image output by the sub-encoding module, and send the integrated image features in the to-be-processed RGB fog image to the sub-decoding module; The decoding module is used to reconstruct the image features in the integrated RGB fog image to be processed to obtain a target defogging image.

2. The image defogging method based on red channel guidance according to claim 1, characterized in that: The scale transformation convolution layer includes an upsampling layer and a downsampling layer; The upsampling layer is used to enlarge the scale of the first sub-feature map; The downsampling layer is used to reduce the scale of the first sub-feature map.

3. The image defogging method based on red channel guidance according to claim 1, characterized in that: The sub-coding module includes: a first guidance application unit, a first multi-scale dense fusion unit and an encoder; The connections of the components are as follows: The second input layer First boot application unit The first multi-scale dense fusion unit Encoder; The second input layer is used to extract a third sub-feature map of the RGB fog image to be processed after acquiring the RGB fog image to be processed, and send the third sub-feature map to the first guidance application unit; The first guiding application unit is used to obtain the image features of the R channel image, and is used to send the obtained image features of the R channel image to a preset position of the sub-encoding module, so that the sub-encoding module processes the RGB fog image to be processed according to the image features of the R channel image; The first multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-coding module with the current feature map to obtain a fused feature map, and send the fused feature map to the encoder; The encoder is used to transform the scale of the fused feature map and to extract features from the fused feature map.

4. The image defogging method based on red channel guidance according to claim 3 is characterized in that: The sub-decoding module includes: a second guidance application unit, a second multi-scale dense fusion unit, a decoder and a skip feature fusion unit; The connections of the components are as follows: The third input layer Second multi-scale dense fusion unit Skip-connect feature fusion unit Second boot application unit Decoder; The third input layer is used to obtain image features in the to-be-processed RGB fog image sent by the encoder; The second multi-scale dense fusion unit is used to fuse the received feature maps sent by each component unit of the sub-decoding module with the current feature map to obtain a fused feature map; The skip feature fusion unit is used to receive a target feature map sent by the encoder of the sub-encoding module, the scale of which is the same as the current feature map, and fuse the target feature map with the current feature map; The second guiding application unit is used to obtain the image features of the R channel image, and to send the obtained image features of the R channel image to a preset position of the sub-decoding module, so that the sub-decoding module processes the RGB fog image to be processed according to the image features of the R channel image; The decoder is used to transform the scale of the features it receives, and to reconstruct the image of the features after the scale transformation to obtain the target defogging image.

5. The image defogging method based on red channel guidance according to claim 1, characterized in that: The sample data set includes: a training data set; The training data set includes multiple pairs of images; each pair of images consists of two corresponding RGB fog images and clear images; The step of using the sample data set to train the defogging model to obtain a trained defogging model includes: The defogging model obtains a target RGB fog image; The defogging model processes the RGB fog image to obtain a defogging image; Calculating a loss function of the dehazed image according to the dehazed image and a clear image corresponding to the target RGB hazed image; Updating model parameters of the defogging model according to the loss function; Repeat the above steps until the loss function meets the preset loss value requirement.

6. The image defogging method based on red channel guidance according to claim 5, characterized in that: The sample data set also includes: a test data set; After the sample data set is used to train the defogging model to obtain the trained defogging model, the method further includes: The trained defogging model is tested using the test data set to obtain a test result.

7. The image defogging method based on red channel guidance according to claim 6, characterized in that: The test results include peak signal-to-noise ratio.

8. An image defogging system based on red channel guidance, applied to the image defogging method based on red channel guidance as claimed in claim 1, characterized in that: include: A sample data building module is used to build a sample data set; Module building module, used to build the dehazing model; The defogging model includes a red channel guiding module and a main module; the red channel guiding module is used to obtain an R channel image of the RGB fog image to be processed, extract image features of the R channel image, and send the image features to the main module; The main module is used to perform a defogging operation on the RGB fog image to be processed according to the image features; A model training module, used for training the defogging model using the sample data set to obtain a trained defogging model; The defogging module is used to defog the real foggy image using the trained defogging model.

Images