A lightweight image defogging method and system

Abstract

The present invention proposes a lightweight image defogging method and system, which relates to the field of image processing technology. The method comprises: obtaining an image to be processed, inputting the image to be processed into a trained image defogging model, and obtaining a target image output by the image defogging model. The image defogging model comprises an encoding layer and a decoding layer. In the encoding layer, the input image to be processed is subjected to multiple convolution processes to generate an intermediate feature map of multi-scale features. In the decoding layer, the intermediate feature map of multi-scale features obtained previously is subjected to defogging and context anchor attention processes in sequence, and the content features of the image are enhanced by cross-scale feature fusion. The present invention optimizes both the processing speed and the processing effect, so as to be suitable for application scenarios requiring real-time processing, meet the use requirements of actual application scenarios, and improve the reasoning speed and robustness of the image defogging model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a lightweight image defogging method and system. Background Art

[0002] With the rapid development of image processing technology, severe weather conditions (including fog, rain, etc.) often lead to a serious decline in the quality of visual information in the acquired images or videos, which has brought great trouble to many practical applications that rely on clear vision, such as autonomous driving, monitoring systems, satellite remote sensing, etc. Therefore, severe weather conditions have become the main problem faced by the practical application of visual technology in computing. Therefore, it is necessary to perform defogging on the input data obtained under severe weather conditions. The purpose of defogging is to enhance the quality of low-quality images or videos to ensure that the visual information of the input data meets the requirements. Traditional defogging methods are mainly based on physical models and prior knowledge to achieve defogging. However, in actual application scenarios, the speed of defogging is extremely high, and it is often necessary to complete the defogging operation in a short time to ensure the timely response and safe operation of the system. Traditional defogging methods often have problems such as complex calculations and insufficient adaptability to specific scenarios, and cannot meet the requirements of actual application scenarios.

[0003] Therefore, how to develop a lightweight and real-time image dehazing method to meet the requirements of various practical application scenarios is an important issue that needs to be urgently solved in the industry. Summary of the invention

[0004] In view of this, the present invention proposes a lightweight image defogging method and system to solve the problems that traditional defogging methods are often complex in calculation and lack adaptability to specific scenes.

[0005] The technical solution of the present invention is achieved in this way:

[0006] According to a first aspect, an embodiment of the present invention provides a lightweight image defogging method, the method comprising:

[0007] Get the image to be processed;

[0008] Inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model;

[0009] The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function;

[0010] The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion.

[0011] The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer;

[0012] The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0013] In combination with the first aspect, in a first implementation of the first aspect, obtaining an image to be processed, and inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model specifically includes:

[0014] Acquire the image to be processed;

[0015] Inputting the image to be processed into the point-by-point convolution layer to obtain a first feature map output by the point-by-point convolution layer;

[0016] Inputting the first feature map into the first feature extraction convolutional layer to obtain a second feature map output by the first feature extraction convolutional layer;

[0017] Inputting the second feature map into the second feature extraction convolution layer to obtain a third feature map output by the second feature extraction convolution layer;

[0018] Inputting the third feature map into the third feature extraction convolution layer to obtain a fourth feature map output by the third feature extraction convolution layer;

[0019] Inputting the fourth feature map into the fourth feature extraction convolution layer to obtain a fifth feature map output by the fourth feature extraction convolution layer;

[0020] Inputting the first to fifth feature maps into the defogging layer to obtain the first to fifth defogging feature maps respectively output by the defogging layer;

[0021] Inputting the first to fifth defogging feature maps into the context anchor attention layer to obtain the first to fifth context anchor feature maps respectively output by the context anchor attention layer;

[0022] The first to fifth context anchor feature maps are input into the cross-scale feature fusion layer to obtain a target image output by the cross-scale feature fusion layer.

[0023] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, the first to fourth feature extraction convolutional layers each include a first convolutional layer, a first linear rectification layer, and a batch normalization layer;

[0024] The first convolution layer is used to perform convolution processing on the input feature map; the linear rectification layer is used to perform linear rectification processing on the feature map after convolution processing; and the batch normalization layer is used to perform batch normalization processing on the feature map after linear rectification processing.

[0025] In combination with the second implementation of the first aspect, in the third implementation of the first aspect, the defogging layer includes a second convolution layer, an attention layer, a second linear rectification layer, a first fusion layer, a hole convolution layer, and a second fusion layer;

[0026] The second convolution layer is used to perform dimensionality reduction processing on the input feature map to obtain a first intermediate feature map; the attention layer is used to perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map; the second linear rectification layer is used to perform linear rectification processing on the first intermediate feature map; the first fusion layer is used to perform linear rectification processing on the first intermediate feature map and then perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map; the dilated convolution layer is used to perform dilated convolution processing on the third intermediate feature map; the second fusion layer is used to perform dilated convolution processing on the third intermediate feature map and then perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map.

[0027] In combination with the third implementation of the first aspect, in the fourth implementation of the first aspect, the context anchor attention layer includes an average pooling layer, a third convolutional layer, a horizontal feature extraction layer, a vertical feature extraction layer, a third fusion layer and a smoothing layer;

[0028] The average pooling layer is used to downsample the input dehazed feature map; the third convolution layer is used to convolve the dehazed feature map after the downsampling process to obtain a downsampled feature map; the horizontal feature extraction layer is used to extract features in the horizontal direction of the downsampled feature map; the vertical feature extraction layer is used to extract features in the vertical direction of the downsampled feature map; the third fusion layer is used to fuse the features extracted in the horizontal and vertical directions; the smoothing layer is used to smooth the fused features.

[0029] In combination with the fourth implementation of the first aspect, in the fifth implementation of the first aspect, the cross-scale feature fusion layer includes an upsampling layer, a fourth fusion layer and a feature restoration layer;

[0030] The upsampling layer is used to perform an interpolation upsampling operation on the input feature map; the fourth fusion layer is used to perform feature fusion on the feature map after the interpolation upsampling operation and the feature map of the previous level scale feature adjacent to the feature map, until the feature maps of all scale features are fused; the feature restoration layer is used to restore the feature map that has fused all scale features to the same scale as the image to be processed, so as to obtain a target image.

[0031] In combination with the fifth implementation of the first aspect, in the sixth implementation of the first aspect, inputting the first to fifth context anchor feature maps into the cross-scale feature fusion layer to obtain the target image output by the cross-scale feature fusion layer specifically includes:

[0032] Inputting the fifth context anchor feature map into the upsampling layer, and inputting the fifth context anchor feature map and the fourth context anchor feature map after interpolation upsampling processing into the fourth fusion layer, to obtain a first fusion feature map output by the fourth fusion layer;

[0033] Inputting the first fused feature map into the upsampling layer, and inputting the first fused feature map and the third context anchor feature map after interpolation upsampling processing into the fourth fusion layer to obtain a second fused feature map output by the fourth fusion layer;

[0034] Inputting the second fused feature map into the upsampling layer, and inputting the second fused feature map and the second context anchor feature map after interpolation upsampling processing into the fourth fusion layer, to obtain a third fused feature map output by the fourth fusion layer;

[0035] Inputting the third fused feature map into the upsampling layer, and inputting the third fused feature map after interpolation upsampling and the first context anchor feature map into the fourth fusion layer, to obtain a fourth fused feature map output by the fourth fusion layer;

[0036] The fourth fused feature map is input into the feature restoration layer to obtain the target image output by the feature restoration layer.

[0037] In combination with the fourth implementation of the first aspect, in the seventh implementation of the first aspect, the expressions for processing the third fusion layer and the smoothing layer are:

[0038]

[0039] in, represents the downsampled feature map; Indicates feature extraction in the horizontal direction; Indicates feature extraction in the vertical direction; Indicates Smoothing.

[0040] In combination with the first aspect, in an eighth implementation of the first aspect, the expression of the total loss function is:

[0041]

[0042] in, represents the total loss function; represents the mean absolute error coefficient; represents the mean square error loss coefficient; The total array representing the sample image; Indicates Set sample images; represents the output haze-free image; Represents the input sample image.

[0043] According to a second aspect, an embodiment of the present invention further provides a lightweight image defogging system, the system comprising:

[0044] An image acquisition module, used for acquiring an image to be processed;

[0045] An image defogging module is used to input the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model;

[0046] The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function;

[0047] The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion.

[0048] The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer;

[0049] The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0050] According to the third aspect, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described lightweight image defogging methods are implemented.

[0051] According to a fourth aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described lightweight image defogging methods.

[0052] According to a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the lightweight image defogging method as described in any one of the above items.

[0053] The lightweight image defogging method and system of the present invention have the following beneficial effects compared with the prior art:

[0054] The image to be processed is defogged by the trained image defogging model image to obtain a fog-free target image. The defogging model adopts a lightweight network model of the encoder-decoder framework. The image defogging model includes an encoding layer and a decoding layer. In the encoding layer, the input image to be processed is subjected to multiple convolution processes to generate an intermediate feature map of multi-scale features. In the decoding layer, the intermediate feature map of multi-scale features obtained before is defogged and context anchor attention processed in turn, and the content features of the image are enhanced by cross-scale feature fusion to reduce the fog features. Through such a network structure setting, the image defogging model can not only capture the feature information of the image to be processed from multiple scales, but also maintain the details and quality of the image to be processed as much as possible, and effectively remove the bad weather features in the image to be processed. At the same time, it also effectively utilizes limited computing resources, reduces computing overhead, maintains processing speed while reducing the impact of bad weather on the image. By optimizing both processing speed and processing effect, it is suitable for real-time operation on edge devices, etc., so it is suitable for application scenarios requiring real-time processing, meets the use requirements of actual application scenarios, and improves the reasoning speed and robustness of the image defogging model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 Schematic diagram of the process of the lightweight image defogging method of the present invention;

[0057] Figure 2 It is a schematic diagram of the process of defogging layer processing in the lightweight image defogging method of the present invention;

[0058] Figure 3 It is a flowchart of the context anchor point attention layer processing in the lightweight image defogging method of the present invention;

[0059] Figure 4 It is a structural schematic diagram of a lightweight image defogging system of the present invention;

[0060] Figure 5 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] With the rapid development of image processing technology, severe weather conditions (including fog, rain, etc.) often lead to a serious decline in the quality of visual information in the acquired images or videos. This is because the above weather conditions will reduce the contrast and color saturation of the image, making the details of the objects in the image blurred. This has brought great troubles to many practical applications that rely on clear visual information, such as autonomous driving, monitoring systems, satellite remote sensing, etc. Therefore, severe weather conditions have become the main problem faced by the practical application of computer vision technology.

[0063] Although some practical application scenarios assume that the input image or video has clear visual information, in most cases the acquired image or video does not meet the above ideal conditions. Therefore, it is necessary to perform defogging on the input data acquired under severe weather conditions. The purpose of defogging is to enhance the quality of low-quality images or videos to ensure that the visual information of the input data meets the requirements. Traditional defogging methods are mainly based on physical models and prior knowledge to achieve defogging, such as the defogging algorithm and image enhancement method used by the atmospheric scattering model. The defogging algorithm used by the atmospheric scattering model achieves image restoration by estimating parameters such as global atmospheric light and transmittance. Image enhancement methods such as histogram equalization perform defogging by enhancing contrast.

[0064] However, in actual application scenarios, the speed of dehazing is extremely high, and it is often necessary to complete the dehazing operation in a short time to ensure the timely response and safe operation of the system. Obviously, traditional dehazing methods either have complex model structures and many parameters, which result in large consumption of computing resources and are difficult to implement on resource-constrained devices, and cannot guarantee processing speed; or they rely on the estimation of a certain feature of the image, which is not applicable to real scenes and lacks adaptability to the scene.

[0065] In summary, how to develop a lightweight and real-time image dehazing method to meet the requirements of various practical application scenarios is an important issue that needs to be urgently solved in the industry.

[0066] The lightweight image defogging method provided in this specification can be applied to electronic devices with image processing capabilities, aiming to reduce the impact of bad weather on images while maintaining processing speed, so as to be suitable for application scenarios requiring real-time processing. The electronic device may include a notebook, a desktop computer, a smart phone, a smart wearable device (virtual reality glasses, smart watches, etc.), a tablet computer, etc. Of course, the lightweight image defogging method provided in this specification can also be applied to applications running in the above-mentioned electronic devices. For example, the control method of the autonomous driving can be applied to a browser with image processing capabilities, and can also be applied to software with image processing capabilities that responds immediately.

[0067] See also Figure 1 , Figure 1 A flowchart of a lightweight image defogging method according to an embodiment of the present invention is shown. The method may include the following steps:

[0068] S101: Obtain an image to be processed.

[0069] In this embodiment, the image to be processed may be stored in advance in the electronic device, or may be acquired by the electronic device from the outside, for example, acquired by the electronic device from an external image acquisition device, or a video image frame extracted by the electronic device from external video data, etc.

[0070] There is no restriction on the specific acquisition form of the image to be processed, and it is only necessary to ensure that the electronic device can acquire the image to be processed.

[0071] S102, input the image to be processed into the trained image defogging model to obtain the target image output by the image defogging model. The target image is the fog-free image obtained after the image to be processed is defogged. The target image has better image quality than the image to be processed, and the details of the objects in the image are clearer, so as to meet the requirements of various practical application scenarios such as autonomous driving, monitoring systems, satellite remote sensing, etc. for input data.

[0072] In this embodiment, the image defogging model is trained based on sample images in a self-supervised manner. The image defogging model adopts a lightweight network model of an encoder-decoder framework to adapt to the relatively low computing power requirements of platforms such as edge computing. The encoder-decoder framework of the image defogging model includes an encoding layer (encoder) and a decoding layer (decoder), wherein the input image to be processed is subjected to multiple convolution processes in the encoding layer to generate an intermediate feature map of multi-scale features, and in the decoding layer, the previously obtained intermediate feature map of multi-scale features is sequentially defogged and context anchor attention processed, and the content features of the image are enhanced by cross-scale feature fusion, thereby reducing the fog features. Finally, the model performance is optimized by a preset total loss function, so that the difference between the target image and the image to be processed is minimized when the model is used.

[0073] In this embodiment, all offline original clear foggy images are collected as sample images to provide data support for the training of subsequent image defogging models. Similarly, the sample images can also be stored in the electronic device in advance, or they can be obtained by the electronic device from the outside world. For example, the electronic device obtains the video image frame from the external image acquisition device, or the electronic device extracts the video image frame from the external video data, etc. There is no restriction on the specific acquisition form of the sample image, it only needs to ensure that the electronic device can obtain the sample image.

[0074] Preferably, foggy and fog-free images at different viewing angles and depths are selected from the OTS and NH-HAZE public data sets to produce the required sample images. For example, 500 groups of synthesized foggy images at different depths and corresponding real fog-free images at different viewing angles are selected from the OTS public data set, and then 50 groups of foggy images and corresponding real fog-free images are selected from the NH-HAZE public data set, and the two groups of images are mixed to form a sample image data set.

[0075] More specifically, the encoding layer includes a point-by-point convolution layer and four feature extraction convolution layers with different numbers of channels arranged in series, the feature extraction convolution layer is the first feature extraction convolution layer, the second feature extraction convolution layer, the third feature extraction convolution layer and the fourth feature extraction convolution layer, and the decoding layer includes a defogging layer, a context anchor attention layer and a cross-scale feature fusion layer.

[0076] The point-by-point convolution layer in the encoding layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer in the encoding layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction. The defogging layer in the decoding layer is used to perform convolution processing on the input features to obtain the first intermediate feature map , for the first intermediate feature map Perform channel attention processing and pixel attention processing in sequence to obtain the second intermediate feature map , and, for the first intermediate feature map After linear rectification, the second intermediate feature map Perform feature fusion to obtain the third intermediate feature map , for the third intermediate feature map After the hole convolution processing and the second intermediate feature map Perform feature fusion to obtain the fourth intermediate feature map ; The context anchor attention layer in the decoding layer is used to downsample the features output by the dehazing layer, extract features in the horizontal and vertical directions respectively, and perform feature fusion on the extracted horizontal and vertical features; The cross-scale feature fusion layer in the decoding layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0077] The lightweight image defogging method of the present invention performs defogging on the image to be processed through the trained image defogging model image to obtain a fog-free target image. The defogging model adopts a lightweight network model of an encoder-decoder framework. The image defogging model includes a coding layer and a decoding layer. In the coding layer, the input image to be processed is subjected to multiple convolution processes to generate an intermediate feature map of multi-scale features. In the decoding layer, the intermediate feature map of the multi-scale features obtained before is sequentially defogged and context anchor attention processed, and the content features of the image are enhanced by cross-scale feature fusion, thereby reducing the fog features. Through such a network structure setting, the image defogging model can not only capture the feature information of the image to be processed from multiple scales, but also maintain the details and quality in the image to be processed as much as possible, and effectively remove the bad weather features in the image to be processed, while also effectively utilizing limited computing resources, reducing computing overhead, maintaining processing speed while reducing the impact of bad weather on the image, and optimizing both processing speed and processing effect, so as to be suitable for real-time operation on edge devices, etc., so as to be suitable for application scenarios requiring real-time processing, meet the use requirements of actual application scenarios, and improve the reasoning speed and robustness of the image defogging model.

[0078] In the encoding layer of the image defogging model, all network structures are set in series. The input image to be processed is first processed by the point-by-point convolution layer. In this embodiment, the convolution kernel of the point-by-point convolution layer is 1×1, and the number of output channels is 64. The purpose of the point-by-point convolution layer is to adjust the number of channels of the image to be processed and keep its spatial dimension unchanged. The data output by the point-by-point convolution layer is the first feature map .

[0079] Then, it passes through four feature extraction convolution layers in sequence. The output data of these four feature extraction convolution layers with different numbers of channels are the second feature map P1, the third feature map P2, the third feature map P3 and the fourth feature map P4. The structures of each feature extraction convolution layer are similar, consisting of the first convolution layer, the first linear rectification layer (ReLU activation function layer) and the batch normalization layer (Batch Normalization layer). Specifically, the convolution kernel size of the first convolution layer is 3×3, the step size is 2, and it is used to perform convolution processing on the input feature maps; the first linear rectification layer is used to perform linear rectification processing on the feature maps after convolution processing; the batch normalization layer is used to perform batch normalization processing on the feature maps after linear rectification processing. Such a setting can reduce the internal covariate shift, make the distribution of input data of each layer more stable, and accelerate the convergence speed. After the input data, i.e., the feature map, is input into these four feature extraction convolution layers, the output feature map is half the height and width of the input feature map, and the number of channels of the output feature map is equal to the number of channels of the corresponding feature extraction convolution layer.

[0080] In this embodiment, according to the processing order, the channel numbers of the four feature extraction convolutional layers are set to 128, 256, 512 and 1024 respectively.

[0081] See also Figure 2 In the decoding layer of the image defogging model, a lightweight defogging layer is constructed. The first to fifth feature maps obtained previously are input to the defogging layer. The defogging layer includes the second convolution layer, the attention layer, the second linear rectification layer, the first fusion layer, the hole convolution layer, and the second fusion layer. In the defogging layer, a two-dimensional convolution operation of the second convolution layer is first performed to reduce the dimensionality of the input feature map to obtain the first intermediate feature map. , then, the first intermediate feature map It will go through two branches for processing, the first branch is the first intermediate feature map After the channel attention processing and pixel attention processing of the attention layer, the second intermediate feature map is obtained. , the second branch is the first intermediate feature map After being processed by the SiLU activation function in the second linear rectification layer, it is then combined with the second intermediate feature map After the first fusion layer is fused, the third intermediate feature map is obtained , followed by the third intermediate feature map After the dilated convolution layer, a dilated convolution operation is performed, and then combined with the second intermediate feature map After the second fusion layer, the fourth intermediate feature map is obtained The output feature map dimensions of each feature map after the lightweight defogging layer are the same as the input feature map dimensions. Figure 1In this way, the processing speed can be maintained while enhancing the feature information of the image, thereby reducing the fog feature. After the five convolutional layers of the point-by-point convolutional layer and the four feature extraction convolutional layers are processed by the defogging layer, the first to fifth defogging feature maps are obtained respectively.

[0082] In this embodiment, the dilation rate of the dilated convolutional layer is set to 6, which can expand the receptive field without increasing the number of parameters and the amount of calculation, while keeping the resolution of the feature map unchanged.

[0083] More specifically, the third intermediate feature map And the fourth intermediate feature map The expressions of the fusion process are:

[0084] (1)

[0085] (2)

[0086] In formula (1) and (2), Represents the first intermediate feature map The first fusion weight after the channel attention processing and pixel attention processing of the attention layer; represents the protection constant. In this embodiment, is a non-zero constant, which can avoid the denominator being 0; Represents a dilated convolution operation; Indicates the operation of changing the feature map dimension. The expression of the SiLU activation function in the second linear rectification layer is:

[0087] (3)

[0088] In formula (3), Represents the input feature map, such as the first intermediate feature map in formula (1) .

[0089] See also Figure 3 In the decoding layer of the image dehazing model, an improved context anchor attention layer based on the context anchor attention (CAA) mechanism is also set. The data received by the context anchor attention layer are the first to fifth dehazing feature maps obtained after the dehazing layer processing. The context anchor attention layer consists of an average pooling layer, a third convolutional layer, a horizontal feature extraction layer, a vertical feature extraction layer, a third fusion layer and a smoothing layer.

[0090] Specifically, the input dehazed feature map is first downsampled by the average pooling layer, and then processed by the third convolution layer with a convolution kernel of 1×1 to obtain the downsampled feature map. Then, the downsampled feature map is subjected to horizontal and vertical feature extraction respectively, and then the horizontal and vertical features are subjected to feature fusion after convolution processing by the third fusion layer with a convolution kernel of 1×1. The fused features are then subjected to sigmoid smoothing to reduce the number of parameters and the amount of calculation, making the network structure more lightweight, and finally obtaining the first to fifth context anchor feature maps respectively.

[0091] Take the first dehazing feature map as input Take this as an example to illustrate, the expression processed by the context anchor attention layer is:

[0092] (4)

[0093] (5)

[0094] In formula (4), Indicates that the average pooling operation is performed in sequence The downsampled feature map is obtained after the convolution operation with a convolution kernel size of 1×1; in formula (5), Indicates feature extraction in the horizontal direction. Indicates feature extraction in the vertical direction; Indicates Smoothing; Represents the contextual anchor feature map obtained after the improved contextual anchor attention processing.

[0095] In the decoding layer of the image dehazing model, the cross-scale feature fusion layer performs feature fusion across multi-scale features, and the cross-scale feature fusion layer consists of an upsampling layer, a fourth fusion layer, and a feature restoration layer.

[0096] The specific fusion process is that the fifth feature map P4 is processed by the improved context anchor attention layer after the defogging layer, and then an interpolation upsampling operation is performed through the upsampling layer, that is, the fifth context anchor feature map is interpolated and upsampled, and then the fourth context anchor feature map obtained by the fourth feature map P3 after the defogging layer and the improved context anchor attention layer is fused through the fourth fusion layer to obtain the first fused feature map R1; the first fused feature map R1 is interpolated and upsampled through the upsampling layer, and then the third context anchor feature map obtained by the third feature map P2 after the defogging layer and the improved context anchor attention layer is fused through the fourth fusion layer to obtain the second fused feature map R2; similar steps can be used to obtain the third fused feature map R3 and the fourth fused feature map R4 in turn. Finally, the fourth fused feature map R4 is processed by the convolution of the feature restoration layer with a convolution kernel of 1×1, and the channel is restored to 3 channels to obtain the required target image, which is the reconstructed fog-free image. The expression of the fusion process is:

[0097] (6)

[0098] (7)

[0099] (8)

[0100] (9)

[0101] In formulas (6) to (9), It indicates the operation of expanding the feature map dimension by upsampling by a preset multiple. In this embodiment, the preset multiple is set to 2.

[0102] The total loss function in the image defogging model is to use the sample image as the comparison image of the defogging image, that is, the original clear haze-free image is used as the comparison object of the reconstructed haze-free image, and the mean absolute error loss and mean square error loss of the two images are included in the total loss function. During training, the difference between the defogging image and the unprocessed sample image is minimized as the training goal, so as to update the network parameters of each layer of the image defogging model. The expression of the total loss function is:

[0103] (10)

[0104] In formula (10), represents the total loss function; represents the mean absolute error coefficient; represents the mean square error loss coefficient; Indicates the total number of sample groups in the data set, that is, the total array of sample images; Indicates the first Set sample images; The sample image is a fog-free image obtained by reconstructing the foggy image through the image defogging model, or it can be a target image obtained by reconstructing the image to be processed through the image defogging model; Represents the original clear sample image, which can also be the image to be processed.

[0105] Preferably, the specific value of the mean absolute error loss coefficient is 0.6, and the specific value of the mean square error loss coefficient is 0.4.

[0106] Accordingly, step S101 specifically includes:

[0107] S1011. Obtain an image to be processed.

[0108] S1012. Input the image to be processed into a point-by-point convolution layer to obtain a first feature map output by the point-by-point convolution layer.

[0109] S1013. Input the first feature map into the first feature extraction convolutional layer to obtain a second feature map output by the first feature extraction convolutional layer.

[0110] S1014. Input the second feature map into the second feature extraction convolution layer to obtain a third feature map output by the second feature extraction convolution layer.

[0111] S1015. Input the third feature map into the third feature extraction convolution layer to obtain a fourth feature map output by the third feature extraction convolution layer.

[0112] S1016. Input the fourth feature map into a fourth feature extraction convolutional layer to obtain a fifth feature map output by the fourth feature extraction convolutional layer.

[0113] In this way, the first to fifth feature maps output by the encoding layer are obtained.

[0114] S1017, input the first to fifth feature maps into the defogging layer to obtain the first to fifth defogging feature maps respectively output by the defogging layer.

[0115] S1018. Input the first to fifth dehazing feature maps into the context anchor attention layer to obtain the first to fifth context anchor feature maps respectively output by the context anchor attention layer.

[0116] S1019: Input the first to fifth context anchor feature maps into a cross-scale feature fusion layer to obtain a target image output by the cross-scale feature fusion layer.

[0117] The following describes a system provided by an embodiment of the present invention. The system described below and the method described above can be referenced to each other.

[0118] See also Figure 4 , Figure 4 A schematic diagram of the structure of a lightweight image defogging system according to an embodiment of the present invention is shown, and the system may include:

[0119] The image acquisition module 10 is used to acquire the image to be processed.

[0120] In this embodiment, the image to be processed may be stored in advance in the electronic device, or may be acquired by the electronic device from the outside, for example, acquired by the electronic device from an external image acquisition device, or a video image frame extracted by the electronic device from external video data, etc.

[0121] There is no restriction on the specific acquisition form of the image to be processed, and it is only necessary to ensure that the electronic device can acquire the image to be processed.

[0122] The image defogging module 20 is used to input the image to be processed into the trained image defogging model to obtain the target image output by the image defogging model. The target image is the fog-free image obtained after the image to be processed is defogged. The target image has better image quality than the image to be processed, and the details of the objects in the image are clearer, so as to meet the requirements of various practical application scenarios such as autonomous driving, monitoring systems, satellite remote sensing, etc. for input data.

[0123] In this embodiment, the image dehazing model adopts a lightweight network model of the encoder-decoder framework to adapt to the relatively low computing power requirements of platforms such as edge computing. The image dehazing model includes a coding layer and a decoding layer, wherein the input image to be processed is subjected to multiple convolution processes in the coding layer to generate an intermediate feature map of multi-scale features, and in the decoding layer, the previously obtained intermediate feature map of multi-scale features is subjected to dehazing and context anchor attention processing in turn, and the content features of the image are enhanced by cross-scale feature fusion, thereby reducing the fog features. Finally, the model performance is optimized by a preset total loss function, so that the difference between the target image and the image to be processed is minimized when the model is used.

[0124] That is, the image dehazing model is trained based on sample images in a self-supervised manner.

[0125] In this embodiment, all offline original clear foggy images are collected as sample images to provide data support for the training of subsequent image defogging models. Similarly, the sample images can also be stored in the electronic device in advance, or they can be obtained by the electronic device from the outside world. For example, the electronic device obtains the video image frame from the external image acquisition device, or the electronic device extracts the video image frame from the external video data, etc. There is no restriction on the specific acquisition form of the sample image, it only needs to ensure that the electronic device can obtain the sample image.

[0126] Preferably, foggy and fog-free images at different viewing angles and depths are selected from the OTS and NH-HAZE public data sets to produce sample images required for synthetic training. For example, 500 groups of synthesized foggy images at different depths and corresponding real fog-free images at different viewing angles are selected from the OTS public data set, and then 50 groups of foggy images and corresponding real fog-free images are selected from the NH-HAZE public data set, and the two groups of images are mixed to form a synthetic sample image data set.

[0127] More specifically, the encoding layer includes a point-by-point convolutional layer and four feature extraction convolutional layers with different numbers of channels arranged in series, and the decoding layer includes a defogging layer, a contextual anchor attention layer, and a cross-scale feature fusion layer.

[0128] The point-by-point convolution layer in the encoding layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer in the encoding layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction. The defogging layer in the decoding layer is used to perform convolution processing on the input features to obtain the first intermediate feature map , for the first intermediate feature map Perform channel attention processing and pixel attention processing in sequence to obtain the second intermediate feature map , and, for the first intermediate feature map After linear rectification, the second intermediate feature map Perform feature fusion to obtain the third intermediate feature map , for the third intermediate feature map After the hole convolution processing and the second intermediate feature map Perform feature fusion to obtain the fourth intermediate feature map ; The context anchor attention layer in the decoding layer is used to downsample the features output by the dehazing layer, extract features in the horizontal and vertical directions respectively, and perform feature fusion on the extracted horizontal and vertical features; The cross-scale feature fusion layer in the decoding layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0129] The lightweight image defogging system of the present invention performs defogging on the image to be processed through the trained image defogging model image to obtain a fog-free target image. The defogging model adopts a lightweight network model of an encoder-decoder framework. The image defogging model includes a coding layer and a decoding layer. In the coding layer, the input image to be processed is subjected to multiple convolution processes to generate an intermediate feature map of multi-scale features. In the decoding layer, the intermediate feature map of the multi-scale features obtained before is sequentially defogged and context anchor attention processed, and the content features of the image are enhanced by cross-scale feature fusion, thereby reducing the fog features. Through such a network structure setting, the image defogging model can not only capture the feature information of the image to be processed from multiple scales, but also maintain the details and quality in the image to be processed as much as possible, and effectively remove the bad weather features in the image to be processed, while also effectively utilizing limited computing resources, reducing computing overhead, maintaining processing speed while reducing the impact of bad weather on the image, and optimizing both processing speed and processing effect, so as to be suitable for real-time operation on edge devices, etc., so as to be suitable for application scenarios requiring real-time processing, meet the use requirements of actual application scenarios, and improve the reasoning speed and robustness of the image defogging model.

[0130] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510 (processor), a communication interface 520 (Communications Interface), a memory 530 (memory) and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic command in the memory 530 to execute the lightweight image defogging method, which includes:

[0131] Get the image to be processed;

[0132] Inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model;

[0133] The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function;

[0134] The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion.

[0135] The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer;

[0136] The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0137] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memor), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0138] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, the computer can execute the lightweight image defogging method provided by the above methods, the method comprising:

[0139] Get the image to be processed;

[0140] Inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model;

[0141] The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function;

[0142] The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion.

[0143] The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer;

[0144] The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0145] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the above-mentioned lightweight image defogging method, the method comprising:

[0146] Get the image to be processed;

[0147] Inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model;

[0148] The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function;

[0149] The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion.

[0150] The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer;

[0151] The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

[0152] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A lightweight image defogging method, characterized by: The method comprises: Get the image to be processed; Inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model; The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function; The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion. The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer; The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

2. The lightweight image defogging method according to claim 1, characterized in that: Inputting the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model specifically includes: Inputting the image to be processed into the point-by-point convolution layer to obtain a first feature map output by the point-by-point convolution layer; Inputting the first feature map into the first feature extraction convolutional layer to obtain a second feature map output by the first feature extraction convolutional layer; Inputting the second feature map into the second feature extraction convolution layer to obtain a third feature map output by the second feature extraction convolution layer; Inputting the third feature map into the third feature extraction convolution layer to obtain a fourth feature map output by the third feature extraction convolution layer; Inputting the fourth feature map into the fourth feature extraction convolution layer to obtain a fifth feature map output by the fourth feature extraction convolution layer; Inputting the first to fifth feature maps into the defogging layer to obtain the first to fifth defogging feature maps respectively outputted by the defogging layer; Inputting the first to fifth dehazing feature maps into the context anchor attention layer to obtain the first to fifth context anchor feature maps respectively output by the context anchor attention layer; The first to fifth context anchor feature maps are input into the cross-scale feature fusion layer to obtain a target image output by the cross-scale feature fusion layer.

3. The lightweight image defogging method according to claim 2, characterized in that: The first to fourth feature extraction convolution layers each include a first convolution layer, a first linear rectification layer, and a batch normalization layer; The first convolution layer is used to perform convolution processing on the input feature map; the linear rectification layer is used to perform linear rectification processing on the feature map after convolution processing; and the batch normalization layer is used to perform batch normalization processing on the feature map after linear rectification processing.

4. The lightweight image defogging method according to claim 3, characterized in that: The defogging layer includes a second convolution layer, an attention layer, a second linear rectification layer, a first fusion layer, a hole convolution layer and a second fusion layer; The second convolution layer is used to perform dimensionality reduction processing on the input feature map to obtain a first intermediate feature map; the attention layer is used to perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map; the second linear rectification layer is used to perform linear rectification processing on the first intermediate feature map; the first fusion layer is used to perform linear rectification processing on the first intermediate feature map and then perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map; the dilated convolution layer is used to perform dilated convolution processing on the third intermediate feature map; the second fusion layer is used to perform dilated convolution processing on the third intermediate feature map and then perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map.

5. The lightweight image defogging method according to claim 4, characterized in that: The context anchor attention layer includes an average pooling layer, a third convolutional layer, a horizontal feature extraction layer, a vertical feature extraction layer, a third fusion layer and a smoothing layer; The average pooling layer is used to downsample the input dehazed feature map; the third convolution layer is used to convolve the dehazed feature map after the downsampling process to obtain a downsampled feature map; the horizontal feature extraction layer is used to extract features in the horizontal direction of the downsampled feature map; the vertical feature extraction layer is used to extract features in the vertical direction of the downsampled feature map; the third fusion layer is used to fuse the features extracted in the horizontal and vertical directions; the smoothing layer is used to smooth the fused features.

6. The lightweight image defogging method according to claim 5, characterized in that: The cross-scale feature fusion layer includes an upsampling layer, a fourth fusion layer and a feature restoration layer; The upsampling layer is used to perform an interpolation upsampling operation on the input feature map; the fourth fusion layer is used to perform feature fusion on the feature map after the interpolation upsampling operation and the feature map of the previous level scale feature adjacent to the feature map, until the feature maps of all scale features are fused; the feature restoration layer is used to restore the feature map that has fused all scale features to the same scale as the image to be processed, so as to obtain a target image.

7. The lightweight image defogging method according to claim 6, characterized in that: The step of inputting the first to fifth context anchor feature maps into the cross-scale feature fusion layer to obtain a target image output by the cross-scale feature fusion layer specifically includes: Inputting the fifth context anchor feature map into the upsampling layer, and inputting the fifth context anchor feature map and the fourth context anchor feature map after interpolation upsampling processing into the fourth fusion layer, to obtain a first fusion feature map output by the fourth fusion layer; Inputting the first fused feature map into the upsampling layer, and inputting the first fused feature map and the third context anchor feature map after interpolation upsampling processing into the fourth fusion layer to obtain a second fused feature map output by the fourth fusion layer; Inputting the second fused feature map into the upsampling layer, and inputting the second fused feature map and the second context anchor feature map after interpolation upsampling processing into the fourth fusion layer, to obtain a third fused feature map output by the fourth fusion layer; Inputting the third fused feature map into the upsampling layer, and inputting the third fused feature map after interpolation upsampling and the first context anchor feature map into the fourth fusion layer, to obtain a fourth fused feature map output by the fourth fusion layer; The fourth fused feature map is input into the feature restoration layer to obtain the target image output by the feature restoration layer.

8. The lightweight image defogging method according to claim 5, characterized in that: The expressions for processing the third fusion layer and the smoothing layer are: ； in, represents the downsampled feature map; Indicates feature extraction in the horizontal direction; Indicates feature extraction in the vertical direction; Indicates that Smoothing.

9. The lightweight image defogging method according to claim 1, wherein: The expression of the total loss function is: ； in, represents the total loss function; represents the mean absolute error coefficient; represents the mean square error loss coefficient; The total array representing the sample image; Indicates Set sample images; represents the output haze-free image; Represents the input sample image.

10. A lightweight image defogging system, characterized in that: The system comprises: An image acquisition module, used for acquiring an image to be processed; An image defogging module is used to input the image to be processed into a trained image defogging model to obtain a target image output by the image defogging model; The image dehazing model is trained based on sample images in a self-supervised manner, and the network parameters are adjusted by a total loss function; The image defogging model includes an encoding layer and a decoding layer. The encoding layer is used to perform multiple convolution processes on the input image to be processed to generate an intermediate feature map of multi-scale features. The decoding layer is used to perform defogging and context anchor point attention processing on the intermediate feature map of multi-scale features in sequence, and perform cross-scale feature fusion. The encoding layer includes a point-by-point convolution layer, a first feature extraction convolution layer, a second feature extraction convolution layer, a third feature extraction convolution layer and a fourth feature extraction convolution layer which are sequentially connected in series, and the decoding layer includes a defogging layer, a context anchor point attention layer and a cross-scale feature fusion layer; The point-by-point convolution layer is used to adjust the number of channels of the image to be processed while keeping the spatial dimension of the image to be processed unchanged; the feature extraction convolution layer is used to reduce the dimension and width of the features output by the point-by-point convolution layer and perform feature extraction, and output feature maps of different scales; the defogging layer is used to perform convolution processing on the input features to obtain a first intermediate feature map, perform channel attention processing and pixel attention processing on the first intermediate feature map in sequence to obtain a second intermediate feature map, and perform linear rectification processing on the first intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a third intermediate feature map, and perform hole convolution processing on the third intermediate feature map and perform feature fusion with the second intermediate feature map to obtain a fourth intermediate feature map; the context anchor attention layer is used to perform feature extraction in the horizontal and vertical directions after downsampling the features output by the defogging layer, and perform feature fusion on the extracted features in the horizontal and vertical directions; the cross-scale feature fusion layer is used to perform feature fusion on the features output by the context anchor attention layer in scale order.

Images