Image defogging method based on autoregressive small sample learning

Abstract

The invention discloses an image defogging method based on autoregressive small sample learning, and aims to improve defogging performance. According to the method, non-uniformly distributed haze images and clean images form small sample image pairs, clean-haze image training is carried out, and a point diffusion model of a non-uniform fog image is calculated. Then, generating a haze image in an approximate real environment by using a large number of clean images, and obtaining a defogging model through regression learning; according to the method, a collaborative multi-stage regression architecture is adopted, and the defogging effect is optimized in combination with a real clean image and an approximate haze image. A large number of data sets are generated from small samples, and a defogging model with higher precision is trained. The defogging method is natural in effect, high in robustness and widely applicable to haze removal in natural scenes such as vehicle driving and outdoor monitoring.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and image defogging, and in particular to an image defogging method based on autoregressive small sample learning. Background Art

[0002] With the rapid development of modern society, haze weather has frequently appeared in many regions around the world and has become a common environmental problem. Haze not only has a serious impact on public health and the ecological environment, but also brings significant safety hazards in scenarios such as vehicle driving and outdoor monitoring. In the field of transportation, haze will greatly reduce the driver's visual range, increase the risk of traffic accidents, and threaten road traffic safety. In the field of security monitoring, haze causes blurred monitoring images, making it difficult to capture key details, greatly reducing the effectiveness of the monitoring system, and affecting the accuracy of event tracing and image analysis. Therefore, how to use effective image processing technology to clarify images in haze weather has become an important research direction in current computer vision and intelligent image processing technology.

[0003] At present, the mainstream methods in the field of image dehazing are divided into physical model-based methods and deep learning-based methods. Physical model-based methods restore haze-free images by constructing image degradation models, such as atmospheric scattering models, with the help of physical assumptions. However, such methods are sensitive to lighting conditions and scene content, and have limited applicability. Deep learning-based methods such as FFANet and AOD-Net use haze images and haze-free images for end-to-end model training, achieving good dehazing effects. However, such methods usually rely on large-scale haze-clear image pairs as training data. In actual scenes, it is extremely difficult to obtain haze images and corresponding clear images at the same time and perspective, which limits the practicality and further development of these methods.

[0004] The patent with patent number CN202110688264.0 discloses an image defogging method that combines a defogging module, a high dynamic range (HDR) module, and a deep learning architecture ReflectNet. It improves the defogging effect through a combination of multiple modules, but its method relies on the processing of high dynamic range images, and has limited applicability to illumination changes and dynamic environments in actual haze scenes. The patent with patent number CN202111304319.X proposes image reconstruction based on the illumination and fog features of the original haze image and the clear image, and trains the defogging model in combination with two loss functions. Although its method is effective, it also faces the problem of difficulty in data acquisition. It relies on a large number of high-quality haze and clear image pairs for modeling and training, which limits its scope of application. The patent with patent number CN202010988868.2 improves the quality of defogging images by fusing the image restoration network with the image enhancement network and combining adaptive weight calculation. Its gated fusion network enhances the robustness of the dehazing algorithm, but its multi-network collaborative design requires high computing resources and is costly in actual deployment. In addition, its dehazing performance is still limited by the scale and quality of training data.

[0005] Traditional image dehazing methods rely on large-scale, high-quality haze-clear image pairs for training. However, due to the uncertainty of haze weather and the difficulty in collecting haze-free images from the same perspective, it is extremely challenging to obtain such data in actual scenarios.

[0006] The present invention aims to overcome the data collection difficulties and achieve efficient and accurate image defogging under limited small sample data conditions; improve the adaptability and robustness of the defogging model in complex haze scenes; reduce the dependence on the scale of training data, and provide an efficient image defogging solution for application scenarios such as vehicle driving and outdoor monitoring, thereby promoting the intelligent development of related fields. Summary of the invention

[0007] In view of the defects of the prior art, the present invention proposes an image defogging method based on autoregressive small sample learning. By combining autoregressive learning and a point diffusion model, a limited small sample data set is used to train the point diffusion model to generate a large number of haze images that are close to the real environment, solving the problem of difficult data acquisition for haze-clear images; a collaborative multi-stage regressive deep convolutional neural network is designed, and through a supervised attention module, a channel attention block and a cross-channel feature fusion mechanism, stage-by-stage optimized image restoration is achieved to improve the fine texture restoration capability of the defogging model; multiple loss functions and learning rate optimization strategies are combined to significantly enhance the convergence performance and adaptability of the network, providing a practical technical solution for efficient defogging in complex environments. The image defogging method based on autoregressive small sample learning described in the present invention includes the following steps:

[0008] S1: Construction of small sample datasets of natural images and haze images: The camera is installed on a fixed bracket to collect clear natural images in haze-free weather as natural images, and a glass sheet is placed 5-10 cm in front of the camera to generate haze images with different transmittances and uneven distribution of fog concentrations by spraying rain and fog. A group of natural images and haze images constitute a small sample data set, and multiple small sample data constitute a small sample dataset.

[0009] S2: Constructing a collaborative multi-stage haze elimination network: This network is a deep convolutional neural network. The deep convolutional neural network is based on a collaborative multi-stage regression architecture and is divided into three stages to gradually restore the image. Specifically, it includes the following stages:

[0010] The first stage divides the input image into 4 non-overlapping image blocks, and the second stage divides the input image into 2 non-overlapping image blocks. Each stage extracts rich context information through a sub-network of the encoder-decoder structure;

[0011] A supervised attention module is introduced between the first stage and the second stage, and a supervised attention module is introduced between the second stage and the third stage. The supervised attention module generates an attention map using the prediction results of the previous stage to calibrate the features of the previous stage, and transfers the calibrated features to the next stage;

[0012] In the third stage, the original resolution image data is input and a sub-network without downsampling operation is used to extract fine texture features;

[0013] S3: Training the collaborative multi-stage haze elimination network: Use the small sample data set in step S1 to train the deep convolutional neural network in step S2, use the Adam optimizer to update the network parameters, and optimize the network based on the weighted combination loss function. The calculation formula of the loss function is:

[0014]

[0015] in, L SSIM , L MSE , L MAE The weight coefficients of the loss function are set as Where L SSIM is the structural similarity loss, L SSIM =1-SSIM, the calculation formula of SSIM is:

[0016]

[0017] Among them, x and y are the true value and the predicted result respectively, μ x and μ y is the mean, σx and σ y is the standard deviation, σ xy is the covariance, c1=(k1L) 2 and c2=(K2L) 2 , L is the dynamic range of pixel values, k1=0.01, K2=0.02 are constants.

[0018] Among them, L MSE is the mean square error loss, and the calculation formula is:

[0019]

[0020] Among them, L MAE is the mean absolute error loss, and the calculation formula is:

[0021]

[0022] Among them, y i is the true value, is the predicted value, the number of samples is n, and the deep convolutional neural network adopts a learning rate adjustment method combining a warm-up strategy and cosine annealing during training, wherein the calculation formula of the warm-up stage is:

[0023]

[0024] Where t is the current number of training times, min_lr is the minimum learning rate, max_lr is the maximum learning rate, and warmup_steps is the number of warmup steps; the formula for calculating the cosine annealing change with increasing learning rate cycles is:

[0025]

[0026] T max，i+1 =T max,i ×T mult (7)

[0027] Among them, T cur is the number of iterations since the end of the previous cycle, T max,i is the length of the ith cycle, T mult is the increasing rate, and T max,i According to the increasing ratio T mult Increase;

[0028] S4: Application of the haze removal model: The haze image to be processed is input into the trained collaborative multi-stage haze removal network to generate the corresponding clear dehazed image as the output result.

[0029] As a preferred technical solution of the present invention, the collaborative multi-stage regression architecture described in step S2 includes the following module features:

[0030] A channel attention block is embedded in the encoder-decoder subnetwork, which optimizes the image feature extraction effect through a global weighted approach, where the features are first transformed through two layers of convolution operations and ReLU activation functions, and then the channel weights are generated in combination with global average pooling, and feature optimization is completed through residual connections;

[0031] Information is transferred between stages through a cross-stage feature fusion mechanism, which adjusts the channel dimensions of the encoder and decoder output features of the previous stage respectively, and then fuses them with the features of the current stage to avoid the loss of feature information during the stage-by-stage recovery process;

[0032] In the network design, skip connections are combined with channel attention blocks to further optimize the feature expression ability, and residual connections are used to improve the flow of feature information.

[0033] As a technical preferred solution of the present invention, the channel attention block comprises the following steps:

[0034] The input feature map x is transformed by two layers of convolution operations and an activation function to obtain the intermediate feature map, which is calculated as follows:

[0035] y1=δ(conv1(x)) (8)

[0036] y2=conv2(y1) (9)

[0037] Among them, conv1 and conv2 represent convolution operations, and δ represents the ReLU activation function;

[0038] Perform global average pooling on the intermediate feature map y2 to obtain the average value y of each channel avg , and its calculation formula is:

[0039]

[0040] The average value y of the channel avg Through two layers of convolution operations, dimensionality reduction and dimensionality increase are performed, and the channel weight Z is generated by combining the ReLU activation function and the Sigmoid activation function. c , and its calculation formula is:

[0041] Z c =σ(W2δ(W1y avg )) (11)

[0042] Among them, y avg is the channel value after pooling, y 2ijcis the value of the input feature map (i, j, c) position, H and W are the height and width of the feature map, W1 and W2 are the weights of the convolutional layer, δ is the ReLU activation function, σ is the Sigmoid activation function, and Z c is the final channel weight;

[0043] By channel weight Z c The intermediate feature map y2 is weighted to obtain the weighted feature, and its calculation formula is:

[0044] x′ ijc =Z c ×y 2ijc (12)

[0045] Finally, add to the input: x′ ijc = x′ ijc +x, and get the final output of the channel attention block.

[0046] As a technical preferred solution of the present invention, the supervised attention module in S2 is used to calibrate and optimize features between stages, and its implementation method includes the following steps:

[0047] The input feature F is transformed into in Convert to R S , and its calculation formula is:

[0048] R S =Conv 1×1 (F in ) (13)

[0049] Then R S Add to the original input image I to get X S , and its calculation formula is:

[0050] X S =I+R S (14)

[0051] Next, for X S A 1×1 convolution and a sigmoid activation function are used to generate a pixel-by-pixel attention mask M, which is calculated as:

[0052] M=σ(Conv 1×1 (X S )) (15)

[0053] Finally, use the attention mask M and the input feature F after 1×1 convolution in Weighted, and added to the input feature to get the output feature F out , and its calculation formula is:

[0054] Fout =Conv 1×1 (F in )×M+F in (16)

[0055] The calibrated feature F is generated through the above steps out , which is used to pass on to the next stage.

[0056] As a technical preferred solution of the present invention, the cross-stage feature fusion mechanism is used to realize the integration and optimization of features of different stages, wherein each stage includes an encoder and a decoder, and its implementation method includes the following steps:

[0057] The characteristics of the first stage encoder output and the decoder output features To adjust the channel dimension, the calculation formula is:

[0058]

[0059] Among them, Conv 1×1 Convolution operation is used to transform the number of channels of features; the adjusted encoder features and decoder features are combined with the second stage encoder output features The calculation formula for fusion is:

[0060]

[0061] Among them, the output characteristics of the first stage encoder are The decoder output features are The output characteristics of the stage 2 encoder are

[0062] The characteristics of the second stage encoder output and the decoder output features To adjust the channel dimension, the calculation formula is:

[0063]

[0064] The adjusted encoder features and decoder features are combined with the original resolution block output features of the third stage After fusion, the final fusion feature is:

[0065]

[0066] Through the above steps, a cross-channel feature fusion module is introduced between different stages to ensure information sharing between features of each stage, enhance the network's perception of key features, and effectively avoid feature loss, thereby improving image restoration performance.

[0067] As a technical preferred solution of the present invention, in step S3, the small sample data set in step S1 is used to train the deep convolutional neural network in step S2, and an expanded haze data set is generated through a point diffusion model and used for regression learning. The implementation method includes the following steps:

[0068] Using the natural-haze image pairs in the small sample dataset, a point diffusion model is trained to generate images with haze characteristics;

[0069] Inputting a plurality of clear natural images into the point diffusion model to generate a haze image with haze distribution; recombining the generated haze image and its corresponding natural image into a haze-natural image pair to form an expanded haze data set, the size of which can be adjusted according to actual needs;

[0070] Using the expanded haze data set to perform regression learning on a deep convolutional neural network to optimize the network's defogging capabilities for different haze scenes;

[0071] The haze image to be processed is input into the trained dehazing model to generate a final dehazed image, which retains the clear texture and natural color information of the image.

[0072] Compared with the prior art, the present invention has the following beneficial effects:

[0073] Overcoming the problem of insufficient data and improving the generalization ability of the model: This paper uses a small sample data set to generate a large amount of synthetic image data with real haze characteristics through a point diffusion model, effectively expanding the size of the haze-clear image pair data set. By expanding the data set, not only does it solve the training bottleneck caused by insufficient real data, but it also significantly enhances the generalization ability of the defogging model in different haze scenes, making it applicable to a variety of complex environments.

[0074] Achieve efficient, accurate and stable dehazing effect: This invention uses a multi-stage regression deep convolutional neural network to perform stage-by-stage image restoration, combined with a supervised attention module, channel attention block and cross-channel feature fusion mechanism, to accurately extract the hidden details in the image and effectively remove haze. At the same time, the network design without downsampling operation further improves the fine texture quality of the restored image.

[0075] Enhanced image detail retention capability: The present invention adopts weighted loss function (including structural similarity loss, mean square error loss and mean absolute error loss) and residual connection design, which can not only accurately remove fog in the image, but also retain the detailed texture and color information of the image, ensuring that the defogged image is real and natural, meeting the high requirements of image quality in scenarios such as vehicle driving and outdoor monitoring.

[0076] Efficient convergence and model performance optimization: By combining the preheating strategy with the learning rate adjustment method of cosine annealing, the present invention can effectively accelerate the convergence speed of deep convolutional neural network training and improve the stability and robustness of the dehazing model, thereby achieving excellent performance under limited resources.

[0077] Strong adaptability and wide application scenarios: The defogging method designed in the present invention has strong adaptability, can complete model training under small sample data conditions, and achieve stable defogging effects under a variety of haze environments and lighting conditions. It is widely used in actual scenarios that require clear images, such as vehicle driving, outdoor monitoring, and security systems, and provides reliable technical support for the intelligent development of related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 A flowchart of an image defogging method based on autoregressive small sample learning provided by the present invention;

[0079] Figure 2 A method flow chart of an autoregressive small sample defogging method according to an embodiment of the present invention is provided;

[0080] Figure 3 A channel attention structure diagram of an embodiment of the present invention is provided;

[0081] Figure 4 A structure diagram of a supervisory attention module is provided for an embodiment of the present invention;

[0082] Figure 5 A cross-channel feature fusion module structure diagram is provided for an embodiment of the present invention;

[0083] Figure 6 The present invention provides a comparison diagram of the defogging effect of an embodiment of the present invention. DETAILED DESCRIPTION

[0084] The present invention is further described below in conjunction with the examples. However, the present invention can be implemented in many different ways and should not be construed as being limited to the examples shown; on the contrary, these examples provide those skilled in the art with implementation methods that meet applicable legal requirements.

[0085] Example 1: Figure 1 As shown, the image defogging method based on autoregressive small sample learning described in the present invention is implemented by the following steps:

[0086] S1: Use a camera to collect natural images as fog-free images in the foggy dataset. Suitable weather is required during the collection process. Fix the camera with a bracket and add glass pieces and spray rain and fog 5-10 cm in front of the camera to shoot natural images and corresponding haze images with different transmittances and uneven fog concentrations to form a set of data sets. When shooting, ensure that the camera focuses on the background, atmospheric conditions and static objects, cover a variety of background scenes, and take measures to minimize the reflection effect of glass. In order to reduce the model's dependence on samples, 120 sets of small sample data sets are collected here for training.

[0087] The collected data sets were downsampled to 720×480, and 100 groups were taken as training sets and 20 groups as test sets to train the deep convolutional neural network.

[0088] S2: Figure 2 As shown in the figure, a deep convolutional neural network is constructed for the autoregressive small sample learning dehazing method.

[0089] The deep convolutional neural network adopts a collaborative multi-stage regression architecture, which consists of three stages. Each stage uses a lightweight sub-network to progressively restore the image. The first two stages divide the image into 4 and 2 non-overlapping blocks respectively, and use the encoder-decoder sub-network to learn rich contextual information. At the same time, a supervised attention module is introduced between each two stages. The supervised attention module uses the prediction results of the previous stage to calculate the attention map, recalibrates the features of the previous stage, and then passes it to the next stage, providing effective supervision for progressive image restoration. In addition, a cross-stage feature fusion mechanism is used to exchange information between stages to avoid information loss. The last stage is the original image, and a sub-network without downsampling operation is used to allow the output image to retain the required fine texture. The encoder-decoder sub-network adds a channel attention block to extract features of each scale of the image. The feature map of the skip connection is also processed using the channel attention block. Finally, it is fused by upsampling and convolution operations, which can better alleviate the gradient disappearance and improve the feature expression ability. The original resolution subnetwork is composed of multiple original resolution modules. It extracts and optimizes feature information by integrating channel attention blocks and additional convolution operations, and uses residual connections to promote the flow and fusion of information, effectively improving the network's perception and processing capabilities of features.

[0090] like Figure 3 As shown, the channel attention block first transforms the features through two convolutional layers and an activation function:

[0091] y1=δ(conv1(x)) (8)

[0092] y2=conv2(y1) (9)

[0093] The features are weighted by global average pooling, feature channel dimensionality reduction and dimensionality increase.

[0094]

[0095] Z c =σ(W2δ(W1y avg )) (11)

[0096] where y avg is the channel value after pooling, y 2ijc is the value of the input feature map at position (i, j, c), H and W are the height and width of the feature map, W1 and W2 are the weights of the convolutional layer, δ is the ReLU activation function, σ is the Sigmoid activation function, and Z c is the final channel weight.

[0097] Then by weighting the output:

[0098] x′ ijc =Z c ×y 2ijc (12)

[0099] Finally add to the input: x′ ijc = x′ ijc +x, and get the final output of the channel attention block.

[0100] like Figure 4 As shown in the figure, the supervised attention module uses a 1×1 convolution to transform the input feature F in Convert to R S :

[0101] R S =Conv 1×1 (F in ) (13)

[0102] Then R S Add to the original input image I to get X S :

[0103] X S =I+R S (14)

[0104] Next, for X S Generate a pixel-wise attention mask M using a 1×1 convolution and a sigmoid activation function:

[0105] M=σ(Conv 1×1 (X S )) (15)

[0106] Finally, use the attention mask M and the input feature F after 1×1 convolutionin Weighted, and added to the input feature to get the output feature F out :

[0107] F out =Conv 1×1 (F in )×M+F in (16)

[0108] like Figure 5 As shown in the cross-channel feature fusion module, the output feature of the encoder in the first stage is The decoder output features are The output characteristics of the stage 2 encoder are

[0109] Adjust the channel dimension of the features of the encoder and decoder in stage one:

[0110]

[0111] Among them, Conv 1×1 Represents a 1×1 convolution operation.

[0112] Finally, the fusion features:

[0113]

[0114] The output characteristics of the stage 2 encoder are The output characteristics of the decoder are The output features of the original resolution block in stage three are

[0115] Adjust the channel dimension of the features of the encoder and decoder in stage 2:

[0116]

[0117] Among them, Conv 1×1 Represents a 1×1 convolution operation.

[0118] Finally, the fusion features:

[0119]

[0120] S3: Training a collaborative multi-stage haze elimination network: Using the small sample data set described in step S1 to train the deep convolutional neural network described in step S2, using the Adam optimizer to update the network parameters, and optimizing the network based on a weighted combination loss function.

[0121] Loss function: The final dehazing result is compared with the clean natural image to calculate the loss function. The Adam optimizer is used to train the network and update the network parameters. The loss function formula is:

[0122]

[0123] in L SSIM , L MSE , L MAE The weight coefficients of the loss function are set as

[0124] Where L SSIM is the structural similarity loss, L SSIM =1-SSIM, the SSIM calculation formula is as follows:

[0125]

[0126] Where x and y are the true value and the predicted result respectively, μ x and μ y is the mean, σ x and σ y is the standard deviation, σ xy is the covariance, c1=(k1L) 2 , c2=(K2L) 2 , L is the dynamic range of pixel values, k1 and K2 are constants, where k1=0.01, K2=0.02.

[0127] L MSE is the mean square error loss, and the calculation formula is as follows:

[0128]

[0129] L MAE is the mean absolute error loss, calculated as follows:

[0130]

[0131] Among them, y i is the true value, is the predicted value, and the sample size is n.

[0132] Learning rate: The maximum learning rate of the Adam optimizer is 1e-4, and the minimum learning rate is 1e-6. The cosine annealing strategy with an increased cycle in the preheating phase is adopted. The preheating cycle is 5, the initial cosine annealing cycle is 20, and the increment ratio is 2.

[0133] The learning rate warm-up phase change formula is:

[0134]

[0135] Where t is the current number of training steps, min_lr is the minimum learning rate, max_lr is the maximum learning rate, and warmup_steps is the number of warmup steps.

[0136] The formula for the cosine annealing change with increasing learning rate cycles is:

[0137]

[0138] T max，i+1 =T max,i ×T mult (7)

[0139] Where T cur is the number of iterations since the end of the previous cycle, T max,i is the length of the ith cycle, T mult is the increasing rate, and T max,i According to the increasing ratio T mult Increase.

[0140] S4: Figure 6 As shown in the figure, the point diffusion model of clean-haze images is trained with a small sample data set. 10,000 clean natural images are used to run the point diffusion model to generate 1,000 uniform haze images. The haze to clean image pairs are reorganized to form 10,000 haze data sets. The generated haze data sets are subjected to regression learning to obtain the final dehazing model. The haze image to be processed is input to obtain the final dehazed image.

[0141] Example 2: This example uses a suburban road driving assistance scenario in a certain city, and applies an image defogging method based on autoregressive small sample learning to verify the performance of this method in an actual haze environment, especially the defogging effect and detail retention capability under different haze concentrations. The experiment is based on field-collected images, combined with a point diffusion model to expand the data set, and uses a trained deep convolutional neural network model to defog real-time images.

[0142] A variety of road scenes were selected for experimental data collection, including straight roads, curves, green areas, urban building backgrounds, etc., covering common driving environments. The acquisition equipment is a high-definition camera fixed on the windshield of the vehicle, which simulates the haze environment through a transparent glass sheet and a spray device. The spray device can adjust the spray volume to accurately control the haze concentration and collect foggy images under light, moderate and heavy haze conditions. The shooting angle and time period are consistent at each concentration to ensure that the lighting conditions are relatively constant and reduce the impact of light changes on the image. In addition, the vehicle travels at a speed of 40-60 kilometers per hour to simulate real driving conditions. Finally, 120 sets of clear-haze image pairs were collected, each of which contains a fog-free clear image and a corresponding foggy image.

[0143] In order to overcome the limitation of collecting real clear-haze image pairs, the present invention first uses the point diffusion model to expand the small sample data set. The point diffusion model generates 1000 high-quality uniform haze images from 10,000 clear natural images by learning the distribution relationship between clear images and haze images. These generated haze images cover different fog concentration and transmittance characteristics, and after recombining with clear images, 10,000 sets of haze-clear image pairs are formed. The expanded data set has a wide distribution characteristic, which greatly improves the generalization ability of the subsequent training model in complex scenes.

[0144] The model training is based on an expanded dataset, and a collaborative multi-stage regression deep convolutional neural network is used for image dehazing. The network extracts contextual information through a staged encoder-decoder architecture and optimizes image features stage by stage through a supervised attention module. During training, the Adam optimizer is used to update parameters with a weighted combination loss function as the target. The loss function combines structural similarity loss, mean square error loss, and mean absolute error loss, with weight coefficients set to 1.5, 2, and 1, respectively. During the warm-up phase of the training process, the learning rate is first increased from 1×10 -6 Gradually increase to 1×10 -4 , and then the cosine annealing strategy is used to gradually reduce the learning rate to accelerate convergence and improve model performance. After 50 rounds of training, the model reaches the optimal state and can accurately restore the input haze image.

[0145] Among them, the quantitative indicators include peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), and the calculation formula is as follows:

[0146] PSNR is used to evaluate the restoration accuracy of the dehazed image, and the calculation formula is:

[0147]

[0148] Where MAX is the maximum value of the image pixel (for 8-bit images, MAX=255); MSE is the mean square error, calculated as:

[0149]

[0150] Where H and W are the height and width of the image, and I(i, j) and I′(i, j) are the grayscale values ​​of the real image and the dehazed image at the pixel position (i, j), respectively.

[0151] SSIM is used to evaluate the structural similarity of images, and the calculation formula is:

[0152]

[0153] Among them, μx and μ y is the mean of the image (x, y); and is the variance of the image (x, y); σ xy is the covariance of the two images; where C1 = (K1L) 2 ,C2=(K2L) 2 , L is the dynamic range of pixel value, K1=0.01, K2=0.03.

[0154] In the vehicle driving experiment, the trained defogging model was used to process the haze images collected by the vehicle camera in real time. The test scenes included three conditions: light, moderate and heavy haze. The experiment compared the original haze image with the defogging image, and analyzed its visual effects and quantitative indicators, including peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). Under light haze conditions, the PSNR value of the defogging image reached 32.1dB, the SSIM value was 0.94, and the visual effect was close to that of a haze-free environment; under moderate haze conditions, the PSNR value was 29.5dB, the SSIM value was 0.91, and the image texture details were well restored; under heavy haze conditions, the PSNR value dropped to 26.8dB, the SSIM value was 0.87, but it could still retain key road and vehicle information, significantly improving image recognizability. Its quantitative effects are shown in Tables 1 and 2.

[0155] Table 1: Quantitative comparison between the present invention and the traditional method

[0156]

[0157]

[0158] Table 2: Image quality improvement before and after dehazing

[0159] Haze concentration index Foggy Image Dehazed image (this invention) Improvement Mild haze PSNR(dB) 21.5 32.1 31.8 SSIM 0.68 0.94 0.78 Moderate haze PSNR(dB) 18.2 29.5 33.9 SSIM 0.55 0.91 1.08 Heavy smog PSNR(dB) 14.8 26.8 36 SSIM 0.42 0.87 1.35

[0160] As can be seen from Table 1, the present invention has achieved quantization results that are superior to FFANet and AOD-Net under light, moderate and severe haze conditions. Especially in the severe haze environment, the PSNR of the dehazed image is improved by more than 2.5dB, and the SSIM value is also significantly improved, indicating that the present invention has better image detail recovery capabilities in high-concentration haze scenes. Table 2 further shows the improvement in image quality before and after dehazing. It can be seen that the PSNR improvement of the present invention under moderate and severe haze conditions is 11.3dB and 12.0dB respectively, and the SSIM value improvement is as high as 0.45, which fully demonstrates the significant effect of the present invention under different concentrations of haze.

[0161] The experiment of this embodiment shows that the method of the present invention generates a large amount of haze data through the point diffusion model under small sample conditions, and combines the collaborative multi-stage regression deep convolutional neural network to achieve efficient, accurate and stable image defogging. The experimental results not only verify the superior performance of the present invention under different haze concentration conditions, but also demonstrate its practical application value in driving assistance, providing strong technical support for safe vehicle driving.

[0162] The above embodiments only express the implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention.

Claims

1. An image dehazing method based on autoregressive small sample learning, characterized by: The following steps are involved: S1: Construction of small sample datasets of natural images and haze images: The camera is installed on a fixed bracket, and clear natural images are collected in haze-free weather as natural images; a glass sheet is placed 5-10 cm in front of the camera, and rain and fog are sprayed on the glass sheet to form fog with different transmittances and uneven concentration distribution, and images are collected as haze images; a group of natural images and haze images constitute a small sample data, and multiple small sample data constitute a small sample dataset; S2: Constructing a collaborative multi-stage haze elimination network: This network is a deep convolutional neural network. The deep convolutional neural network is based on a collaborative multi-stage regression architecture and is divided into three stages to gradually restore the image, including the following: The first stage divides the input image into 4 non-overlapping image blocks, and the second stage divides the input image into 2 non-overlapping image blocks. Each stage extracts rich context information through a sub-network of the encoder-decoder structure; A supervised attention module is introduced between the first stage and the second stage, and a supervised attention module is introduced between the second stage and the third stage. The supervised attention module generates an attention map using the prediction results of the previous stage to calibrate the features of the previous stage, and transfers the calibrated features to the next stage; In the third stage, the original resolution image data is input and a sub-network without downsampling operation is used to extract fine texture features; S3: Training the collaborative multi-stage haze elimination network: Use the small sample data set in step S1 to train the deep convolutional neural network in step S2, use the Adam optimizer to update the network parameters, and optimize the network based on the weighted combination loss function. The calculation formula of the loss function is: in, L SSIM , L MSE , L MAE The weight coefficients of the loss function are set as Where L SSIM is the structural similarity loss, L SSIM =1-SSIM, the calculation formula of SSIM is: Among them, x and y are the true value and the predicted result respectively, μ x and μ y is the mean, σ x and σ y is the standard deviation, σ xy is the covariance, c1=(k1L) 2 , c2=(K2L) 2 , L is the dynamic range of pixel values, k1 and K2 are constants, where k1 = 0.01, K2 = 0.02; L MSE is the mean square error loss, and its calculation formula is: L MAE is the mean absolute error loss, which is calculated as follows: Among them, y i is the true value, is the predicted value, n is the number of samples, and the deep convolutional neural network adopts a learning rate adjustment method combining a warm-up strategy and cosine annealing during training, wherein the calculation formula of the warm-up stage is: Among them, t is the current number of training times, min_lr is the minimum learning rate, max_lr is the maximum learning rate, and warmup_steps is the number of warmup steps; The formula for calculating the cosine annealing change with increasing learning rate period is: T max，i+1 =T max,i ×T mult (7) Among them, T cur is the number of iterations since the end of the previous cycle; T max,i is the length of the ith period; T mult is the increasing rate, and T max,i According to the increasing ratio T mult Increase; S4: Application of the haze removal model: The haze image to be processed is input into the trained collaborative multi-stage haze removal network to generate the corresponding clear dehazed image as the output result.

2. The image defogging method based on autoregressive small sample learning according to claim 1, characterized in that: The collaborative multi-stage regression architecture in step S2 includes the following module features: A channel attention block is embedded in the encoder-decoder subnetwork, which optimizes the image feature extraction effect through a global weighted approach, where the features are first transformed through two layers of convolution operations and ReLU activation functions, and then the channel weights are generated in combination with global average pooling, and feature optimization is completed through residual connections; Information is transferred between stages through a cross-stage feature fusion mechanism, which adjusts the channel dimensions of the encoder and decoder output features of the previous stage respectively, and then fuses them with the features of the current stage to avoid the loss of feature information during the stage-by-stage recovery process; In the network design, skip connections are combined with channel attention blocks to further optimize the feature expression ability, and residual connections are used to improve the flow of feature information.

3. The image defogging method based on autoregressive small sample learning according to claim 2, characterized in that: The channel attention block includes the following: The input feature map x is transformed by two layers of convolution operations and an activation function to obtain the intermediate feature map, which is calculated as follows: y1=δ(conv1(x)) (8) y2=conv2(y1) (9) Among them, conv1 and conv2 represent convolution operations, and δ represents the ReLU activation function; Perform global average pooling on the intermediate feature map y2 to obtain the average value y of each channel avg , and its calculation formula is: The average value y of the channel avg Through two layers of convolution operations, dimensionality reduction and dimensionality increase are performed, and the channel weight Z is generated by combining the ReLU activation function and the Sigmoid activation function. c , and its calculation formula is: Z c =σ(W2δ(W1y avg )) (11) Among them, y avg is the channel value after pooling, y 2ijc is the value of the input feature map (i, j, c) position, H and W are the height and width of the feature map, W1 and W2 are the weights of the convolutional layer, δ is the ReLU activation function, σ is the Sigmoid activation function, and Z c is the final channel weight; By channel weight Z c The intermediate feature map y2 is weighted to obtain the weighted feature, and its calculation formula is: x′ ijc =Z c ×y 2ijc (12) Finally, the weighted features are added to the input: x′ ijc = x′ ijc +x, and get the final output of the channel attention block.

4. The image defogging method based on autoregressive small sample learning according to claim 1, characterized in that: The supervised attention module in step S2 is used to calibrate and optimize features between stages, and its implementation method includes the following steps: The input feature F is transformed into in Convert to R S , and its calculation formula is: R S =Conv 1×1 (F in ) (13) Then R S Add to the original input image I to get X S , and its calculation formula is: X S =I+R S (14) Next, for X S A 1×1 convolution and a sigmoid activation function are used to generate a pixel-by-pixel attention mask M, which is calculated as: M=σ(Conv 1×1 (X S )) (15) Finally, use the attention mask M and the input feature F after 1×1 convolution in Weighted, and added to the input feature to get the output feature F out , and its calculation formula is: F out =Conv 1×1 (F in )×M+F in (16) The calibrated feature F is generated through the above steps out , which is used to pass on to the next stage.

5. The image defogging method based on autoregressive small sample learning according to claim 2, characterized in that: The cross-stage feature fusion mechanism is used to realize the integration and optimization of features of different stages, wherein each stage includes an encoder and a decoder, and its implementation method includes the following steps: The characteristics of the first stage encoder output and the decoder output features To adjust the channel dimension, the calculation formula is: Among them, Conv 1×1 It is a convolution operation, which is used to transform the number of channels of features; The adjusted encoder features and decoder features are combined with the second stage encoder output features The calculation formula for fusion is: Among them, the output characteristics of the first stage encoder are The decoder output features are The output characteristics of the stage 2 encoder are The characteristics of the second stage encoder output and the decoder output features To adjust the channel dimension, the calculation formula is: The adjusted encoder features and decoder features are combined with the original resolution block output features of the third stage After fusion, the final fusion feature is: Through the above steps, a cross-channel feature fusion module is introduced between different stages to ensure information sharing between features of each stage, enhance the network's perception of key features, effectively avoid feature loss, and improve image restoration performance.

6. The image defogging method based on autoregressive small sample learning according to claim 1, characterized in that: In step S3, the deep convolutional neural network in step S2 is trained using the small sample data set in step S1, and an expanded haze data set is generated through a point diffusion model and used for regression learning. The implementation method includes the following steps: Using the natural-haze image pairs in the small sample dataset, a point diffusion model is trained to generate images with haze characteristics; Inputting a plurality of clear natural images into the point diffusion model to generate a haze image with haze distribution; recombining the generated haze image and its corresponding natural image into a haze-natural image pair to form an expanded haze data set, the size of which can be adjusted according to actual needs; Using the expanded haze data set to perform regression learning on a deep convolutional neural network to optimize the network's defogging capabilities for different haze scenes; The haze image to be processed is input into the trained dehazing model to generate a final dehazed image, which retains the clear texture and natural color information of the image.

Images