Training-free image defogging method, system and device and storage medium

Abstract

The invention relates to the technical field of image enhancement, in particular to a training-free image defogging method, a training-free image defogging system, training-free image defogging equipment and a storage medium, on the basis of not needing a training set, a to-be-defogged image learns probability distribution of a reference image pixel level, and the distribution acts on a to-be-defogged noise-added image to obtain a defogged initial noise image; in the process of denoising the defogged initial noise image, pixel-level distribution features in the reference image are injected into the foggy day image through cross attention processing, so that the pixel features of the foggy day image are better guided to be converted into the reference image; meanwhile, based on self-attention processing, key structure elements such as object contours and relative spatial relations in the foggy day image can be better reserved in the defogging process, spatial structure features are enhanced, the defogging effect is improved, the defogged image is obtained, and the method is applied to the field of image defogging processing and is high in reliability and practicability.

Description

Technical Field

[0001] The present invention relates to the technical field of image enhancement, and specifically to a fog removal method, system, device and storage medium for images without training. Background Art

[0002] In the prior art, image fog removal is often performed through deep learning methods and diffusion models. However, both of these methods require a large amount of data sets for model training, which consume a great deal of energy and resources. Moreover, for the data sets used for model training, one is to artificially add fog to sunny images to simulate foggy images and thus create synthetic foggy data, and the other is to directly capture foggy images and their corresponding real clear images using a fog machine in a real scene. However, there are fundamental differences between these artificially generated foggy images and the images captured under real foggy conditions, which limits the generalization ability of the deep learning models trained on these data sets. Therefore, when the model encounters brand-new, untrained real foggy images, it is unable to accurately extract and process the fog information in the images, making it difficult to effectively remove the fog. As a result, the defogged images are insufficient in terms of clarity, color fidelity, and detail retention, thereby reducing the reliability and practicality of such fog removal methods in real scenarios. Summary of the Invention

[0003] The purpose of the present invention is to provide a fog removal method, system, device and storage medium for images without training that can achieve high-quality and high-efficiency fog removal without the need for data set training.

[0004] The technical solution of the present invention is as follows: A fog removal method for images without training includes the following operations: S1. The reference image and the image to be defogged are respectively subjected to noise addition processing for several times to obtain a reference noise-added image and a noise-added image to be defogged; S2. The reference noise-added image and the noise-added image to be defogged are subjected to feature fusion processing to obtain an initial noise image for fog removal; S3. The dehazed initial noisy image is subjected to several denoising processes based on attention processing to obtain a dehazed image. In the operation of the denoising process based on attention processing: the dehazed initial noisy image is subjected to several downsampling processes and feature extraction processes to obtain a dehazed initial noisy feature map; the dehazed initial noisy feature map is subjected to cross-attention processing to obtain a cross-attention feature map; the cross-attention feature map is subjected to self-attention processing to obtain a self-attention feature map; the self-attention feature map is subjected to denoising processing and several upsampling processes to obtain a first denoised image, which is used to perform the second round of denoising processing based on attention processing. The operation of subjecting the dehazed initial noisy feature map to cross-attention processing is specifically as follows: the reference noisy image and the noisy image to be dehazed are respectively subjected to feature extraction processing to obtain a reference noisy feature map and a noisy feature map to be dehazed; the query features of the dehazed initial noisy feature map are fused with the query features of the noisy feature map to be dehazed to obtain a query fusion feature; the key features and value features of the reference noisy feature map are subjected to attention processing with the query fusion feature to obtain a cross-attention feature map.

[0005] The operation of the feature fusion processing in S2 is specifically as follows: the features of the reference noisy image and the noisy image to be dehazed are respectively extracted and then subjected to instance normalization operation to obtain a reference noisy normalized image and a noisy normalized image to be dehazed; based on the mean and variance of the reference noisy normalized image, and the mean and variance of the noisy normalized image to be dehazed, a dehazed initial noisy image is obtained.

[0006] The operation of obtaining the dehazed initial noisy image based on the mean and variance of the reference noisy normalized image, and the mean and variance of the noisy normalized image to be dehazed is realized by the following formula: , is the dehazed initial noisy image, , are respectively the mean and variance of the noisy normalized image to be dehazed , , are respectively the mean and variance of the reference noisy normalized image.

[0007] In S1, the number of the same type of ground objects in the reference image and the image to be dehazed is not less than the same type number threshold, and the pixel area ratio of the target ground object in the reference image is not less than the area ratio threshold, and the clarity of the reference image is not less than the clarity threshold.

[0008] The calculation formula of the clarity of the reference image is as follows: , is the clarity, is the pixel value of the pixel point with coordinates in the reference image, , are the total numbers of the horizontal and vertical coordinates respectively, , are the pixel gradient magnitudes of the pixel points with coordinates in the x, y direction in the reference image, is the gray value of the pixel point with coordinates in the reference image, , , are the first weight, the second weight, and the third weight respectively.

[0009] The operation of the feature extraction process is specifically as follows: The initial haze-removing noise image is subjected to convolution, batch normalization, activation function processing, convolution, and batch normalization to obtain a fused normalized image; the fused normalized image and the initial haze-removing noise image are added element-wise and then subjected to ReLU activation function processing to obtain an initial haze-removing noise feature map.

[0010] The operation of the self-attention process in S3 is specifically as follows: The cross-attention feature map is subjected to max pooling and average pooling respectively to obtain a first max pooling feature map and a first average pooling feature map; the first max pooling feature map and the first average pooling feature map are respectively subjected to multi-layer perceptron processing, then feature map fusion processing based on weight sharing and non-linear processing are performed to obtain an intermediate feature map; the intermediate feature map and the cross-attention feature map are multiplied element-wise and then subjected to average pooling and max pooling respectively to obtain a second max pooling feature map and a second average pooling feature map; the second max pooling feature map and the second average pooling feature map are subjected to splicing, convolution, and non-linear processing to obtain a self-attention feature map.

[0011] An image haze-removing system without training, which is used to implement the above-mentioned image haze-removing method without training, includes: A noise-added image generation module, which is used to perform noise-adding processing on the reference image and the image to be haze-removed respectively to obtain a reference noise-added image and a haze-removing noise-added image; An initial haze-removing noise image generation module, which is used to perform feature fusion processing on the reference noise-added image and the haze-removing noise-added image to obtain an initial haze-removing noise image; A defogging image generation module is used to obtain a defogged image by performing several denoising processes based on attention processing on the initial defogging noise image. In the operation of the denoising process based on attention processing: the initial defogging noise image is subjected to several downsampling processes and feature extraction processes to obtain an initial defogging noise feature map; the initial defogging noise feature map is subjected to cross-attention processing to obtain a cross-attention feature map; the cross-attention feature map is subjected to self-attention processing to obtain a self-attention feature map; the self-attention feature map is subjected to denoising processing and several upsampling processes to obtain a first denoised image, which is used to perform the second round of denoising processing based on attention processing. The operation of subjecting the initial defogging noise feature map to cross-attention processing is specifically as follows: the reference noisy image and the noisy image to be defogged are respectively subjected to feature extraction processing to obtain a reference noisy feature map and a noisy feature map to be defogged; the query features of the initial defogging noise feature map are fused with the query features of the noisy feature map to be defogged to obtain a query fusion feature; the key features and value features of the reference noisy feature map are subjected to attention processing with the query fusion feature to obtain a cross-attention feature map.

[0012] An image defogging device without training includes a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned image defogging method without training is implemented.

[0013] A computer-readable storage medium is used to store a computer program. Among them, when the computer program is executed by a processor, the above-mentioned image defogging method without training is implemented.

[0014] The beneficial effects of the present invention are as follows: An image defogging method without training provided by the present invention enables the image to be defogged to learn the pixel-level probability distribution of the reference image on the basis of not requiring a training set, and applies this distribution to the noisy image to be defogged to obtain an initial defogging noise image; then, in the process of denoising the initial defogging noise image, the pixel-level distribution features in the reference image are injected into the foggy image through cross-attention processing, which better guides the conversion of the pixel features of the foggy image to the reference image. At the same time, based on self-attention processing, it is ensured that key structural elements such as the object contour and relative spatial relationship in the foggy image can be better retained during the defogging process, enhancing the spatial structure features and improving the defogging effect to obtain a defogged image. An image defogging method without training provided by the present invention, the reference image selected for defogging the image to be defogged may not be a bright and clear image with the same scene as the image to be defogged, as long as the bright and clear degree of the reference image is high. In this way, not only can the technical problem that the corresponding image to be defogged cannot be defogged when there is no standard image with the same scene in the prior art be solved, but also the calculation cost is reduced, and the generalization ability and stability of the defogging method are improved. A fog removal method for images without training provided by the present invention has conducted a large number of experiments on a foggy dataset and achieved better results in both no-reference and full-reference evaluation metrics. It can recover high-quality images with natural visual features and clear structural content from low-quality foggy images, and has strong practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By reading the detailed description of the preferred embodiments below, the solutions and advantages of the present application will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0016] In the drawings: Figure 1 is the fog removal effect diagram of this embodiment in the example; Figure 2 is the summary of the fog removal effect diagrams of various methods on the RTTS real-world foggy dataset in the example; Figure 2 In, (a) is the foggy image, (b) is the fog removal effect diagram of the FFA method, (c) is the fog removal effect diagram of the GDN method, (d) is the fog removal effect diagram of the IA-YOLO method, (e) is the fog removal effect diagram of the Dehamer method, (f) is the fog removal effect diagram of the FFA method, (g) is the fog removal effect diagram of the method of this embodiment; Figure 3 is the summary of the fog removal effect diagrams of various methods on the mildly foggy images in the NH-HAZE artificial simulated foggy dataset in the example; Figure 3 In, (a) is the foggy image, (b) is the fog removal effect diagram of the FFA method, (c) is the fog removal effect diagram of the Dehamer method, (d) is the fog removal effect diagram of the GDN method, (e) is the fog removal effect diagram of the MSBDN method, (f) is the fog removal effect diagram of the IA-YOLO method, (g) is the fog removal effect diagram of the method of this embodiment, (h) is the Ground truth diagram; Figure 4 is the summary of the fog removal effect diagrams of various methods on the severely foggy images in the NH-HAZE artificial simulated foggy dataset in the example; Figure 4 In, (a) is the foggy image, (b) is the fog removal effect diagram of the FFA method, (c) is the fog removal effect diagram of the Dehamer method, (d) is the fog removal effect diagram of the GDN method, (e) is the fog removal effect diagram of the MSBDN method, (f) is the fog removal effect diagram of the IA-YOLO method, (g) is the fog removal effect diagram of the method of this embodiment, (h) is the Ground truth diagram; Figure 5 is the visualization result of object detection on the RTTS real foggy dataset in the example; Figure 5 In, (a) is the foggy image, (b) is the fog removal effect diagram of the method of this embodiment. Specific Embodiments

[0017] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0018] This embodiment provides an image dehazing method without training, including the following operations: S1. The reference image and the image to be dehazed are respectively subjected to noise addition processing for several times to obtain a reference noisy image and a noisy image to be dehazed; S2. The reference noisy image and the noisy image to be dehazed are subjected to feature fusion processing to obtain an initial dehazing noise image; S3. The initial dehazing noise image is subjected to denoising processing based on attention for several times to obtain a dehazed image; in the operation of denoising processing based on attention: the initial dehazing noise image is subjected to several downsampling processes and feature extraction processes to obtain an initial dehazing noise feature map; the initial dehazing noise feature map is subjected to cross-attention processing to obtain a cross-attention feature map; the cross-attention feature map is subjected to self-attention processing to obtain a self-attention feature map; the self-attention feature map is subjected to denoising processing and several upsampling processes to obtain a first denoised image for performing the second round of denoising processing based on attention; the operation of subjecting the initial dehazing noise feature map to cross-attention processing is specifically as follows: the reference noisy image and the noisy image to be dehazed are respectively subjected to feature extraction processing to obtain a reference noisy feature map and a noisy feature map to be dehazed; the query features of the initial dehazing noise feature map are fused with the query features of the noisy feature map to be dehazed to obtain a query fusion feature; the key features and value features of the reference noisy feature map are subjected to attention processing with the query fusion feature to obtain a cross-attention feature map.

[0019] S1. The reference image and the image to be dehazed are respectively subjected to noise addition processing for several times to obtain a reference noisy image and a noisy image to be dehazed.

[0020] In this embodiment, the reference image selected for dehazing the image to be dehazed may not be a bright and clear image with the same scene as the image to be dehazed, as long as the bright and clear degree of the reference image is high. This can solve the technical problem in the prior art that the corresponding image to be dehazed cannot be dehazed when there is no standard image with the same scene. For the dehazing effect, see Figure 1 .

[0021] In addition, to further improve the effect of the defogging method in this embodiment, it is set that the number of the same type of ground objects in the reference image and the image to be defogged is not less than the threshold of the number of the same type, and the pixel area ratio of the target ground object in the reference image is not less than the area ratio threshold (the target ground object can be set according to the actual defogging requirements), and the clarity of the reference image is not less than the clarity threshold, which can make the scene in the reference image close to the scene in the image to be defogged, facilitating the image to be defogged to learn more pixel clarity features in the reference image during the subsequent defogging process, thereby improving the subsequent defogging effect.

[0022] To facilitate quickly selecting a high-quality reference image suitable for the image to be defogged, the calculation formula for the above-mentioned clarity is defined as follows: , where is the clarity, is the pixel value of the pixel point with coordinates in the reference image, , are the total numbers of the horizontal and vertical coordinates respectively, The larger, the higher the brightness of the reference image and the clearer the image color; , are the pixel gradient amplitudes of the pixel point with coordinates in the reference image in the x, y direction respectively, The larger, the richer the image edges and details and the clearer the image; is the gray value of the pixel point with coordinates in the reference image, The larger, the larger the sum of the products of the gray differences of the neighboring pixels and the clearer the image, , , are the first weight, the second weight and the third weight respectively.

[0023] S2. The reference noisy image and the noisy image to be defogged are subjected to feature fusion processing to obtain an initial defogging noise image.

[0024] By subjecting the reference noisy image and the noisy image to be defogged to feature fusion processing, the image to be defogged learns the probability distribution at the pixel level of the reference image, and this distribution is applied to the noisy image to be defogged to obtain an initial defogging noise image, facilitating the subsequent defogging process to better adapt to the image to be processed and improving the generalization ability and processing effect of the defogging method.

[0025] The operation of feature fusion processing is specifically as follows: After extracting the features of the reference noisy image and the noisy image to be dehazed (which can be achieved through convolution operations), instance normalization operations are performed to obtain the reference noisy normalized image and the noisy image to be dehazed normalized image; Based on the mean and variance of the reference noisy normalized image, as well as the mean and variance of the noisy image to be dehazed normalized image, an initial dehazed noise image is obtained.

[0026] The operation of obtaining the initial dehazed noise image based on the mean and variance of the reference noisy normalized image and the mean and variance of the noisy image to be dehazed can be achieved through the following formula: , is the initial dehazed noise image, , are respectively the mean and variance of the noisy image to be dehazed normalized image , , are respectively the mean and variance of the reference noisy normalized image.

[0027] In addition, in order to enable the initial dehazed noise image to retain more content of the foggy image and inherit more of the distribution characteristics of the clear color pixels of the reference image in this embodiment, so as to facilitate improving the subsequent dehazing effect, the operation of feature fusion processing can also be: Fusing the transmittance map of the noisy image to be dehazed with the noisy image to be dehazed (which can be achieved through the Hadamard product method), retaining the structural features in the noisy image to be dehazed to obtain the spatial structure map of the noisy image to be dehazed; The reference noisy image and the noisy image to be dehazed are processed by instance normalization operations to obtain an initial noise fusion map; The initial noise fusion map and the spatial structure map of the noisy image to be dehazed are subjected to weighted fusion processing based on spatial weights to obtain an initial dehazed noise image.

[0028] The operation of weighted fusion processing based on spatial weights can be achieved through the following formula: , ) , is the initial dehazed noise image, is the spatial weight, is the initial noise fusion map, is the spatial structure map of the noisy image to be dehazed, is the Sigmoid function, is the transmittance map, , are the first learnable parameter and the second learnable parameter.

[0029] S3. The initial haze-removed noisy image undergoes several denoising processes based on attention processing to obtain a haze-removed image.

[0030] During the denoising process of the initial haze-removed noisy image, cross-attention processing is used to inject the pixel-level distribution features in the reference image into the foggy image, better guiding the conversion of the pixel distribution features of the foggy image to the reference image. At the same time, self-attention processing is also used to ensure that key structural elements such as object contours and relative spatial relationships in the foggy image can be better retained during the haze removal process, enhancing the spatial structure features and improving the haze removal effect.

[0031] The denoising process based on attention processing includes downsampling processing, feature extraction processing, cross-attention processing (realizing the injection of pixel-level distribution features in the reference image into the foggy image), self-attention processing (enhancing the structural content of the foggy image), denoising processing (removing the noise in the foggy image and gradually restoring the image), and upsampling processing.

[0032] Taking the execution of the first round of denoising processing based on attention processing as an example, the operations when the initial haze-removed noisy image undergoes denoising processing based on attention processing are as follows: The initial haze-removed noisy image undergoes several downsampling processes and feature extraction processes to obtain an initial haze-removed noisy feature map that has become multi-scale, low spatial resolution but high channel number; the initial haze-removed noisy feature map undergoes cross-attention processing to fuse the feature map of the pixel-level probability distribution of the reference image and the structural-level features of the foggy data, realizing the transfer of the pixel-level distribution features of the reference image to the foggy image to obtain a cross-attention feature map; the cross-attention feature map undergoes self-attention processing to ensure that key structural elements such as object contours and relative spatial relationships in the foggy image can be better retained during the haze removal process, focusing on the key areas and details of the foggy image, enhancing the spatial structure features, and obtaining a self-attention feature map, solving the problem of easy loss of original structural details during the generation process; the self-attention feature map undergoes denoising processing and several upsampling processes to obtain a first denoised map, which is used to execute the second round of denoising processing based on attention processing.

[0033] The specific operation of the feature extraction process is as follows: The initial haze-removed noisy image undergoes convolution, batch normalization processing, activation function processing, convolution, and batch normalization processing to obtain a fused normalized image; the fused normalized image and the initial haze-removed noisy image are added element-wise and then subjected to ReLU activation function processing to obtain an initial haze-removed noisy feature map.

[0034] The operation of processing the self-attention feature map through cross-attention is specifically as follows: The reference noisy image and the noisy image to be dehazed are respectively processed through feature extraction to obtain a reference noisy feature map and a noisy feature map to be dehazed; The query features of the initial dehazing noise feature map are fused with the query features of the noisy feature map to be dehazed to obtain query fusion features; The key features and value features of the reference noisy feature map are subjected to attention processing with the query fusion features to obtain a cross-attention feature map.

[0035] The operation of obtaining the cross-attention feature map in each round above can be obtained through the following formula: , ), is the cross-attention feature map of the t th round, is the cross-mixing degree parameter, and its value range is [0, 1], which can change the value to adjust the dehazing intensity, is preferably 0.5, d is the word vector dimension value, is the query fusion feature of the t th round, is the query feature of the initial dehazing noise feature map of the t th round - 1. When t = 1, is the query feature of the initial dehazing noise feature map, is the query feature of the noisy feature map to be dehazed in the t th round. The noisy feature map to be dehazed in the t th round is obtained by subjecting the noisy image to be dehazed (the image to be dehazed has undergone s times of noise addition processing) in the s th round to feature extraction processing. , are respectively the key feature and value feature of the reference noisy feature map in the t th round, is the transpose of the key feature of the reference noisy feature map in the t th round. The reference noisy feature map in the t th round is the sIt is obtained by performing feature extraction on the reference noise images of each round (the reference images are obtained after s times of noise addition). The total number of times S of noise addition is the same as the total number of times T of denoising. The reference noise images of the 1st, 2nd, 3rd... t... T rounds used in the cross-attention processing are respectively the reference noise images of the Sth, S - 1th, S - 2th... s... 1st rounds during the noise addition process; similarly, the noisy images to be dehazed of the 1st, 2nd, 3rd... t... T rounds used in the cross-attention processing are respectively the noisy images to be dehazed of the Sth, S - 1th, S - 2th... s... 1st rounds during the noise addition process, which can maintain the consistency during the processing.

[0036] Generally speaking, during several times of noise addition to the reference image and the image to be dehazed, the noisy images of each time step (each time step s during the noise addition process) are retained, and feature extraction operations are respectively performed on the two groups of noisy images. At the same time, during the cross-attention processing of each subsequent round, the query features of the denoised image generated in the previous round (the previous time step t during the denoising process) are merged with the query features of the noisy feature map of the image to be dehazed at the corresponding time step to form a fused query feature; the key features and value features of the reference noisy feature map at the corresponding time step are subjected to attention processing with the query fused feature, and the feature map of the pixel-level probability distribution of the reference image and the structural-level features of the foggy data are fused to obtain the denoised image of the current round, which is used to perform the denoising processing based on attention processing of the next round.

[0037] To verify the effect of the method of this embodiment, the following experiments were conducted.

[0038] In terms of the dataset. A comprehensive experiment was carried out using the publicly available large-scale dataset RESIDE. This dataset is divided into subsets with different uses (training or evaluation) or sources (indoor or outdoor) according to different data sources and image contents. In the experiment, synthetic foggy images and their corresponding clear images were selected from the ITS (indoor training set) subset, and at the same time, extremely challenging real-world foggy images (without corresponding clear images) were selected from the RTTS (real-world task-driven test set) subset for dehazing experiments. In addition, non-uniform foggy images generated by a fog machine from the NH-HAZE dataset were also used to evaluate the performance and generalization ability of the method of this embodiment.

[0039] In terms of evaluation metrics. For the defogging results of the synthetic data selected from the ITS subset, the full-reference evaluation metrics widely adopted in the defogging task were used: Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). However, there are fundamental differences in the fog distribution between synthetic foggy images and real foggy images, and there is a lack of clean and clear images as references in real foggy data. Therefore, in addition to PSNR and SSIM, no-reference metrics were also adopted, including Natural Image Quality Evaluator (NIQE) and Blind / No-reference Image Spatial Quality Evaluator (BRISQUE), to evaluate the defogging results of the real foggy images selected from the RTTS subset. These no-reference metrics made up for the limitations of PSNR and SSIM. In addition, the object detection performance on the defogged images was used as a no-reference evaluation criterion for a specific task, and a task-driven evaluation method was adopted to evaluate the effectiveness of the defogging algorithm.

[0040] Experiment-related parameters. The experiments were conducted on an Ubuntu 18.04 computing platform equipped with an NVIDIA RTX4090 GPU with 24GB of video memory. In terms of the software environment, in order to improve the experimental efficiency and the maintainability of the code, CUDA 11.1 version, Python 3.8.5 version, and PyTorch 1.11.0 version were used. During the experiment, most configurations were set to default values, and only some key parameters were adjusted. The number of reverse sampling steps of the Denoising Diffusion Probability Model (DDPM) was set to 50, and the number of steps for saving features was also set to 50. The downsampling factor was set to 8. All experiments were repeated 5 times with different random seeds to ensure the stability of the results.

[0041] Existing technologies for comparison. The existing technologies for comparison include IA-YOLO, FFA-Net, GDN, Dehamer, and MSBDN. Among them, except for IA-YOLO using the atmospheric scattering model, the remaining models were trained using the synthetic foggy images and their corresponding real clear images in the RESIDE dataset.

[0042] Qualitative comparison of real-world and artificially simulated foggy images. From Figure 2It can be seen that on the RTTS real-world foggy dataset, serious noise artifacts and ghosting phenomena occur in FFA and GDN, which greatly affect the defogging effect. On the other hand, IA-YOLO shows obvious graininess and overexposure problems, with clear white borders on both sides of the image. Although Dehamer and MSBDN remove fog to a certain extent, they can only eliminate relatively close fog and have little effect on distant fog. In addition, during the experiment, it was also observed that MSBDN sometimes introduces blurry spots and scale distortion. In real foggy scenarios and in the absence of reference images, the above-mentioned existing technologies expose relatively obvious limitations. Their defogging effects are not ideal, and it is difficult to effectively handle complex and variable scenarios, which reflects their insufficient adaptability to different scenarios. In contrast, the method proposed in this study shows significant advantages, with better defogging effects and satisfactory performance in color restoration, clarity, and detail retention.

[0043] In addition, the experiment was also conducted on the NH-HAZE dataset. This dataset uses a professional fog generator to simulate the conditions of real-world foggy scenarios, generating images with non-uniform fog and forming image pairs with corresponding fog-free images. The experimental results Figure 3 and Figure 4 . It can be observed from Figure 3 that for the defogging of non-uniform foggy images, the method of this embodiment achieves a higher degree of scene restoration. It can recover high-quality images with natural visual features and clear content structures from low-quality and blurred foggy images. However, other existing technologies perform poorly in defogging these foggy images. It can be observed from Figure 4 that the method of this embodiment can recover clear, sharp, high-contrast, and detail-rich fog-free images.

[0044] Quantitative comparison between real-world foggy images and artificially simulated foggy images. In the study of defogging images without a real reference (Ground truth reference) in the experiment, the no-reference metrics Natural Image Quality Evaluation (NIQE) and Blind / Referenceless Image Spatial Quality Evaluation (BRISQUE) were used to evaluate the defogging effect of real foggy images selected from the RTTS dataset to make up for the limitations of Peak Signal-to-Noise Ratio (PSNR) / Structural Similarity Index (SSIM). Table 1 shows the average values of NIQE and BRISQUE. For all these metrics, lower values indicate better defogging performance. The detailed results are shown in Table 1, and it can be found that the method of this embodiment achieves the lowest values of the Natural Image Quality Evaluator (NIQE) and the Blind / Referenceless Image Spatial Quality Evaluator (BRISQUE), outperforming other defogging methods. Obviously, the defogging effect of the method of this embodiment for foggy images is the most satisfactory.

[0045] Table 1 Summary of the effects of various defogging methods on the real-world dataset RTTS

[0046] In addition, the defogged images from the NH-HAZE dataset and the ITS subset of the RESIDE dataset were evaluated and compared. Table 2 shows the average values of the full-reference metrics peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). For all these metrics, higher values indicate better performance. By analyzing Table 2, it can be seen that the method of this embodiment achieved the optimal values in all structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) metrics. The experimental results show that compared with other defogging methods, the defogging effect of the method of this embodiment is better, and it has good structure perception ability and color retention ability.

[0047] Table 2 Summary of the effects of various defogging methods on the full-reference foggy-day datasets NH - HAZE and RESIDE(ITS)

[0048] Compare with the state-of-the-art object detection methods under foggy conditions. In practical applications, image defogging is usually used as a preprocessing step for more advanced vision tasks such as object detection. Therefore, the object detection performance on the defogged images is used as a no-reference, task-specific evaluation criterion to evaluate the defogging effect of real foggy-day images that lack clear real reference images. Specifically, a task-driven evaluation method is adopted. Several state-of-the-art foggy-day image object detection models are used to detect the images in the RTTS dataset, and the mean average precision (mAP) is obtained. These object detection methods include DS-Net, IA-YOLO, YOLOX, SWDA, TogetherNet, MS-DAYOLO, and the results of using YOLOv8 to detect the defogged images of the method of this embodiment. All algorithms are ranked according to the mAP (%) results to evaluate the impact of the defogging algorithm on the performance of downstream tasks. The mAP results are shown in Table 3.

[0049] Table 3 Summary of the comparison of object detection effects of different methods on the RTTS test set

[0050] As shown in Table 3, compared with existing advanced foggy-day object detection algorithms, the method of this embodiment achieved better results in object detection after dehazing the RTTS dataset. Specifically, after the images in the RTTS dataset were dehazed using the method of this embodiment and then object detection was performed through YOLOv8, the highest mean average precision (mAP) was obtained. This indicates that the method of this embodiment improves the image quality while effectively removing the fog, making the objects in the image clearer and easier to distinguish, thereby enhancing the accuracy and robustness of object detection. This further proves the reliability and effectiveness of the method of this embodiment in practical scenarios and provides an effective solution for image dehazing.

[0051] To more intuitively demonstrate the effectiveness of our dehazing method, the visualization results of YOLOv8 detecting the original image and the dehazed image were compared, as Figure 5 shown. It can be observed that the dehazed image obtained by the method of this embodiment can produce detection results with higher confidence and fewer missed detections. These examples show that under real foggy-day conditions, the method of this embodiment can adaptively output clearer images with sharper edges, thereby helping the object detection model accurately identify and locate objects in foggy-day images.

[0052] This embodiment also provides an image dehazing system without training for implementing the above-mentioned image dehazing method without training, including: A noise-added image generation module for performing noise-added processing on the reference image and the image to be dehazed respectively for several times to obtain a reference noise-added image and a noise-added image to be dehazed; A dehazing initial noise image generation module for performing feature fusion processing on the reference noise-added image and the noise-added image to be dehazed to obtain a dehazing initial noise image; A defogging image generation module is used to obtain a defogged image by performing several denoising processes based on attention processing on the initial noisy defogging image. In the operation of the denoising process based on attention processing: the initial noisy defogging image undergoes several downsampling processes and feature extraction processes to obtain an initial noisy defogging feature map; the initial noisy defogging feature map undergoes cross-attention processing to obtain a cross-attention feature map; the cross-attention feature map undergoes self-attention processing to obtain a self-attention feature map; the self-attention feature map undergoes denoising processing and several upsampling processes to obtain a first denoised map, which is used to perform the second round of denoising processing based on attention processing. The operation of the initial noisy defogging feature map undergoing cross-attention processing is specifically as follows: the reference noisy image and the noisy image to be defogged respectively undergo feature extraction processing to obtain a reference noisy feature map and a noisy feature map to be defogged; the query features of the initial noisy defogging feature map are fused with the query features of the noisy feature map to be defogged to obtain a query fusion feature; the key features and value features of the reference noisy feature map are subjected to attention processing with the query fusion feature to obtain a cross-attention feature map.

[0053] This embodiment also provides an image defogging device without training, including a processor and a memory. Among them, when the processor executes the computer program stored in the memory, the above-mentioned image defogging method without training is implemented.

[0054] This embodiment also provides a computer-readable storage medium for storing a computer program. Among them, when the computer program is executed by a processor, the above-mentioned image defogging method without training is implemented.

[0055] The image defogging method without training provided in this embodiment enables the image to be defogged to learn the pixel-level probability distribution of the reference image on the basis of not requiring a training set, and applies this distribution to the noisy image to be defogged to obtain an initial noisy defogging image; then, in the process of denoising the initial noisy defogging image, cross-attention processing is used to inject the pixel-level distribution features in the reference image into the foggy image, better guiding the conversion of the pixel distribution features of the foggy image to those of the reference image. At the same time, self-attention processing is also used to ensure that key structural elements such as the object contours and relative spatial relationships in the foggy image can be better retained during the defogging process, enhancing the spatial structure features and improving the defogging effect to obtain a defogged image. The image defogging method without training provided in this embodiment, for the reference image selected to defog the image to be defogged, may not be a bright and clear image with the same scene as the image to be defogged. As long as the bright and clear degree of the reference image is high, this can not only solve the technical problem in the prior art that the corresponding image to be defogged cannot be defogged when there is no standard image of the same scene, but also reduce the calculation cost and improve the generalization ability and stability of the defogging method. A fog removal method for images without training provided in this embodiment has conducted a large number of experiments on a foggy dataset and achieved better results in both no-reference and full-reference evaluation metrics. It can recover high-quality images with natural visual features and clear structural content from low-quality foggy images, and has strong practicality.

Claims

1. An image defogging method without training, characterized in that, It includes the following operations: S1. The reference image and the image to be dehazed are respectively subjected to noise addition processing several times to obtain a reference noisy image and a dehazed noisy image; S2. The reference noisy image and the dehazed noisy image are subjected to feature fusion processing to obtain an initial dehazed noise image; S3. The initial dehazed noise image is subjected to noise reduction processing based on attention processing several times to obtain a dehazed image; In the operation of noise reduction processing based on attention processing: the initial dehazed noise image is subjected to several downsampling processes and feature extraction processes to obtain an initial dehazed noise feature map; The initial dehazed noise feature map is subjected to cross-attention processing to obtain a cross-attention feature map; The cross-attention feature map is subjected to self-attention processing to obtain a self-attention feature map; the self-attention feature map is subjected to noise reduction processing and several upsampling processes to obtain a first noise reduction map, which is used to perform the second round of noise reduction processing based on attention processing; The operation of subjecting the initial dehazed noise feature map to cross-attention processing is specifically as follows: the reference noisy image and the dehazed noisy image are respectively subjected to feature extraction processing to obtain a reference noisy feature map and a dehazed noisy feature map; the query feature of the initial dehazed noise feature map is fused with the query feature of the dehazed noisy feature map to obtain a query fusion feature; the key feature and value feature of the reference noisy feature map are subjected to attention processing with the query fusion feature to obtain a cross-attention feature map.

2. The image defogging method without training according to claim 1, characterized in that, In the above S2, the operation of feature fusion processing is specifically as follows: The features of the reference noisy image and the dehazed noisy image are respectively extracted and then subjected to instance normalization operation to obtain a reference noisy normalized map and a dehazed noisy normalized map; Based on the mean and variance of the reference noisy normalized map, and the mean and variance of the dehazed noisy normalized map, an initial dehazed noise image is obtained.

3. The image defogging method without training according to claim 2, wherein The operation of obtaining the initial dehazed noise image based on the mean and variance of the reference noisy normalized map, and the mean and variance of the dehazed noisy normalized map is realized by the following formula: ， is the initial noise image for haze removal, and are the mean and variance of the noisy and normalized image to be dehazed, respectively, and are the mean and variance of the reference noisy and normalized image, respectively.

4. A method for image dehazing without training according to claim 1, characterized in that, In the above S1, the number of the same ground object types in the reference image and the image to be dehazed is not less than the same type number threshold, and the pixel area ratio of the target ground object in the reference image is not less than the area ratio threshold, and the clarity of the reference image is not less than the clarity threshold.

5. A method for image dehazing without training according to claim 4, characterized in that, The calculation formula for the clarity of the reference image is as follows: ， For clarity, is the pixel value of the pixel point with coordinates in the reference image, and are the total numbers of horizontal and vertical coordinates respectively, and are the pixel gradient magnitudes of the pixel point with coordinates in the reference image in the x, y direction respectively, is the gray value of the pixel point with coordinates in the reference image, and and are the first weight, the second weight, and the third weight respectively.

6. The image defogging method without training according to claim 1, wherein In the above S3, the operation of feature extraction processing is specifically as follows: The initial dehazed noise image is subjected to convolution, batch normalization processing, activation function processing, convolution and batch normalization processing to obtain a fused normalized image; the fused normalized image and the initial dehazed noise image are subjected to element-wise addition and then ReLU activation function processing to obtain an initial dehazed noise feature map.

7. The haze removal method for images without training according to claim 1, wherein In the above S3, the operation of self-attention processing is specifically as follows: The cross-attention feature map is respectively subjected to max pooling processing and average pooling processing to obtain a first max pooling feature map and a first average pooling feature map; the first max pooling feature map and the first average pooling feature map are respectively subjected to multi-layer perceptron processing, and then feature map fusion processing based on weight sharing and non-linear processing are performed to obtain an intermediate feature map; After the intermediate feature map and the cross-attention feature map are multiplied element by element, average pooling processing and max pooling processing are respectively performed to obtain a second max pooling feature map and a second average pooling feature map; the second max pooling feature map and the second average pooling feature map are subjected to splicing, convolution and non-linear processing to obtain a self-attention feature map.

8. An image dehazing system without training, which is used to implement the image dehazing method without training according to claim 1, and is characterized in that Including: A noise-added image generation module, configured to perform noise-added processing on the reference image and the image to be dehazed for several times respectively to obtain a reference noise-added image and a noise-added image to be dehazed; A dehazing initial noise image generation module, configured to perform feature fusion processing on the reference noise-added image and the noise-added image to be dehazed to obtain a dehazing initial noise image; A dehazed image generation module, configured to perform de-noising processing based on attention for several times on the dehazing initial noise image to obtain a dehazed image; in the operation of the de-noising processing based on attention: the dehazing initial noise image is subjected to several downsampling processes and feature extraction processes to obtain a dehazing initial noise feature map; The dehazing initial noise feature map is subjected to cross-attention processing to obtain a cross-attention feature map; The cross-attention feature map is subjected to self-attention processing to obtain a self-attention feature map; the self-attention feature map is subjected to de-noising processing and several upsampling processes to obtain a first de-noised map, which is used to perform the second round of de-noising processing based on attention; the operation of subjecting the dehazing initial noise feature map to cross-attention processing is specifically as follows: the reference noise-added image and the noise-added image to be dehazed are respectively subjected to feature extraction processing to obtain a reference noise-added feature map and a noise-added feature map to be dehazed; the query feature of the dehazing initial noise feature map is fused with the query feature of the noise-added feature map to be dehazed to obtain a query fusion feature; the key feature and value feature of the reference noise-added feature map are subjected to attention processing with the query fusion feature to obtain a cross-attention feature map.

9. An image dehazing device that does not require training, characterized in that, It includes a processor and a memory. Among them, when the processor executes the computer program stored in the memory, it implements the image dehazing method without training according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, For storing a computer program, wherein when the computer program is executed by a processor, it implements the image dehazing method without training according to any one of claims 1-7.

Images