Dust fog weather image processing method and system based on space-time frequency domain weighting

Abstract

The embodiment of the invention discloses a dust fog weather image processing method and system based on space-time frequency domain weighting, and the method comprises the steps: obtaining a target image which is an initial degraded image which is captured in severe weather and is to be processed; constructing a pseudo time image sequence by simulating a physical process of image degradation caused by atmospheric scattering based on the target image; performing space-time frequency domain transformation on the pseudo time image sequence, and obtaining frequency domain representation of the pseudo time image sequence; performing weighting processing on the frequency domain representation through a deconvolution kernel; performing inverse transformation on the weighted frequency domain representation and reconstructing a time-space domain of the pseudo time image sequence; and aggregating the pseudo time image sequence after the reconstruction of the time-space domain in the time dimension to obtain a processed degenerated image. The method can effectively improve the definition and contrast of image details, and is suitable for degeneration recovery scenes such as defogging and dedusting.

Description

Technical Field

[0001] This application relates to the technical field of computer vision and image processing, specifically to a method and system for processing dust and fog weather images based on spatiotemporal frequency domain weighting. Background Technology

[0002] In the fields of computer vision and image processing, the performance of outdoor vision systems (such as those used in autonomous driving and video surveillance) is highly dependent on the quality of the input images. However, under adverse weather conditions such as fog, dust, and haze, the scattering of visible light by atmospheric particulate matter can lead to severe degradation of captured images, primarily manifested as decreased overall contrast, image blurring, color distortion, and loss of detailed texture information. This degradation not only severely affects the visual appeal of the image but also poses a significant challenge to subsequent high-level vision tasks such as object detection and semantic segmentation, thus limiting the reliable application of vision systems in complex real-world environments.

[0003] To address the aforementioned problems, existing image restoration methods can be broadly categorized into two types. One type is traditional methods based on physical models, such as traditional prior algorithms centered on atmospheric scattering models. While these methods offer some physical interpretability, their restoration effectiveness heavily relies on accurate estimations of key parameters like atmospheric light and transmittance. In practical applications, these parameters are often difficult to obtain adaptively, leading to insufficient generalization ability and poor adaptability to complex degradation scenarios with varying concentrations and lighting conditions. The other type is data-driven methods based on deep learning. These methods learn the mapping relationship from degraded images to clear images by training end-to-end network models. However, their performance is largely limited by the quality and scale of the training dataset. Because paired degraded-clear images are difficult to obtain on a large scale in real-world scenes, and because there are differences between synthetic data and real-world degradation, deep learning models often perform poorly when faced with complex and varied degradation patterns in the real world, exhibiting weak generalization ability.

[0004] Therefore, under the current technological background, there is an urgent need to develop an image restoration method that is robust, adaptable to different degradation conditions, and does not rely on precise physical prior knowledge or large-scale high-quality paired datasets, in order to improve the practicality of vision systems in real harsh weather environments. Summary of the Invention

[0005] This application proposes a method and system for processing dust and fog weather images based on spatiotemporal frequency domain weighting, in order to overcome the deficiencies of the prior art.

[0006] According to a first aspect of the embodiments of this application, a method for processing dust and fog weather images based on spatiotemporal frequency domain weighting is provided, including: Acquire a target image, which is an initial degraded image captured under severe weather conditions and to be processed; Based on the target image, a pseudo-temporal image sequence is constructed by simulating the physical process of image degradation caused by atmospheric scattering. The blurriness of the pseudo-temporal image sequence increases with the time dimension. The pseudo-time image sequence is subjected to spatiotemporal frequency domain transformation to obtain the frequency domain representation of the pseudo-time image sequence; The frequency domain representation is weighted by a deconvolution kernel, wherein the weighting parameters of the deconvolution kernel are adjusted in real time according to the atmospheric conditions of the target image to enhance high-frequency details and suppress noise. The weighted frequency domain representation is inversely transformed to reconstruct the spatiotemporal domain of the pseudo-temporal image sequence; The pseudo-temporal image sequence after reconstruction of the spatiotemporal domain is aggregated along the time dimension to obtain the processed de-degradation image.

[0007] In some implementations, constructing a pseudo-temporal image sequence based on the target image by simulating the physical process of image degradation caused by atmospheric scattering includes: In the time dimension, by simulating the physical process of image degradation caused by atmospheric scattering, a Gaussian filter is used to filter the target image to generate the pseudo-time image sequence; The scaling parameter of the Gaussian filter increases monotonically with the increase of the frame number in the time dimension.

[0008] In some embodiments, performing spatiotemporal frequency domain transformation on the pseudo-time image sequence includes: The pseudo-time image sequence is transformed from the spatiotemporal domain to the frequency domain using a three-dimensional discrete Fourier transform.

[0009] In some implementations, the weighting of the frequency domain representation using a deconvolution kernel includes: The deconvolution kernel is constructed based on the spatial frequency components and spatiotemporal frequency components of the image; The amplitude of the deconvolution kernel is positively correlated with the modulus of the spatial frequency and associated with the temporal frequency component, and is used to reverse atmospheric scattering in the frequency domain.

[0010] In some embodiments, the method further includes: Before performing spatiotemporal frequency domain transformation on the pseudo-time image sequence, the pseudo-time image sequence is zero-padding. After reconstructing the spatiotemporal domain of the pseudo-temporal image sequence, the boundary regions introduced by zero-padding are cropped.

[0011] In some implementations, before reconstructing the spatiotemporal domain of the pseudo-temporal image sequence, the method further includes: The target image is subjected to repeated noise suppression processing multiple times; Based on the target image after noise suppression processing, the spatiotemporal domain of the reconstructed pseudo-temporal image sequence is performed to obtain multiple enhanced images; The average value of the multiple enhanced images is taken at the pixel level to generate a de-degradation image, which is used to suppress the noise amplification effect.

[0012] In some embodiments, the repeated noise suppression processing of the target image includes: In each noise suppression process performed on the target image, Gaussian noise is added to the target image and a noisy image is generated; In this process, different Gaussian noise is added to the target image each time, so as to generate different noisy images.

[0013] In some implementations, the number of repetitions of the noise suppression process is determined based on the noise level of the target image.

[0014] In some embodiments, the target image is a color image including multiple color channels, and the method further includes: The steps of constructing a pseudo-temporal image sequence, performing a spatiotemporal frequency domain transformation on the pseudo-temporal image sequence to obtain its frequency domain representation, weighting the frequency domain representation using a deconvolution kernel, performing an inverse transformation on the weighted frequency domain representation to reconstruct the spatiotemporal domain of the pseudo-temporal image sequence, and aggregating the reconstructed spatiotemporal pseudo-temporal image sequence in the time dimension to obtain a de-degraded image are performed independently on each color channel of the target image. The de-degraded images obtained from each color channel are merged to generate a color de-degraded image.

[0015] According to a second aspect of this application, a dust and fog weather image processing system based on spatiotemporal frequency domain weighting is provided, comprising: The target image acquisition module is used to acquire a target image, which is an initial degraded image captured under severe weather conditions and to be processed. A pseudo-temporal image sequence construction module is used to construct a pseudo-temporal image sequence based on the target image by simulating the physical process of image degradation caused by atmospheric scattering. The blurriness of the pseudo-temporal image sequence increases with the time dimension. The spatiotemporal frequency domain transformation module is used to perform spatiotemporal frequency domain transformation on the pseudo-time image sequence and obtain the frequency domain representation of the pseudo-time image sequence; The weighted processing module is used to perform weighted processing on the frequency domain representation using a deconvolution kernel, wherein the weighted processing parameters of the deconvolution kernel are adjusted in real time according to the atmospheric conditions of the target image to enhance high-frequency details and suppress noise. The spatiotemporal domain construction module is used to perform an inverse transformation on the weighted frequency domain representation and reconstruct the spatiotemporal domain of the pseudo-time image sequence; The image aggregation module is used to aggregate the pseudo-temporal image sequence after reconstruction in the spatiotemporal domain along the time dimension to obtain the processed de-degradation image.

[0016] The beneficial effects of the dust and fog weather image processing method and system based on spatiotemporal frequency domain weighting in this application include at least the following: This application's embodiments, by acquiring the target image, clarify that the algorithm's input is a single degraded image, laying the foundation for data-driven processing. This eliminates the reliance on additional prior information (such as depth maps or atmospheric physical parameters), improving the method's practicality and convenience in real-world scenarios. By simulating the physical process of atmospheric scattering, a single image is expanded into a dynamic degradation sequence. This effect provides a data foundation rich in temporal evolution information for subsequent frequency domain analysis, thereby transforming the complex spatial degradation problem into a more manageable spatiotemporal evolution problem. By performing spatiotemporal frequency domain transformation on the pseudo-time image sequence and obtaining its frequency domain representation, the image sequence is mapped from the spatiotemporal domain to the frequency domain, achieving the separation and representation of image details (high frequency), contours (low frequency), and the degradation process. This creates conditions for subsequent precise weighted processing, enabling the algorithm to operate directly in the frequency dimension. Frequency domain weighted processing is performed using adaptively adjustable deconvolution kernels; parameters can be adjusted in real time according to the image's atmospheric conditions, allowing the weighted processing to specifically enhance high-frequency details attenuated by scattering while simultaneously suppressing noise. This effect directly determines the quality of the restored image, achieving adaptive enhancement for images with different degrees of degradation. By accurately mapping the frequency domain enhancement effect back to the image space, a sequence of images with improved quality is generated, completing the crucial transformation from "frequency domain processing" to "spatial domain results," ensuring that all enhancement operations are reflected in pixel values. By aggregating the optimized sequence in the temporal dimension, the restoration information of each frame in the sequence is fused, ultimately outputting a clear image, stabilizing and improving the overall quality of the restoration result, and avoiding the instability that may arise from single-frame restoration. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the dust and fog weather image processing method based on spatiotemporal frequency domain weighting according to an embodiment of this application. Figure 2 This is a flowchart illustrating a specific embodiment of the dust and fog weather image processing method based on spatiotemporal frequency domain weighting according to this application. Figure 3 This is a comparison of the image recovery results under dusty and foggy weather conditions according to an embodiment of this application; Figure 4This is a schematic diagram of the structure of a dust and fog weather image processing system based on spatiotemporal frequency domain weighting according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following description of the dust and fog weather image processing method and system based on spatiotemporal frequency domain weighting will be provided in conjunction with the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some, not all, of the embodiments of this application. The components of the embodiments of this application described and shown in the accompanying drawings can typically be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed embodiments of the present application, but merely to illustrate selected embodiments of the present application. Other embodiments obtained by those skilled in the art based on the embodiments of the present application without inventive effort are all within the scope of protection of the embodiments of the present application.

[0020] It can be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it will not be further defined and explained in subsequent figures according to the embodiments of this application.

[0021] This application discloses a method for processing dusty and foggy weather images based on spatiotemporal frequency domain weighting. This method is executed based on a dusty and foggy weather image processing system based on spatiotemporal frequency domain weighting, aiming to solve the problem of insufficient adaptability of image degradation recovery to complex scene environments in existing technologies. The method includes steps 110-160.

[0022] Step 110: Obtain the target image.

[0023] The target image is an initial degraded image captured under severe weather conditions and awaiting processing. The initial degraded image is also known as the raw degraded image, such as an image taken in a dusty or foggy scene. The target image has at least the following typical degradation characteristics caused by atmospheric scattering, including but not limited to: low contrast (blurred details), color distortion (such as grayish-white tones caused by haze), or attenuation of edge information (unclear outlines).

[0024] This application embodiment does not require prior knowledge such as defect depth, only requires input of the degraded image, and this step provides a data source for all subsequent processing (such as sequence construction, frequency domain transformation, weighting and reconstruction).

[0025] Step 120: Based on the target image, a pseudo-temporal image sequence is constructed by simulating the physical process of image degradation caused by atmospheric scattering.

[0026] This step, also known as the sequence construction step, involves the blurring of the pseudo-time image sequence increasing with the time dimension.

[0027] In some implementations, constructing a pseudo-temporal image sequence based on the target image by simulating the physical process of image degradation caused by atmospheric scattering includes: simulating the physical process of image degradation caused by atmospheric scattering in the time dimension by using a Gaussian filter to filter the target image (at different scales) to generate the pseudo-temporal image sequence.

[0028] For example, the scale parameter of the Gaussian filter monotonically increases with the frame number in the time dimension, used to generate a pseudo-time image sequence with increasing blur levels, thus simulating the light scattering process. Specifically, the original image can be filtered multiple times using a Gaussian filter with a linearly increasing scale parameter to generate the sequence. The initial frame of the sequence is the original image or a slightly blurred image, and the blur level of each subsequent frame increases sequentially, thereby simulating a degradation process from clear to completely blurred in the time dimension.

[0029] Step 130: Perform a spatiotemporal frequency domain transformation on the pseudo-time image sequence to obtain the frequency domain representation of the pseudo-time image sequence.

[0030] This step is also known as the frequency domain transformation step. Spatiotemporal frequency domain transformation is a three-dimensional signal processing technique that considers both the spatial dimension (rows and columns) and the pseudo-temporal dimension (sequence frame order) of the image. Its purpose is to decompose the scattering degradation features in the sequence into frequency components, where: spatial frequency corresponds to the texture and edge information of image details, high-frequency components represent details, and low-frequency components represent the overall contour. Temporal frequency (denoted by ω) corresponds to the evolution of blur in the pseudo-temporal sequence; high-frequency components reflect rapid fluctuations in blur changes, and low-frequency components reflect the overall degradation trend. The frequency domain representation refers to the complex frequency spectrum obtained after the transformation, whose dimension is consistent with the original sequence (temporal frequency × spatial frequency). This representation maps the image degradation process to a quantifiable frequency distribution, providing a basis for the weighted processing of the deconvolution kernel.

[0031] In some implementations, the spatiotemporal frequency domain transformation of the pseudo-time image sequence includes: using a three-dimensional discrete Fourier transform to transform the pseudo-time image sequence from the spatiotemporal domain to the frequency domain.

[0032] For example, the sequence can be zero-padded before the transformation to improve the boundary effects of frequency domain processing.

[0033] Step 140: Weight the frequency domain representation using a deconvolution kernel.

[0034] This step, also known as the frequency domain weighting step, involves adjusting the weighting parameters of the deconvolution kernel in real time based on the atmospheric conditions of the target image to enhance high-frequency details and suppress noise.

[0035] In some implementations, the weighting of the frequency domain representation by the deconvolution kernel includes constructing the deconvolution kernel based on the spatial frequency components and spatiotemporal frequency components of the image.

[0036] For example, the deconvolution kernel parameters are adjusted based on the current atmospheric conditions of the image being processed.

[0037] For example, the amplitude of the deconvolution kernel is positively correlated with the modulus of the spatial frequency and associated with the temporal frequency component, used to reverse atmospheric scattering in the frequency domain to enhance high-frequency spatial details of the image in the frequency domain. For example, the correlation between the amplitude of the deconvolution kernel and the modulus of the spatial frequency can be adapted to different weather visibility conditions through a diffusion coefficient.

[0038] Weighting this frequency domain representation using deconvolution kernels can enhance spatial high-frequency details that are attenuated by scattering, while suppressing noise and invalid frequency components. Step 150: Perform an inverse transform on the weighted frequency domain representation and reconstruct the spatiotemporal domain of the pseudo-time image sequence.

[0039] This step is also known as the image reconstruction step. The inverse transform is the reverse process of the previous three-dimensional spatiotemporal frequency domain transformation (step 140). Specifically, it uses a three-dimensional discrete Fourier inverse transform to map the processed frequency domain representation back from the frequency domain to the spatiotemporal domain. This is a precise mathematical inverse operation, ensuring that no information is lost during the transformation. The reconstructed spatiotemporal domain means that the output of the inverse transform is a three-dimensional spatiotemporal data volume with the same dimensions as the original pseudo-time image sequence. This reconstructed sequence contains enhanced image information, where each frame corresponds to the image at the same position in the original pseudo-time sequence, but its pixel values ​​have been repaired, and details and contrast have been improved.

[0040] In this embodiment of the application, the enhancement effect in the frequency domain (such as high-frequency detail enhancement and noise suppression) is actually reflected in the pixel value, generating a visually improved image sequence. The reconstructed spatiotemporal domain sequence is the direct input to the subsequent "aggregation" step 160. Only by obtaining this reconstructed sequence can the final single enhanced image be obtained through temporal aggregation.

[0041] Step 160: Aggregate the pseudo-temporal image sequence after reconstruction of the spatiotemporal domain along the time dimension to obtain the processed de-degradation image.

[0042] The processed de-degradation image can be understood as an enhanced de-degradation image. Aggregation can be determined, for example, by calculating the average value.

[0043] In some implementations, the method further includes: zero-padding the pseudo-time image sequence before performing a spatiotemporal frequency domain transformation on the pseudo-time image sequence; and after reconstructing the spatiotemporal domain of the pseudo-time image sequence, cropping the boundary regions introduced by the zero-padding, i.e., trimming away the extra boundaries introduced by the zero-padding, and restoring the original size of the image.

[0044] In some implementations, before reconstructing the spatiotemporal domain of the pseudo-time image sequence, the method further includes: performing repeated noise suppression processing on the target image; based on the noise-suppressed target image, performing the reconstruction of the spatiotemporal domain of the pseudo-time image sequence to obtain multiple enhanced images; and averaging the multiple enhanced images at the pixel level to generate a de-degradation image for suppressing noise amplification effects.

[0045] For example, the noise suppression process of the target image is repeated multiple times, which includes adding Gaussian noise to the target image and generating a noisy image in each noise suppression process of the target image.

[0046] For example, different Gaussian noise is added to the target image each time to generate different noisy images.

[0047] For example, the number of repetitions of the noise suppression process is determined based on the noise level of the target image.

[0048] In some implementations, the target image is a color image that includes multiple color channels, such as R, G, and B color channels.

[0049] In some embodiments, the method further includes: independently performing the following steps for each color channel of the target image: constructing a pseudo-temporal image sequence; performing a spatiotemporal frequency domain transformation on the pseudo-temporal image sequence to obtain a frequency domain representation of the pseudo-temporal image sequence; performing weighted processing on the frequency domain representation using a deconvolution kernel; performing an inverse transformation on the weighted frequency domain representation to reconstruct the spatiotemporal domain of the pseudo-temporal image sequence; and aggregating the reconstructed spatiotemporal pseudo-temporal image sequence in the time dimension to obtain a de-degraded image; merging the de-degraded images obtained for each color channel to generate a color de-degraded image.

[0050] This application's embodiments, by acquiring the target image, clarify that the algorithm's input is a single degraded image, laying the foundation for data-driven processing. This eliminates the reliance on additional prior information (such as depth maps or atmospheric physical parameters), improving the method's practicality and convenience in real-world scenarios. By simulating the physical process of atmospheric scattering, a single image is expanded into a dynamic degradation sequence. This effect provides a data foundation rich in temporal evolution information for subsequent frequency domain analysis, thereby transforming the complex spatial degradation problem into a more manageable spatiotemporal evolution problem. By performing spatiotemporal frequency domain transformation on the pseudo-time image sequence and obtaining its frequency domain representation, the image sequence is mapped from the spatiotemporal domain to the frequency domain, achieving the separation and representation of image details (high frequency), contours (low frequency), and the degradation process. This creates conditions for subsequent precise weighted processing, enabling the algorithm to operate directly in the frequency dimension. Frequency domain weighted processing is performed using adaptively adjustable deconvolution kernels; parameters can be adjusted in real time according to the image's atmospheric conditions, allowing the weighted processing to specifically enhance high-frequency details attenuated by scattering while simultaneously suppressing noise. This effect directly determines the quality of the restored image, achieving adaptive enhancement for images with different degrees of degradation. By accurately mapping the frequency domain enhancement effect back to the image space, a sequence of images with improved quality is generated, completing the crucial transformation from "frequency domain processing" to "spatial domain results," ensuring that all enhancement operations are reflected in pixel values. By aggregating the optimized sequence in the temporal dimension, the restoration information of each frame in the sequence is fused, ultimately outputting a clear image, stabilizing and improving the overall quality of the restoration result, and avoiding the instability that may arise from single-frame restoration.

[0051] See attached document Figure 2 As shown, in a specific embodiment of this application, a dust and fog weather image processing method based on spatiotemporal frequency domain weighting includes the following steps S110-S190.

[0052] S110. Obtain the original degraded image. As input.

[0053] in, Represents the pixel space coordinates of the image. Indicates the RGB color channel, Separated according to channel dimension Single-channel image And perform the same processing as steps S120-S180 on the image of each channel.

[0054] S120. A pseudo-temporal image sequence is constructed using Gaussian filters of varying degrees to simulate the scattering process of light propagating in the atmosphere.

[0055] For example, let the original degraded image be... The length of the constructed pseudo-time image sequence is The sequence is denoted as Each frame Obtained through Gaussian filtering, and the size of the Gaussian kernel varies with... It increases gradually. The Gaussian kernel function uses a standard two-dimensional Gaussian distribution, and its mathematical expression is shown as follows: ; Where G() is a two-dimensional Gaussian kernel function; These are the local coordinates within the kernel centered at the origin of the kernel image, and are the... The standard deviation of the Gaussian kernel corresponding to the frame, the size of the Gaussian kernel and Relatedly, the Gaussian kernel takes odd-numbered dimensions. To ensure effective coverage. Frame Image Through the original image With Gaussian kernel The discrete convolution is calculated, and the expression is shown below:

[0056] in, It is the half-width of the convolution window. During convolution, the scale parameter of the Gaussian kernel... Frame number Increasing, that is ,in It is the minimum standard deviation. It is an incremental step size. The larger the frame number, the higher the degree of blurring, and the later it is arranged in the sequence, in order to simulate the temporal evolution of light scattering.

[0057] Preferred pseudo-time series length 50 can be taken. It can be 0. 0.17 is acceptable.

[0058] S130, Pseudo-time image sequence Perform zero-filling.

[0059] For example, suppose the original sequence array The time dimension has a length of The spatial dimension is During filling, the filling is evenly distributed at the beginning and end of the time dimension, as well as at the edges of each spatial dimension. Pixels with a circle value of 0. After filling, the expanded sequence is obtained. The time dimension becomes Spatial dimensions become ,in, , .

[0060] Preferred, 30 is acceptable.

[0061] S140, regarding the extended sequence Perform a Fourier transform in the spatiotemporal dimension to obtain the frequency domain feature representation of the sequence.

[0062] For example, the transformation frequency is defined as the time frequency. and spatial frequency The transformation process is based on the three-dimensional discrete Fourier transform, and its mathematical expression is shown as follows:

[0063] in It is the imaginary unit.

[0064] S150, Construct deconvolution kernel .

[0065] For example, the spatiotemporal grid is divided into The spatial matrix of the image column direction is obtained. The spatial matrix of the row direction Time domain matrix Its mathematical expression is shown as:

[0066]

[0067] in, These represent the subscript indices of the matrix elements. , Substitute them respectively into the index coordinates of each matrix ( The element at the specified position; The diffusion coefficient is... Adjustments are made based on the degradation level of the restored image to control the frequency domain weighting during the deconvolution process. Preferably, A value of 27 is acceptable. For example, suppose the element U is a matrix of size (10, 10, 10). When constructing the element at position (i=1, j=4, k=6) of matrix U, then... = The calculation of other elements follows the same method as the calculation of U elements. Based on this, the elements in the four matrices are constructed from scratch.

[0068] S160. Weighting the frequency domain representation using a deconvolution kernel yields the weighted sequence frequency domain features, the mathematical expression of which is shown below:

[0069] in, The weighted frequency domain features are designed to enhance high-frequency spatial components and compress time-frequency bandwidth.

[0070] S170. Perform a spatiotemporal inverse Fourier transform on the weighted frequency domain features to reconstruct the spatiotemporal domain sequence. The mathematical expression for the inverse transform is shown as follows:

[0071] in It is the imaginary unit.

[0072] S180, For each color channel Calculate the reconstructed sequence The mathematical expression for the enhanced image of this channel, obtained by averaging over the time dimension, is shown as follows:

[0073] Optional, final channel enhancement image Alternatively, one can take the earlier pseudo-interval time series. The time average of the frame image The selection of the image restoration and enhancement effect can be dynamically adjusted. Preferably, 10 is acceptable.

[0074] S190. Trim the zero-value boundaries filled in step S130 to restore the original space size. Finally, enhance the image for all channels. Merge along the channel dimension to reconstruct a multi-channel image. The final dehazing / dust-removing enhanced image is output.

[0075] In another specific embodiment of this application, in order to evaluate the effectiveness of the embodiments of this application, three common image restoration and enhancement methods were selected for comparison: histogram equalization, Retinex method, and dark channel prior. To evaluate the defect detection performance and efficiency of each algorithm, two evaluation metrics were used: visible edge ratio. and gradient enhancement ratio .

[0076] For example, visible edge ratio The definition expression is shown below:

[0077] in, and These represent the number of visible edges in the restored image and the original image, respectively. Visible edges are captured using the Candy operator with a lower threshold of 50. The size of the value reflects, to some extent, the algorithm's ability to enhance minute details in the image. A higher value indicates that the algorithm enhances the image texture details better.

[0078] For example, gradient boosting ratio The definition is as follows:

[0079] in, It is the set of visible edges in the restored image. It is a single edge of the set. This is the gradient ratio between the restored image and the original image. Gradient enhancement ratio. It reflects the gradient relationship between the pixel values ​​of the enhanced image and the original image. The higher the value, the better the algorithm enhances image contour information and image contrast.

[0080] See attached document Figure 3 As shown, the algorithm restores images under dust and fog conditions. In the fog image restoration task, compared with the image before processing, the road lane lines are clearer, and the cyclist's facial expression and cracked asphalt pavement are more visible. In the dust image restoration and enhancement task, the algorithm can restore the details of buildings obscured by sand and dust. Overall, the algorithm performs excellently in capturing image details and texture features, and can improve the distinguishability between objects in the field of view. Therefore, based on the qualitative comparison of different image restoration and enhancement methods, the algorithm proposed in this application has the highest visible edge ratio in both dust and fog scene images. (63.360 and 55.342 respectively) and gradient enhancement ratio (2.580 and 1.939 respectively).

[0081] This application's embodiments, by adjusting the diffusion coefficient parameter in the deconvolution kernel, can flexibly adapt to different concentrations of haze weather conditions, achieving varying degrees of defogging enhancement effects. This application can significantly improve the contrast of degraded images, with a marked increase in the visible edge ratio and gradient enhancement ratio before and after restoration. It is particularly effective for restoring severely degraded images, and requires no accurate input of material parameters, demonstrating strong robustness.

[0082] See attached document Figure 4 As shown, this application also discloses a dust and fog weather image processing system based on spatiotemporal frequency domain weighting, including: a target image acquisition module 410, a pseudo-time image sequence construction module 420, a spatiotemporal frequency domain transformation module 430, a weighting processing module 440, a spatiotemporal domain construction module 450, and an image aggregation module 460.

[0083] The target image acquisition module 410 is used to acquire a target image, which is an initial degraded image captured under severe weather conditions and awaiting processing.

[0084] The pseudo-time image sequence construction module 420 is used to construct a pseudo-time image sequence based on the target image by simulating the physical process of image degradation caused by atmospheric scattering. The blurriness of the pseudo-time image sequence increases with the time dimension.

[0085] The spatiotemporal frequency domain transformation module 430 is used to perform spatiotemporal frequency domain transformation on the pseudo-time image sequence and obtain the frequency domain representation of the pseudo-time image sequence.

[0086] The weighted processing module 440 is used to perform weighted processing on the frequency domain representation through a deconvolution kernel, wherein the weighted processing parameters of the deconvolution kernel are adjusted in real time according to the atmospheric conditions of the target image to enhance high-frequency details and suppress noise.

[0087] The spatiotemporal domain construction module 450 is used to perform an inverse transformation on the weighted frequency domain representation and reconstruct the spatiotemporal domain of the pseudo-time image sequence.

[0088] The image aggregation module 460 is used to aggregate the pseudo-time image sequence after reconstruction in the spatiotemporal domain along the time dimension to obtain the processed de-degradation image.

[0089] The functions of each unit in the dust and fog weather image processing system based on spatiotemporal frequency domain weighting provided in the above embodiments of this application can be implemented through the steps of the above-described dust and fog weather image processing method based on spatiotemporal frequency domain weighting. Therefore, the specific working process and beneficial effects of each unit in the dust and fog weather image processing system based on spatiotemporal frequency domain weighting provided in the embodiments of this application will not be repeated here.

[0090] This application also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for processing dust and fog weather images based on spatiotemporal frequency domain weighting.

[0091] Those skilled in the art will understand that the embodiments in this application can be provided as methods, systems, or computer program products. Therefore, the embodiments in this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the embodiments in this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of this application, and this application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this application, and these modifications and improvements are also considered to be within the scope of protection of this application.

Claims

1. A method for processing dust and fog weather images based on spatiotemporal frequency domain weighting, characterized in that, include: Acquire a target image, which is an initial degraded image captured under severe weather conditions and to be processed; Based on the target image, a pseudo-temporal image sequence is constructed by simulating the physical process of image degradation caused by atmospheric scattering. The blurriness of the pseudo-temporal image sequence increases with the time dimension. The pseudo-time image sequence is subjected to spatiotemporal frequency domain transformation to obtain the frequency domain representation of the pseudo-time image sequence; The frequency domain representation is weighted by a deconvolution kernel, wherein the weighting parameters of the deconvolution kernel are adjusted in real time according to the atmospheric conditions of the target image to enhance high-frequency details and suppress noise. The weighted frequency domain representation is inversely transformed to reconstruct the spatiotemporal domain of the pseudo-temporal image sequence; The pseudo-temporal image sequence after reconstruction of the spatiotemporal domain is aggregated along the time dimension to obtain the processed de-degradation image.

2. The method according to claim 1, characterized in that, The construction of a pseudo-temporal image sequence based on the target image, by simulating the physical process of image degradation caused by atmospheric scattering, includes: In the time dimension, by simulating the physical process of image degradation caused by atmospheric scattering, a Gaussian filter is used to filter the target image to generate the pseudo-time image sequence; The scaling parameter of the Gaussian filter increases monotonically with the increase of the frame number in the time dimension.

3. The method according to claim 1, characterized in that, The spatiotemporal frequency domain transformation of the pseudo-time image sequence includes: The pseudo-time image sequence is transformed from the spatiotemporal domain to the frequency domain using a three-dimensional discrete Fourier transform.

4. The method according to claim 1, characterized in that, The weighting process of the frequency domain representation using a deconvolution kernel includes: The deconvolution kernel is constructed based on the spatial frequency components and spatiotemporal frequency components of the image; The amplitude of the deconvolution kernel is positively correlated with the modulus of the spatial frequency and associated with the temporal frequency component, and is used to reverse atmospheric scattering in the frequency domain.

5. The method according to claim 1, characterized in that, The method further includes: Before performing spatiotemporal frequency domain transformation on the pseudo-time image sequence, the pseudo-time image sequence is zero-padding. After reconstructing the spatiotemporal domain of the pseudo-temporal image sequence, the boundary regions introduced by zero-padding are cropped.

6. The method according to claim 1, characterized in that, Before reconstructing the spatiotemporal domain of the pseudo-temporal image sequence, the following steps are also included: The target image is subjected to repeated noise suppression processing multiple times; Based on the target image after noise suppression processing, the spatiotemporal domain of the reconstructed pseudo-temporal image sequence is performed to obtain multiple enhanced images; The average value of the multiple enhanced images is taken at the pixel level to generate a de-degradation image, which is used to suppress the noise amplification effect.

7. The method according to claim 6, characterized in that, The repeated noise suppression processing of the target image includes: In each noise suppression process performed on the target image, Gaussian noise is added to the target image and a noisy image is generated; In this process, different Gaussian noise is added to the target image each time, so as to generate different noisy images.

8. The method according to claim 6, characterized in that, The number of repetitions of the noise suppression process is determined based on the noise level of the target image.

9. The method according to claim 1, wherein the target image is a color image including multiple color channels, characterized in that, The method further includes: The steps of constructing a pseudo-temporal image sequence, performing a spatiotemporal frequency domain transformation on the pseudo-temporal image sequence to obtain its frequency domain representation, weighting the frequency domain representation using a deconvolution kernel, performing an inverse transformation on the weighted frequency domain representation to reconstruct the spatiotemporal domain of the pseudo-temporal image sequence, and aggregating the reconstructed spatiotemporal pseudo-temporal image sequence in the time dimension to obtain a de-degraded image are performed independently on each color channel of the target image. The de-degraded images obtained from each color channel are merged to generate a color de-degraded image.

10. A dust and fog weather image processing system based on spatiotemporal frequency domain weighting, characterized in that, include: The target image acquisition module is used to acquire a target image, which is an initial degraded image captured under severe weather conditions and to be processed. A pseudo-temporal image sequence construction module is used to construct a pseudo-temporal image sequence based on the target image by simulating the physical process of image degradation caused by atmospheric scattering. The blurriness of the pseudo-temporal image sequence increases with the time dimension. The spatiotemporal frequency domain transformation module is used to perform spatiotemporal frequency domain transformation on the pseudo-time image sequence and obtain the frequency domain representation of the pseudo-time image sequence; The weighted processing module is used to perform weighted processing on the frequency domain representation using a deconvolution kernel, wherein the weighted processing parameters of the deconvolution kernel are adjusted in real time according to the atmospheric conditions of the target image to enhance high-frequency details and suppress noise. The spatiotemporal domain construction module is used to perform an inverse transformation on the weighted frequency domain representation and reconstruct the spatiotemporal domain of the pseudo-time image sequence; The image aggregation module is used to aggregate the pseudo-temporal image sequence after reconstruction in the spatiotemporal domain along the time dimension to obtain the processed de-degradation image.

Images