A method and system for defogging a continuous-time image

Abstract

The application relates to a continuous-time image defogging method and system, and belongs to the technical field of video image processing. The method comprises the following steps: acquiring a continuous-time foggy image sequence, and generating image sequence samples after dynamic sliding window division; inputting the samples into a pre-trained continuous-time image defogging model, which comprises a convolution feature extraction network, a convolution liquid time constant module, a physical parameter prediction head and an image reconstruction module; extracting deep features through the convolution feature extraction network; performing continuous-time dynamic modeling through the convolution liquid time constant module to output enhanced features; predicting a transmittance map and an atmospheric light value through the physical parameter prediction head; and reconstructing a defogged image sequence based on the transmittance map, the atmospheric light value and an imaging physical model through the image reconstruction module. The application realizes continuous-time modeling of fog dynamics, combines with a physical model constraint, and significantly improves the time sequence consistency, physical interpretability and image quality of image defogging.

Description

Technical Field

[0001] This invention relates to the field of video image processing technology, and in particular to a method and system for dehazing continuous time images. Background Technology

[0002] Currently, image dehazing techniques are mainly divided into two categories: single-image dehazing and video sequence dehazing. Among these, video sequence dehazing presents significantly greater challenges than single-image dehazing. Due to the dynamic changes of fog in time and space, the fog density and distribution in a video sequence may fluctuate over time. Unlike the relatively uniform fog distribution in static images, factors such as fog density variations between video frames, fog layer movement, and illumination disturbances mean that video dehazing algorithms not only need to process spatial information but also capture temporal continuity to maintain consistency across frames.

[0003] From an algorithmic perspective, temporal redundancy is a key issue in video dehazing. Although consecutive frames have high similarity, how to effectively utilize this temporal information without significantly increasing computational cost remains a research challenge. Traditional fixed-window methods have limitations in video processing: if the window is too small, it cannot obtain sufficient temporal context, while if the window is too large, it will increase computational complexity and may introduce irrelevant information from distant frames, thereby reducing dehazing performance. Currently, research on single-image dehazing mainly falls into two categories: (1) methods based on physical models. These methods rely on atmospheric scattering models and estimate transmittance. With global atmospheric light Then, a clear image is reconstructed. For example, Zhang et al. (H. Zhang, V. Sindagi, and VM Patel, “Joint transmission map estimation and dehazing using deep networks,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no.7, pp. 1975–1986, Jul. 2019) proposed an end-to-end network that jointly estimates transmittance and atmospheric light, achieving coupling between physical parameters and neural network learning; Ullah et al. (H. Ullah, K. Muhammad, M. Irfan, et al., “Light-DehazeNet: A novel lightweight CNN architecture for single image dehazing,” IEEE Trans. Image Process., vol. 30, pp. 8968–8982, Dec. 2021) designed a lightweight atmospheric scattering model network, improving dehazing performance while maintaining physical interpretability. However, these methods are mainly for single images and are not suitable for dynamic modeling of continuous frame video sequences. (2) Pure data-driven methods based on deep learning. Lu et al. (LP Lu, Q. Xiong, B. Xu, et al., “Mixdehazenet: Mix structureblock for image dehazing network,” Proc. Int. Joint Conf. Neural Netw. (IJCNN), 2024, pp. 1–10) proposed a multi-scale parallel large kernel convolutional and attention fusion network to enhance feature extraction and contextual information modeling; Xu et al. (J. Xu, X. Hu, L. Zhu, et al., “Video dehazing via a multi-range temporal alignment network with physical prior,” Proc. IEEE / CVFConf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 18053–18062) used a multi-range scene radiometric restoration module to capture the complex spatiotemporal dependencies between video frames.Furthermore, Fan et al. (J. Fan, J. Weng, K. Wang, et al., “Driving-video dehazing with nonaligned regularization for safety assistance,” Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit.(CVPR), 2024, pp. 26109–26119) proposed a two-stage video dehazing framework based on multi-frame spatial alignment and information fusion to improve cross-frame consistency. However, such methods are prone to gradient vanishing or exploding problems during long sequence training, resulting in insufficient utilization of information from distant frames.

[0004] In recent years, networks based on neural ordinary differential equations (ODEs) and continuous-time modeling have been introduced into the field of video processing. Park et al. (S. Park, K. Kim, J. Lee, et al., “Vid-ODE: Continuous-time video generation with neural ordinary differential equation,” Proc. AAAIConf. Artif. Intell., vol. 35, no. 3, pp. 2412–2422, 2021) proposed combining neural ordinary differential equations with convolutional recurrent units (ODE-ConvGRU) to model the continuous-time dynamics of video. Hasani et al. (R. Hasani, M. Lechner, A. Amini, et al., “Liquid time-constant networks,” Proc.AAAI Conf. Artif. Intell., vol. 35, no. 9, pp. 7657–7666, 2021) proposed the Liquid Time-Constant (LTC) network, which, by introducing a learnable time constant, achieves stable continuous dynamic modeling under irregular sampling and performs excellently in temporal prediction tasks. Chahine et al. (M. Chahine, R. Hasani, P. Kao, et al., “Robust flight navigation out of distribution with liquid neural networks,” Sci. Robot., vol. 8, no. 77, pp. eadc8892, Jul.2023) further verified the robustness of LTC networks in complex spatiotemporal vision tasks, laying the foundation for their application in video visual analysis. Meanwhile, to balance accuracy and computational efficiency, scholars have proposed a dynamic sampling and optical flow-guided aggregation mechanism.Yoon et al. (J. Yoon and MK Choi, “Exploring video frame redundancies for efficient data sampling and annotation in instance segmentation,” Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2023, pp. 3308–3317) reduced redundant frame computation through an adaptive sampling strategy; Fan et al. (J. Fan, J. Weng, K. Wang, et al., “Driving-video dehazing with nonaligned regularization for safety assistance,” Proc. IEEE / CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 26109–26119) designed a sliding window reference matching mechanism for video dehazing in unaligned supervised scenes; Zhu et al. (X. Zhu, Y. Wang, J. Dai, et al., “Flow-guided feature aggregation for video object detection,” Proc. IEEE Int. Conf. Comput. Vis.) (ICCV), 2017, pp. 408–417) utilizes an optical flow-guided feature aggregation strategy to maintain temporal consistency. Although the above methods have achieved good results in dynamic scenes, they still have insufficient adaptability for videos with long time spans.

[0005] Despite significant progress in existing image and video dehazing technologies, there are still obvious shortcomings in terms of process mechanisms and performance.

[0006] First, physical methods based on atmospheric scattering models rely on accurate estimations of physical parameters such as transmittance and atmospheric light. Since atmospheric light is often non-uniformly distributed in space and time, the global consistency assumptions of the model are difficult to satisfy in complex real-world scenarios, leading to residual haze and brightness distortion in the reconstructed image. Furthermore, these methods are sensitive to imaging conditions; if the input image is poorly lit or contains highly reflective areas, errors are easily accumulated during the estimation process, affecting the stability of the dehazing effect.

[0007] Secondly, while end-to-end data-driven methods outperform traditional physical models in feature representation, their training process heavily relies on large-scale labeled datasets. Limited by the difficulty of acquiring real-world foggy video data, these methods often train on synthetic data, resulting in insufficient generalization ability in real-world natural scenes. Furthermore, these methods typically ignore imaging physical constraints, easily leading to problems such as color deviation, over-enhancement, or texture artifacts, and their internal decision-making mechanisms are difficult to explain.

[0008] While existing continuous-time modeling methods have shown certain advantages in video generation tasks, their computational complexity is high, and they face a trade-off between numerical stability and inference efficiency when processing large-scale image sequences. Although Liquid Time Constant (LTC) networks possess strong temporal modeling capabilities, current research has not effectively incorporated them into industrial vision or dynamic dehazing tasks, and they lack collaborative optimization designs with adaptive sampling mechanisms. Therefore, existing video dehazing techniques still face bottlenecks in spatiotemporal consistency modeling, dynamic adaptation capabilities, physical interpretability, and computational efficiency. A new image dehazing method that can achieve stable, efficient, and physically consistent image dehazing within a continuous-time framework is needed to improve the model's robustness and generalization performance in real-world dynamic environments. Summary of the Invention

[0009] To address the technical problems of poor temporal consistency, poor dynamic adaptability, weak physical interpretability, and low computational efficiency in existing video dehazing methods, this invention proposes a method and system capable of stable dehazing of continuous-time images within a continuous time frame. This method is used to restore the clarity of continuous-time image sequences affected by fog under complex weather conditions, thereby improving image quality, temporal consistency, and visual interpretability, while also taking into account physical interpretability and dynamic adaptability. It can be widely applied in fields such as intelligent transportation, industrial visual inspection, autonomous driving, and video surveillance.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A method for dehazing continuous-time images includes the following steps:

[0012] Step 1: Obtain a continuous sequence of foggy images and divide the foggy image sequence into dynamic sliding windows to generate image sequence samples;

[0013] Step 2: Input the image sequence samples into a pre-trained continuous-time image dehazing model, which includes a convolutional feature extraction network, a convolutional liquid time constant module, a physical parameter prediction head, and an image reconstruction module;

[0014] Step 3: Extract the depth features of each image frame using the convolutional feature extraction network to obtain the feature sequence;

[0015] Step 4: Perform continuous-time dynamic modeling on the feature sequence using the convolutional liquid time constant module, update the hidden state, and output enhanced features with temporal context information;

[0016] Step 5: Based on the enhanced features, the physical parameter prediction head predicts the transmittance map and atmospheric light value for each image frame;

[0017] Step 6: Using the image reconstruction module, based on the transmittance map, the atmospheric light value, and the imaging physical model, reconstruct the dehazed image and output the dehazed image sequence.

[0018] The present invention also provides a dehazing system for continuous time images, comprising:

[0019] The data acquisition and preprocessing module is used to acquire a continuous time sequence of fogged images, and to dynamically divide the fogged image sequence into a sliding window to generate image sequence samples. The image sequence samples are then input into a pre-trained continuous time image dehazing model, which includes a convolutional feature extraction network, a convolutional liquid time constant module, a physical parameter prediction head, and an image reconstruction module.

[0020] The feature extraction module is used to extract the depth features of each image frame through the convolutional feature extraction network to obtain a feature sequence;

[0021] The continuous-time dynamic modeling module is used to perform continuous-time dynamic modeling on the feature sequence through the convolutional liquid time constant module, update the hidden state, and output enhanced features with temporal context information.

[0022] A physical parameter estimation module is used to predict the transmittance map and atmospheric light value of each image frame based on the enhanced features using the physical parameter prediction head.

[0023] The dehazing image reconstruction and output module is used to reconstruct dehazing images and output a dehazing image sequence based on the transmittance map, the atmospheric light value, and the imaging physical model through the image reconstruction module.

[0024] Compared with the prior art, the beneficial effects of the present invention include:

[0025] (1) This invention introduces convolutional liquid time constant network into the continuous time image dehazing task for the first time. Through continuous time dynamic modeling, it can stably describe the changes of fog over time, fundamentally solving the problem of difficulty in modeling the time-varying nature of fog in continuous time image sequences.

[0026] (2) This invention proposes an interpretable dehazing framework based on atmospheric scattering physical constraints, which explicitly predicts transmittance and atmospheric light parameters, and introduces an imaging physical model in the reconstruction stage to achieve physical consistency constraints, thereby making the network interpretable.

[0027] (3) The present invention adopts a dynamic sliding window sampling and time masking mechanism to solve the problems of video frame length not being fixed and temporal redundancy, and significantly reduces computational overhead while maintaining temporal consistency;

[0028] (4) The present invention constructs a dual-path dehazing reconstruction structure to achieve joint optimization of data-driven decoding and physical reconstruction, taking into account both image visual quality and physical consistency, and improving the robustness of the results under different climatic conditions;

[0029] (5) This invention also proposes several constraint loss functions, which enhance the temporal consistency and physical rationality of the output results, and the model has good numerical stability, making it suitable for real-time deployment in industrial scenarios. Attached Figure Description

[0030] Figure 1 This is a flowchart of the dehazing method for continuous-time images according to an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram illustrating the principle of the dehazing method for continuous-time images according to an embodiment of the present invention.

[0032] Figure 3 This is a flowchart of the ConvLTC module in an embodiment of the present invention;

[0033] Figure 4 This is a schematic diagram of the dynamic sliding window division in an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of physical constraint reprojection in an embodiment of the present invention. Detailed Implementation

[0035] This invention aims to provide an efficient continuous-time image dehazing method that integrates physical constraints and continuous-time dynamic modeling, achieving the following objectives:

[0036] (1) Stability modeling and prediction of fog concentration changes over time;

[0037] (2) Joint optimization of physical consistency and visual continuity between video frames;

[0038] (3) While maintaining high-efficiency reasoning performance, improve the interpretability and reliability of the algorithm in practical applications.

[0039] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and preferred embodiments.

[0040] See Figures 1-2 This embodiment provides a method for dehazing continuous-time images, which includes the following steps:

[0041] Step 1: Data input and preprocessing.

[0042] Acquire a series of foggy images over consecutive time periods The sequence of foggy images In this embodiment, the input image can be a grayscale image with a resolution of [resolution missing] per frame. To improve training stability, the image pixel intensity is normalized to... Interval.

[0043] To address the issues of variable video frame length and temporal redundancy, this step employs a dynamic sliding window partitioning strategy. Specifically, it calculates the structural similarity (SSIM) and optical flow intensity between adjacent image frames. SSIM measures the structural similarity between two images, while optical flow intensity characterizes pixel-level motion information.

[0044] Based on the calculated SSIM and optical flow intensity, consecutive image sequences are adaptively divided into a group of image sequence samples. The principle of division is that when the scene change (measured by SSIM and optical flow intensity) between consecutive frames is small, they are grouped into the same sample; when the change exceeds a preset threshold, a new sample is opened, ultimately forming several image sequence samples. The number of image frames in each sample is not fixed; this is called a dynamic sliding window. The number of image frames contained in each sample... Within the preset range Inside, for example .

[0045] For each image sequence sample, generate a corresponding effective mask matrix. The mask matrix is ​​used to identify which frames and pixels within a sample are valid (e.g., located within image boundaries or not excluded during preprocessing), thus ignoring invalid local regions within the frame in subsequent calculations and preventing them from affecting model training.

[0046] This invention employs a dynamic sliding window sampling and temporal masking mechanism to address the issues of variable video frame length and temporal redundancy. Based on the structural similarity and optical flow variations of adjacent frames, this invention adaptively generates variable-length temporal windows and controls the participation of effective frames in the computation through a masking matrix, achieving efficient modeling of long videos. This mechanism significantly reduces computational overhead while maintaining temporal consistency, improving the real-time performance and scalability of the model in industrial vision and intelligent monitoring.

[0047] Step 2: Input the image sequence samples into the pre-trained continuous-time image dehazing model, which includes a convolutional feature extraction network, a convolutional liquid time constant module, a physical parameter prediction head, and an image reconstruction module.

[0048] Step 3: Convolutional Feature Extraction. A convolutional feature extraction network is used to extract the depth features of each image frame, resulting in a feature sequence.

[0049] Specifically, each frame of the image sequence sample generated in step 1 is sequentially input into the convolutional feature extraction network. This network consists of three stacked convolutional blocks: the first convolutional block is... ,in, Indicates that the convolution kernel is , The number of zero-padding layers is 1; the second convolutional block is... The third convolutional block is After passing through this network, each frame of the image... Transformed into the corresponding feature tensor ,in Represents the space of real numbers. This represents the number of feature channels, i.e., the number of feature maps output by the convolutional layer. Indicates the height of the feature map. This represents the width of the feature map, and The features of all images are combined to form a feature sequence. .

[0050] Step 4: Continuous-time dynamic modeling. The feature sequence is modeled continuously over time using the Convolutional Liquid Time Constant (ConvLTC) module, the hidden states are updated, and enhanced features with temporal context information are output.

[0051] Specifically, the feature sequence obtained in step 3 The input is fed into the ConvLTC module. The ConvLTC module combines the traditional LTC network with a convolutional structure, enabling continuous-time dynamic modeling of sequence features.

[0052] The ConvLTC module maintains an internal hidden state. Its evolution process is described by the following continuous-time differential equation:

[0053]

[0054] in, The learnable time constant, For activation function, For bias terms, For residual terms; Use tanh or ReLU activation functions; As input features, by Obtained by sampling; It is in a hidden state; Learnable convolutional kernels; For deviation; This indicates a convolution operation.

[0055] In discrete time, the Euler approximation method is used to solve the differential equation to update the hidden state:

[0056]

[0057] in, and This represents the iteration variable. In order to be in The hidden state at a given moment, In order to be in The hidden state at a given moment, The length of time variation, This is an adjustment factor for the iteration step size. For the time is Auxiliary input data (such as some features) at that time.

[0058] This invention is the first to introduce Convolutional Liquid Time Constant Network (ConvLTC) into video dehazing tasks, enabling continuous-time dynamic modeling. This invention deeply integrates LTC units with a two-dimensional convolutional structure to construct a temporal visual model with continuous-time inference capabilities. This design overcomes the limitations of traditional RNNs and 3D convolutions, which only model in the discrete time domain, enabling the model to stably describe the dynamic changes of fog at arbitrary sampling rates, fundamentally solving the problem of difficult time-varying fog modeling in video sequences.

[0059] Step 5: Parameter Learning and Estimation. Based on enhanced features, the physical parameter prediction head predicts the physical parameters of each image frame, including the transmittance map and atmospheric light values.

[0060] Specifically, parameter prediction is performed on the hidden state output by the ConvLTC module using two parallel convolutional heads:

[0061] (1) Transmittance Head: It consists of one or more convolutional layers. Output the predicted transmittance map ;

[0062] (2) Atmospheric light head: First, the enhanced features are compressed into a vector through a global average pooling layer, then passed through one or more fully connected layers, and finally the output value is constrained to the range [0, 1] by the Sigmoid activation function, and the final output is the predicted atmospheric light value. .

[0063] This invention proposes an interpretable dehazing framework based on atmospheric scattering physics constraints, explicitly predicting transmittance and atmospheric light parameters. The invention designs a "dual-branch physical parameter header" in the ConvLTC output to simultaneously regress the transmittance map and atmospheric light distribution, and explicitly introduces imaging physics equations during the reconstruction stage, achieving physically consistent reprojection constraints. This structure organically combines traditional atmospheric scattering models with a deep learning end-to-end optimization framework, enabling the network to possess interpretability and physical reducibility, avoiding the black-box problem.

[0064] Step 6: Image Reconstruction. The image reconstruction module reconstructs the dehazed image based on the transmittance map, atmospheric light values, and the imaging physics model, outputting a dehazed image sequence. This step utilizes the physical parameters estimated in Step 5, combined with the imaging physics model and the neural network decoder, to jointly reconstruct the dehazed image.

[0065] Specifically, the formula for the imaging physics model is:

[0066]

[0067] in, For foggy images, For a clear image, Transmittance, It is atmospheric light.

[0068] First, based on the above imaging physics model, the dehazed image based on the imaging physics model can be derived, expressed by the formula:

[0069]

[0070] in, The dehazed image is derived by reverse engineering the imaging physics model; For deviation, adjust Make the dehazed image more realistic; In the above imaging physical model The predicted value, i.e., the predicted value of transmittance; This represents the predicted atmospheric light value.

[0071] Secondly, the enhanced features output by the ConvLTC module are input into a decoder network. This decoder network can adopt a U-Net structure and generate a data-driven dehazed image through operations such as upsampling, convolution, and ReLU activation function. .

[0072] Finally, the dehazed image derived from the imaging physics model will be... and data-driven dehazed images Adaptive fusion is performed to obtain the final reconstructed dehazed image:

[0073]

[0074] in, For the dehazed image generated by the decoder, For adaptive fusion coefficients.

[0075] This invention constructs a dual-path dehazing reconstruction structure to achieve joint optimization of "data-driven decoding + physical reconstruction". During the decoding stage, this invention generates reconstruction results based on neural networks and reconstruction results based on physical models in parallel, and uses adaptive fusion coefficients... This mechanism achieves a dynamic balance between the two. It takes into account both image visual quality and physical consistency, avoids overfitting or distortion of a single network, and improves the robustness of results under different climatic conditions.

[0076] Furthermore, to train the continuous-time image dehazing model, a total loss function is constructed that includes integrated image loss, physical reprojection constraints, temporal transmittance smoothing, spatial total variational regularization, and atmospheric light temporal constraints. Its formula is:

[0077]

[0078] in, These are the loss functions for image reconstruction by the decoder, image reconstruction by the physical model, transmissivity temporal smoothing, transmissivity spatial total variation regularization, and atmospheric light temporal smoothing loss, respectively. These are the weights corresponding to the losses.

[0079] This invention proposes multiple constraint loss functions to enhance temporal consistency and physical consistency. The total loss function incorporates multiple constraints, including image structure similarity loss (MS-SSIM), physical reprojection error, transmissivity temporal smoothing regularization, spatial total variational regularization, and atmospheric light temporal consistency constraints, ensuring that the dehazing results output by the model are visually consistent and physically self-consistent.

[0080] To balance the emphasis on different objectives in the early and later stages of model training, this invention employs a course-based learning strategy to train the loss function. For example, in the first third of training rounds, the focus is on... leading( ), mid-term Later Gradually strengthen physical consistency.

[0081] The following section, using the self-built dataset HazeSeqDynamic of this invention, details the specific implementation steps of the continuous-time image dehazing model during the training phase.

[0082] Step 1: Constructing the Dataset. To more accurately evaluate the temporal dehazing effect of the present invention's dehazing method in real-world environments, this invention uses two large-format cameras to collect fogged and clear images in foggy environments, constructing a self-built dataset, denoted as HazeSeqDynamic. Unlike existing datasets that primarily provide discrete fog levels or synthetic fog, the HazeSeqDynamic dataset can capture continuous changes in fog density between adjacent frames. The HazeSeqDynamic dataset includes a training dataset and a test dataset, where the training dataset contains a total of 5188 fogged and clear images, and the test dataset contains 486 fogged and clear images. After inputting the dataset, dynamic sliding window preprocessing is first performed based on the SSIM and optical flow changes between adjacent frames, such as... Figure 4 As shown, the division principle is to divide consecutive frames with small changes in image targets and small trends in fog concentration into one sample, and the maximum sample size of each sample is 20, and the minimum sample size is 3. Figure 4 This is a schematic diagram of dynamic sliding window partitioning. The diagram shows how consecutive raw video frames are adaptively divided into samples of unequal lengths (e.g., sample 1 contains 5 frames, sample 2 contains 3 frames, ..., sample n contains 4 frames) based on inter-frame SSIM and optical flow intensity, thereby avoiding the problems of information redundancy or insufficient context caused by fixed windows.

[0083] Step Two: Refer to Figure 3 This step involves performing three convolutional layers on the samples to extract and encode depth features. Figure 3 The processing flow of the ConvLTC module in this embodiment of the invention is shown. The input feature sequence passes through the ConvLTC module in sequence, and the hidden state inside it evolves continuously in time according to the aforementioned differential equation, thereby capturing the dynamic information between frames and outputting the enhanced features.

[0084] Step 3: Refer to Figure 3 The generated feature sequences are input into the ConvLTC module for learning, to learn the transmittance coefficients. and atmospheric light coefficient During the learning process, because the ConvLTC module is used, the coefficient value mapped by each coefficient (transmittance coefficient, atmospheric light coefficient) changes continuously over time as the sample time progresses, thus reflecting the characteristics of fog concentration changes. Furthermore, end-to-end learning is also performed during this process; the network simultaneously learns from both foggy image sequences and clear image sequences to capture the mapping relationship between them.

[0085] Step 4: To better implement physical constraints, such as Figure 5As shown, this step fuses the image obtained by reverse engineering the imaging physics model with the end-to-end image. During training, this step uses a learning course strategy: first, a higher proportion (weight) is assigned to the end-to-end image, and then the proportion of the image obtained by reverse engineering the imaging physics model is gradually adjusted to progressively enhance physical consistency.

[0086] Step 5: Decode the information obtained after fusion in Step 4 to finally generate a clear dehazed image sequence.

[0087] The training phase of a continuous-time image dehazing model mainly includes the following processes (with... The maximum window size is (For example)

[0088] (1) Read consecutive frames →Dynamic sliding window partitioning→Assembling tensors With the effective mask matrix (excluding invalid local regions within the frame). ;

[0089] (2) The encoder extracts features frame by frame, obtaining ;

[0090] (3) The ConvLTC module updates the hidden state along the time dimension. ;

[0091] (4) Decoder output Physical head calculation ;

[0092] (5) Based on effective mask matrix Aggregate the loss for valid frames (excluding invalid frames) and update via backpropagation. And all network parameters. This step first filters out valid frames (excluding invalid frames), and then generates a valid mask matrix for each valid frame. (Excluding invalid local areas within the frame), the final dehazing process is only performed on the valid areas of the valid frames.

[0093] During the inference phase, for a new input continuous-time foggy image sequence, the method of this invention can process it in real time according to the following process:

[0094] (1) Real-time streaming of video frames, and packaging them according to the dynamic sliding window method in step 1;

[0095] (2) Input the packaged image sequence samples into the trained continuous-time image dehazing model, perform one forward propagation, and obtain... and ;

[0096] (3) Based on and After image reconstruction, a dehazed image sequence is output. Simultaneously, the continuous-time image dehazing model can also output intermediate predicted transmittance maps. and atmospheric light value As an interpretable signal for fog density and atmospheric light estimation, it provides a basis for decision-making in downstream industrial systems.

[0097] This invention also provides a dehazing system for continuous-time images, which implements the continuous-time image dehazing method described in the foregoing method embodiments. The system includes the following modules:

[0098] The data acquisition and preprocessing module acquires continuous-time hazy image sequences, dynamically divides these sequences into sliding windows to generate image sequence samples, and inputs these samples into a pre-trained continuous-time image dehazing model. This model includes a convolutional feature extraction network, a convolutional liquid time constant module, a physical parameter prediction head, and an image reconstruction module. Specifically, this module calculates the structural similarity and optical flow intensity between adjacent image frames, adaptively dividing the image sequence into multiple image sequence samples based on these parameters. The number of image frames within each image sequence sample varies but remains within a preset range. A corresponding effective mask matrix is ​​generated for each image sequence sample. This module then inputs the processed image sequence samples into the pre-trained continuous-time image dehazing model.

[0099] The feature extraction module is used to extract the depth features of each image frame through a convolutional feature extraction network to obtain a feature sequence. In one specific embodiment, the convolutional feature extraction network consists of three convolutional blocks stacked sequentially. The specific parameter configuration of each convolutional block is described in the aforementioned method embodiment and will not be repeated here.

[0100] The continuous-time dynamic modeling module is used to perform continuous-time dynamic modeling of the feature sequence through the convolutional liquid time constant module, update the hidden state, and output enhanced features with temporal context information. The evolution of the hidden state of the convolutional liquid time constant module follows the continuous-time differential equation given in the aforementioned method embodiments, and the Euler approximation method is used for hidden state updating in discrete time.

[0101] The physical parameter estimation module is used to predict the transmittance map and atmospheric light value for each image frame based on enhanced features using a physical parameter prediction head. The physical parameter prediction head includes a transmittance head and an atmospheric light head. The transmittance head consists of at least one convolutional layer and is used to output a predicted transmittance map with the same resolution as the input image. The atmospheric light head consists of a global average pooling layer, a fully connected layer, and a sigmoid activation function, and is used to output the predicted atmospheric light value.

[0102] The dehazing image reconstruction and output module is used to reconstruct dehazing images based on transmittance maps, atmospheric light values, and imaging physics models, and output a dehazing image sequence. Specifically, this module reverse-engineers the dehazing image based on the imaging physics model, processes the enhancement features through a decoder network to generate data-driven dehazing images, adaptively fuses the two to obtain the final reconstructed dehazing image, and combines the dehazing images of all frames into a dehazing image sequence for output.

[0103] The collaborative workflow of the above modules has been described in detail in the method embodiments and will not be repeated here. Those skilled in the art should understand that the above module division is only a specific implementation of the present invention. In practical applications, the modules can be merged or split as needed, and all such actions fall within the protection scope of the present invention.

[0104] The present invention proposes a method and system for dehazing continuous-time images of dynamic scenes, which overcomes the following shortcomings of existing image and video dehazing technologies:

[0105] Existing physical methods based on atmospheric scattering models assume uniform fog distribution in space and time, making it difficult to handle the non-uniformity and dynamics of fog layers in real-world environments. This results in unstable brightness, blurred edges, and residual fog in the reconstructed images. This invention aims to improve the accuracy of reconstructing dynamic video sequences by introducing a continuous-time modeling mechanism to achieve a smooth description of the time-varying characteristics of fog layers.

[0106] While traditional end-to-end learning models can automatically extract image features, they lack physical interpretability and temporal constraints, making it difficult to guarantee consistency across frames. To address this issue, this invention introduces a Liquid Time-Constant (LTC) unit structure. By embedding a learnable time constant within the neural network, it enables the modeling of continuous-time dynamic processes, enhancing the network's stability and generalization ability in long video sequences.

[0107] Existing video dehazing algorithms generally employ fixed time windows or static frame alignment, resulting in high computational costs and a lack of adaptive selection of key frames. This invention combines a dynamic sampling strategy with a physical prior guidance mechanism to achieve frame-level adaptive feature aggregation, effectively reducing redundant information and improving computational efficiency.

[0108] Some continuous-time models suffer from numerical instability and inference latency in video generation and dehazing tasks. This invention constructs a convolution-based liquid time constant network (ConvLTC) that introduces a convolutional feature extraction module while maintaining temporal continuity, thus balancing spatiotemporal feature representation and computational efficiency. This approach is suitable for industrial vision and dynamic video de-sharpening tasks under complex climatic conditions.

[0109] Compared with existing technologies, the continuous-time video dehazing method based on Convolutional Liquid Time Constant (ConvLTC) networks proposed in this invention achieves significant improvements in image restoration quality, temporal consistency, computational efficiency, and physical interpretability. Specific effects are as follows:

[0110] I. Significantly improved defogging performance and visual consistency:

[0111] Tests were conducted on continuous video sequences with time-varying and spatially unevenly distributed fog density, and the results showed that:

[0112] (1) The method of the present invention can effectively suppress the time fluctuation of fog, maintain natural brightness and contrast during the recovery process, and the output result is more visually realistic;

[0113] (2) Compared with various representative benchmark methods, the dehazing results generated by the present invention have the best performance in terms of detail structure, global brightness and scene naturalness;

[0114] (3) Compared with real and clear images, the results obtained by the present invention are almost identical in terms of brightness, color balance and texture level, showing excellent visual realism.

[0115] Quantitative evaluation results show that the present invention outperforms existing algorithms in all four main metrics:

[0116] (1) Peak signal-to-noise ratio (PSNR) improved by an average of 2.5-4.0 dB;

[0117] (2) Structural similarity (SSIM) improved by an average of 1.4%;

[0118] (3) The average improvement in multi-scale structural similarity (MS-SSIM) is approximately 9.0%;

[0119] (4) Perceptual similarity (LPIPS) decreased by 67.1%, indicating a significant improvement in perceptual quality and structural fidelity.

[0120] These results demonstrate that the dehazed images obtained by this invention are closer to the real scene and have significant advantages in terms of visual naturalness and perceptual consistency.

[0121] II. Enhanced Physical Consistency and Interpretability:

[0122] Unlike traditional black-box end-to-end networks, this invention explicitly predicts transmittance. With atmospheric light Two physical parameters are used, and reprojection constraints are applied through the atmospheric scattering imaging equation.

[0123] Experimental results show that artifacts in the restored image are significantly reduced in the illumination transition region, and brightness changes are smoother. Furthermore, in the experiment removing physical reprojection constraints, PSNR and SSIM both decreased by approximately 3%, while LPIPS increased by approximately 20%, indicating that physical consistency helps avoid overexposure artifacts.

[0124] Therefore, this invention not only achieves high-quality visual reconstruction, but also provides parameter interpretations corresponding to physical processes, providing quantifiable evidence for video environment analysis, monitoring, and industrial inspection.

[0125] III. Advantages in computational efficiency and energy consumption:

[0126] By introducing a dynamic sliding time window and a masking mechanism, this invention significantly improves computational efficiency: at a 240×320 grayscale video resolution, the real-time inference speed can reach 64.57 frames per second (FPS). This is achieved by removing the dynamic sliding window mechanism (fixed...). In the experimental results, all indicators decreased by an average of about 2%, indicating that dynamic time sampling improved the temporal stability.

[0127] This demonstrates that the present invention significantly improves energy efficiency and real-time performance while ensuring accuracy, making it suitable for autonomous driving, industrial vision inspection, and video surveillance systems.

[0128] IV. Stability and Robustness of Long Sequences:

[0129] The ConvLTC module employs a softplus time constant constraint and an Euler approximation discrete update strategy, maintaining numerical stability under long sequence inputs without gradient explosion or convergence oscillations.

[0130] Experiments show that, under irregular sampling rates and sudden changes in illumination, the model still maintains high stability and consistent dehazing effect, proving its robust modeling capability under continuous time conditions.

[0131] V. Overall Effects and Engineering Application Value:

[0132] This invention exhibits excellent numerical stability and engineering deployability. The ConvLTC module employs a learnable time constant with softplus activation and Euler approximation for discrete updates, resulting in numerical stability and real-time operation on embedded GPUs or edge AI platforms. It achieves real-time inference performance, making it suitable for deployment in industrial inspection and intelligent transportation scenarios. The complete ConvLTC framework of this invention achieves an optimal balance between dehazing performance, temporal consistency, visual quality, and physical interpretability.

[0133] Meanwhile, compared with existing technologies, this invention achieves a reasonable balance between structural fidelity and perceptual realism; the output results are naturally bright and have distinct layers, without excessive enhancement or abnormal exposure; since the model is trained based on grayscale, it avoids color drift problems and focuses on the restoration of structure and texture; combining physical priors and dynamic modeling, it has high versatility and deployment stability.

[0134] This invention can achieve physically consistent, temporally stable, and perceptually realistic video defogging effects in complex climatic environments, and is applicable to multiple scenarios such as intelligent transportation, industrial visual inspection, autonomous driving, and security monitoring.

[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A method for dehazing continuous-time images, characterized in that, Includes the following steps: Step 1: Obtain a continuous sequence of foggy images and divide the foggy image sequence into dynamic sliding windows to generate image sequence samples; Step 2: Input the image sequence samples into a pre-trained continuous-time image dehazing model, which includes a convolutional feature extraction network, a convolutional liquid time constant module, a physical parameter prediction head, and an image reconstruction module; Step 3: Extract the depth features of each image frame using the convolutional feature extraction network to obtain the feature sequence; Step 4: Perform continuous-time dynamic modeling on the feature sequence using the convolutional liquid time constant module, update the hidden state, and output enhanced features with temporal context information; Step 5: Based on the enhanced features, the physical parameter prediction head predicts the transmittance map and atmospheric light value for each image frame; Step 6: Using the image reconstruction module, based on the transmittance map, the atmospheric light value, and the imaging physical model, reconstruct the dehazed image and output the dehazed image sequence.

2. The method for dehazing continuous-time images according to claim 1, characterized in that, The process of dynamically dividing the foggy image sequence into sliding windows to generate image sequence samples includes: Calculate the structural similarity and optical flow intensity between adjacent image frames; Based on the structural similarity and optical flow intensity, the image sequence is adaptively divided into multiple image sequence samples. The number of image frames contained in each image sequence sample varies, and the number of frames is within a preset range. Generate a corresponding valid mask matrix for each image sequence sample.

3. The method for dehazing a continuous-time image according to claim 1, characterized in that, The convolutional feature extraction network consists of three convolutional blocks stacked sequentially. The first convolutional block includes a convolutional layer with 1 input channel, 48 output channels, a kernel size of 3×3, 1 zero-padding layer, a batch normalization layer, a ReLU activation function layer, and a max pooling layer with a stride of 2. The second convolutional block includes a convolutional layer with 48 input channels, 96 output channels, a kernel size of 3×3, and 1 zero-padding layer, a batch normalization layer, a ReLU activation function layer, and a max pooling layer with a stride of 2. The third convolutional block includes a convolutional layer with 96 input channels, 192 output channels, a kernel size of 3×3, and 1 zero-padding layer, a batch normalization layer, a ReLU activation function layer, and a max pooling layer with a stride of 2.

4. The method for dehazing a continuous-time image according to claim 1, characterized in that, The hidden state evolution of the convolutional liquid time constant module follows the following continuous-time differential equation: in, The learnable time constant, For activation function, For bias terms, For residual terms; Use tanh or ReLU activation functions; As input features, by Obtained by sampling; It is in a hidden state; Learnable convolutional kernels; For deviation; This indicates a convolution operation.

5. The method for dehazing a continuous-time image according to claim 4, characterized in that, The convolutional liquid time constant module uses the Euler approximation method for hidden state updates in discrete time, and the update formula is: in, In order to be in The hidden state at any given moment; In order to be in The hidden state at any given moment; The length of time that changes; This is an adjustment factor for the iteration step size; For the time is Auxiliary input data at the time.

6. The method for dehazing a continuous-time image according to claim 1, characterized in that, The physical parameter prediction head includes: A transmittance head, consisting of at least one convolutional layer, is used to output a transmittance map with the same resolution as the input image. The atmospheric light head consists of a global average pooling layer, a fully connected layer, and a sigmoid activation function, and is used to output atmospheric light values.

7. The method for dehazing a continuous-time image according to claim 1, characterized in that, Step 6 includes the following steps: based on the imaging physical model, using the formula Derivation of dehazed images based on imaging physics model ,in, For foggy images, For the predicted transmittance, For the predicted atmospheric light value, For deviation; The enhanced features are processed by a decoder network to generate a data-driven dehazed image. ; Dehaze image and dehazed images Adaptive fusion is performed to obtain the reconstructed dehazed image. The formula is as follows: in, .

8. The method for dehazing a continuous-time image according to claim 1, characterized in that, The total loss function during training of the continuous-time image dehazing model is a combination of the loss function of the decoder reconstructing the image, the loss function of the physical model reconstructing the image, the loss function of transmissivity temporal smoothing, the transmissivity spatial total variation regularization, and the atmospheric light temporal smoothing loss.

9. The method for dehazing a continuous-time image according to claim 1, characterized in that, The continuous-time image dehazing model is trained using a course-based learning strategy, and the data-driven dehazing images are first processed. Set a higher weight.

10. A dehazing system for continuous-time images, characterized in that, include: The data acquisition and preprocessing module is used to acquire a continuous time sequence of fogged images, and to dynamically divide the fogged image sequence into a sliding window to generate image sequence samples. The image sequence samples are then input into a pre-trained continuous time image dehazing model, which includes a convolutional feature extraction network, a convolutional liquid time constant module, a physical parameter prediction head, and an image reconstruction module. The feature extraction module is used to extract the depth features of each image frame through the convolutional feature extraction network to obtain a feature sequence; The continuous-time dynamic modeling module is used to perform continuous-time dynamic modeling on the feature sequence through the convolutional liquid time constant module, update the hidden state, and output enhanced features with temporal context information. A physical parameter estimation module is used to predict the transmittance map and atmospheric light value of each image frame based on the enhanced features using the physical parameter prediction head. The dehazing image reconstruction and output module is used to reconstruct dehazing images and output a dehazing image sequence based on the transmittance map, the atmospheric light value, and the imaging physical model through the image reconstruction module.

Images